2.1. Overview

2.2. Basel Requirements

Basel II specifies three means of assessing operational risk capital requirements (BCBS, 2006a, pages 644–683, “The First Pillar – Minimum Capital Requirements”):

• ∙ Basic Indicator Approach (BIA) where capital is based on 15% gross income averaged over the past 3 years;

• ∙ The Standardised Approach (TSA) where capital is based on 12%–18% gross income depending on the line of business; and

• ∙ AMA where operational risk capital is based on a bank’s own model of operational risk.

Use of both the TSA and AMA are conditional on meeting quantitative and qualitative requirements for regulatory approval, although the burden is significantly higher for AMA. Amongst other things this requires the AMA model to combine ILD and ELD, scenario analysis and BEICFs.Footnote ⁵ There is a need for a minimum of 5 yearsFootnote ⁶ of relevant ILD for the calculation of operational risk capital, which should satisfy a 99.9% 1-year confidence level.Footnote ⁷ Loss data should be split by material business line, risk type and by gross loss amounts and recoveries, with “boundary losses” relating to overlaps between credit and market risk losses also identified. ELD must be used where relevant, though the banks needs to develop criteria for how ELD should be used and any adjustments made (e.g. scaling to a bank’s size). Scenario analysis should be used along with BEICFs to provide a forward-looking view of exposure.Footnote ⁸ While AMA models can allow for risk mitigation, the benefit of insurance is capped at 20% of capital, while there are strict criteria for allowing for risk mitigation in operational risk calculations. The overall framework needs to be validated by internal and external auditors as part of the third line of defence.

The requirements of Basel II were elaborated in Operational Risk – Supervisory Guidelines for the Advanced Measurement Approaches (BCBS196) (BCBS, 2011a). This builds on 2009 BCBS research into operational losses and AMA practices in banks (BCBS, 2009a, 2009b). Amongst other things, the paper specifies criteria for gross loss definition, loss data thresholds, grouping of losses, loss event dates to be used in modelling, correlation and dependency assumptions, and scenario analysis. As well as criteria for recording losses, it also suggests that data be collected on “near-miss” events where no loss crystallised or events which result in gains (e.g. for trading loss events where markets move in the bank’s favour). Data should be governed by a data policy while the guidelines set out extensive requirements for the third-line validation and verification of AMA models and data.

Before the issue of the guidelines, the UK FSA also had an Operational Risk Standing Group,Footnote ⁹ which looked at the implementation of Basel II operational risk requirements. This group published a number of papers which are noteworthy in terms of the regulatory expectations set out in the papers.

Alongside the AMA guidelines, the BCBS also published Principles for the Sound Management of Operational Risk in July 2011 (BCBS, 2011b), updating a previous principles paper from 2003. While more concerned with BAU operational risk management, it does cover loss data, scenarios and BEICFs in the form of key risk indicators (KRIs) and key performance indicators. Of particular interest is the follow-up review of the principles published by the BCBS in October 2014 (BCBS, Reference Berger2014a), which expands on the principles and lists noteworthy practices in operational risk loss collection and scenario analysis in appendices III and IV.

In addition to review of the principles, the BCBS also launched a consultation on revisions to standardised approaches (BIA and TSA) in October 2014 which proposed to increase the associated capital requirements significantly (BCBS, Reference Cope and Antonini2014b).Footnote ¹⁰ While the business line distinction under TSA may be dispensed with, this could still add to the complexity around standardised approach inputs and calculations. At the time of writing, the BCBS has also said it will consult on dispensing with AMA altogether,Footnote ¹¹ though it may extend loss data collection and other AMA requirements to non-AMA firms.

2.3. Solvency II Requirements

The Solvency II directive and associated Level 2 text do not delve into great detail about operational risk beyond the specification of the SF operational risk requirement. However, implicit in internal model regulations is the need to ensure that inputs to internal operational risk models are complete, accurate and relevant. This is worthy of a separate paper in itself but suffice to say operational risk inputs to internal models need to meet very high standards to meet with regulatory approval. As noted, even those using SF will need to demonstrate the appropriateness of this requirement to their operational risk profile, which will need to be supported by loss data and scenario analysis.

More specific to operational risk are the requirements of EIOPA’s Systems of Governance (SoG) guidelines which were issued in September 2013 and which most EU regulators have adopted with effect from 1 January 2014 (i.e. 2 years before the main implementation date for Solvency II).Footnote ¹² Inter alia, this sets out the requirement to collect operational loss data and conduct scenario analysis, with the former to include near misses as well as actual losses.

2.4. Prudential Regulatory Authority (PRA) ICA and Systems and Controls (SYSC) Requirements

The PRAFootnote ¹³ sets out ICA requirements for insurers in the section 7 of the Prudential Sourcebook for Insurers (INSPRU) standards in its Prudential Sourcebook (PSB). While requiring all risks to be considered as part of the ICA as well as consideration of scenario analysis, this does not explicitly cover operational risk. However, the FSA expanded on its expectations of ICA operational risk coverage in a series of insurance sector briefings:

• ∙ November 2005 (FSA, 2005, pages 4.14–29) – operational risk measurement was listed as a key area of development, noting that there was a lack of loss data and hence a reliance on scenario analysis.

• ∙ November 2006 (FSA, Reference Finke, Singh and Rachev2006, pages 3.62–63) – this again noted the absence of credible historic loss data and the greater need for management judgement in assessing risk exposures.

• ∙ October 2007 (FSA, Reference Ieva, Paganoni and Ziller2007) – the FSA noted that operational risk was one of the main reasons it had applied a capital add-on to insurers assessments, and set out its expectations including inter alia the need for robust challenge and validation of operational risk assessments and dependency assumptions.

As well as ICAS, the PRA also set out requirements for the management of operational risk in sections 13 and 14.1 of the SYSC part of the PSB, with further requirements for insurers set out in 5.1 of the INSPRU sourcebook. These require consideration of internal and external losses, the state of risk controls and scenario analysis.

2.5. FRC Standards

The FRC issues both generic TASs as well as standards specific to different lines of work. In general, TASs will be targeted at work that is commonly (but not necessarily exclusively) performed by actuaries in the United Kingdom, with distinction made between work reserved for actuaries and required work which is a legal obligation but which may be performed by non-actuaries. While actuaries are bound by their profession to comply with TASs, the standards are a useful guide for non-actuaries in modelling operational risk, setting out detailed lists of points to consider in modelling. Further details of these and other Actuarial profession requirements can be found in Appendix B.

4.1. Operational Risk Categorisation

A key element of the operational risk framework will be a detailed operational risk categorisation system. Operational risk is a heterogeneous category covering all risks associated with people, processes and systems. This will cover risks as diverse as systematic processing errors, cyber crime, health and safety breaches and product literature failings to name but a few of the risks typically covered. There is a need to break operational risk down into more homogenous categories to enable exposures to be properly understood and modelled. This is dependent, however, on a categorisation system that is robust with little scope for ambiguity in terms of how losses and risks should be categorised.

Firms will have their own systems for categorising operational risks but in terms of generic categorisation systems, Basel II identified seven high Level 1 categories of operational risk and 20 Level 2 categories sitting underneath this (BCBS, 2006b). A more detailed yet still freely available categorisation system is that of the Institute and Faculty of Actuaries’ (IFoA’s) Risk Classification Working Party (Kelliher et al., Reference Moosa and Li2013) which starts from the Basel II Level 2 categories and identifies some 350 Level 3 sub-types of operational risk.Footnote ¹⁴

Typically, operational risk is modelled for each Level 2 category (or equivalent level in the firm’s own categorisation system). The Basel Level 1 categories might be insufficiently granular, while there is likely to be insufficient loss data to model each Level 3 sub-type, and running scenario analysis exercises for each Level 3 would be impractical.

Whatever categorisation system is used should be as detailed as possible to reduce the scope for ambiguity in categorisation. Taking the Basel Level 2 categories as an example, there could be confusion as to whether cyber theft of assets should come under the (external fraud) theft and fraud or the system security categories. This in turn could lead to losses being reported in one category when they are meant to be classed and modelled under another. To use a general insurance analogy, this would be akin to classifying flood losses as fire, say, compromising the modelling of each peril.

Similarly, a common scenario analysis failing is where the same risk is considered under two separate categories due to confusion as to which category it should come under, or worse, that the risk is missed altogether.

The IFoA Risk Classification Working Party categorisation did not just refine the Level 2 categories further with Level 3 sub-categories, it also identified areas where these categories could overlap and suggested demarcation lines. Crucially, this extended not just between operational risks but between operational and non-operational risks as well. There can be significant overlaps – for instance, dealing error losses will depend on market movements.

Basel II gives extensive guidance on the demarcation between operational and other risks for banks, particularly regarding “boundary” losses overlapping between credit and operational risks. However, the demarcation for insurance risk is less clear. For instance, failure to follow underwriting guidelines may be viewed as a processing error but this will affect claims experience and reinsurance recoveries. Similarly, while non-disclosure of underwriting information may be viewed as a form of fraud, the impact of this will be implicit in claims experience and so may be implicitly covered under insurance risk.

Considerable confusion can also arise between operational and strategy risks. There is no standard definition of strategy risk like there is for operational risk under Basel II and Solvency II. The IFoA Risk Classification Working Party defined strategy risk in terms of risks to goodwill made up of the value of new business and value generated by future back-book initiatives, with operational risk covering risks affecting balance sheet and embedded values. Generally, goodwill is excluded from regulatory capital resources and so strategy risks affecting this should be similarly excluded from capital requirements (see section 5.1.1 below). Whatever definition of strategy risk is used, it is important that the distinction between this and operational risk and what should and should not be included in economic capital modelling, is clearly understood.

To conclude, whatever the organisation, there is a need for the categorisation system to identify the overlaps between categories and to have a clear view as to whether risks should be covered under operational risk or some other category.

5.1. Operational Risk Events and Losses

Just as it is important to have clarity as to which category an operational risk event falls under, it is equally important that loss data collected captures all impacts relevant to operational risk capital modelling. This includes cash outlays such as

• ∙ customer compensation costs – this should generally include ex gratia payments as these are typically offered to settle complaints and avoid further costs being incurred;

• ∙ other compensation costs including that payable to shareholders, bondholders, staff and members of the general public in respect of operational events;

• ∙ regulatory fines including Ombudsman costs;

• ∙ legal fees and other additional legal costs associated with operational events – this should include the cost of successfully defending legal action if this cannot be recovered, it should also include the costs associated with shareholder lawsuits;

• ∙ additional consultancy costs required to investigate and remediate problems including the cost of section 166 and other reviews carried out at the behest of regulators;

• ∙ IT contractor and other external resources brought in to help rectify problems;

• ∙ property damage – the cost of repairs under leasehold agreements and/or the impairment of property assets owned, plus the cost of replacement accommodation;

• ∙ mailing and other additional customer communication costs;

• ∙ overtime, temporary staff and other unbudgeted staff costs arising as a result of the operational risk event; and

• ∙ other marginal, unbudgeted costs arising as a result of the event.

Capital modelling will generally consider movements in net equity (own funds in the case of Solvency II). Therefore it is important to consider not just cash outlays but other movements in assets and liabilities arising from an operational risk event such as

• ∙ provisions required to be set up, e.g., higher guarantee and option costs arising due to flawed documentation and/or adverse legal rulings;

• ∙ re-statements of existing liability values due to valuation errors or deliberate under-provisioning;

• ∙ asset values being written off due to valuation errors, issues with asset title or other operational event; and

• ∙ loss of tax credits.

For life insurers subject to Solvency II, the present value of future profits may count towards own funds. Therefore there is a need to consider operational risk events which may compromise such future profits. This would include restrictions in charges that can be levied due to flaws in contract wording or regulatory caps imposed. It would also encompass higher future costs arising, e.g., increased disclosure requirements on pension policies.Footnote ¹⁵

Similarly, general insurance technical provisions may reflect expected profits implicit in unearned or bound but not incepted premiums (EPIFP). Operational risk impacts which reduce the profits emerging on existing business and EPIFP should be captured in loss data.

5.2. ILD Collection

Having determined the ILD to be collected, firms need to ensure processes and procedures are in place to capture these data. This should tie in with wider operational risk management and incident reporting procedures. The BCBS recommends that banks should develop data policies and procedures around ILD, including reporting thresholds, loss dates and grouping of losses (BCBS, 2011a, page 20).

Typically, first-line business units will be tasked with recording and reporting on any operational risks that come to light. Detailed instructions should be provided to those charged with recording internal losses to ensure

• ∙ losses are correctly categorised under the correct category;

• ∙ there is clarity as to which impacts should or should not be captured, consistent with section 5.1 above; and

• ∙ insurance and other recoveries (as well as WPF chargebacks) are separately recorded and not netted off, while gains and near misses are also captured.

Loss reporting should capture details on what gave rise to the event including any controls which may have failed. As well as informing BAU operational risk management and helping prevent further recurrences, this information can help in causal analysis of losses and identifying common dependencies between risks.

Note that in designing the loss reporting process, care should be given to the burden this places on first-line staff. If this is overly burdensome, if first-line staff are not engaged with the process, and/or if they do understand the requirements, then the quality of loss data will suffer.

Ideally, there would be a seamless process for the reporting of losses which is intuitive and easy to complete. This should be supported by training on the importance of accurate loss reporting, including the need to report on gains and “near misses”, supported by succinct yet comprehensive guidance on the loss elements from section 5.1 above. Reporting should ideally be done online as opposed to by paper to prevent loss data being lost and to make it easier to collate and interrogate data.

This process should also tie in with the incident reporting process with first-line staff encouraged to report issues before they crystallise as a loss, gain or “near miss”. Loss reporting should be an extension of incident reporting rather than a separate process. Finally, internal audit should check whether all incidents and losses are being reported in a timely and complete manner. There should be sanctions in place for those who do not do so.

5.3. Limitations of ILD

ILD is objective and is specific to the firm’s operational risk profile. However, there are a number of limitations with such data. First, most organisations have only been systematically collecting operational risk data for 15 years or less. This is unlikely to be sufficient to model low frequency, high impact operational risks faced by firms. To use a general insurance analogy, it would be akin to modelling UK windstorm damage using data only going back to a given point (e.g. the year 2000) when this may exclude a large loss (e.g. the great storm of 1987) in the data set.

Even for the period for which data has been collected, it may not capture all losses that have arisen due to the lag that exists between loss occurrence and emergence for many operational risks. For instance, a firm may already have been infiltrated by cyber criminals but the breach has still to be detected. Looking back to 2010, e.g., banks’ loss data sets would have not included the large-scale losses which have emerged since in respect of LIBOR fixing even though this was prevalent before 2010.

Another issue is that even for reported losses, ILD may be incomplete as the loss has still to be fully settled. As the example of PPI has shown, provisions set aside may prove inadequate, and so there is a need to separately consider how existing loss estimates may vary over time – the BCBS AMA guidelines suggest this could be considered as part of scenario analysis (BCBS, 2011a, page 135).

Finally, a key issue with ILD is that it is historical in nature as opposed to forward looking. It will not capture new exposures (e.g. risks associated with the segregation of client funds for a new wrap proposition) while some historic data may be irrelevant due to changes in controls (e.g. improved fraud controls) or to changes in the underlying exposure (e.g. mortgage endowment mis-selling losses may no longer be relevant due to time bars).

To conclude, ILD is unlikely to be adequate on its own to model operational risk. There is a need to consider both larger data sets (i.e. ELD) and to take a prospective view of operational risks through scenario analysis and consideration of BEICFs.

6.1. Sources of ELD

There are a number of sources of external operational loss data. One is the Operational Risk eXchange Association (ORX), which was founded in 2002 for financial services firms to share details of operational loss events on a confidential basis.Footnote ²⁶ It sets standards for operational risk categorisation and loss data which helps ensure consistency in submissions and facilitate ease of comparison. In terms of size, it currently has 85 member firms – mostly banks and asset managers but now including insurers – and processes 15,000 loss events per quarter. As of 30 June 2015, its global banking database had over 450,000 loss events with a total value of €273 billion.

For insurers, another source of external data is that provided by ORIC International.Footnote ²⁷ Similar to ORX, it was founded in 2005 for insurers to share information on operational losses, and sets standards to ensure consistent classification and recording of losses. Originally, set up for UK insurers under the auspices of the ABI, it is expanding internationally and now has 40 members.

Both ORX and ORIC require participating firms to share their own losses in order to participate in the wider database. Some firms may prefer not to share their own losses, which would preclude them accessing confidential loss sharing databases. Such firms could still access databases which collate publicly available details on operational losses. One such source would be IBM Algo First which has details of over 13,000 loss events.Footnote ²⁸

6.2. Collecting and Adjusting ELD

There are three key challenges to using external data in operational risk models. The first is ensuring consistency between the external data and the firm’s model. External data should be capable of being mapped to the risk categories used by the firm to model risk.

It should separate out insurance and other recoveries, and capture the same loss elements as internal date (see sections 5.1 and 5.1.1 above). Ideally, it should also capture gains and “near misses”.

The second challenge will be scaling external losses to reflect the size of the firm. Ideally, external loss details would include details of firm size to allow losses to be scaled, e.g., for employee relations, there would be details of headcount and/or payroll of the firm that suffered the loss to enable the loss to be scaled to one’s own employee totals. In practice, to preserve anonymity of contributing firms, this information may not be available.

An acceptable alternative may be if the ELD provider scaled losses to standard sizes which firms could then adjust according to their own size. However, there is no generally accepted way of scaling external data.

The third challenge is ensuring that the loss data reflects any changes in controls or in the business environment since the losses occurred. Typically, operational controls are strengthened after an event and hence the frequency and/or severity could be very different after the event than before. Complicating matters, while a firm may have sight of changes in its own control framework and business environment, it is unlikely to have such detail on its peers.

As for internal losses, external data should be subject to a review and challenge process to ensure: that it is properly mapped to the firm’s own categories, that relevant impacts have been captured and that recoveries have been separated out. Note the amount of external losses may be far greater than internal losses so this review process needs to be proportionate. For instance, there may be large volumes of low severity card fraud losses in bank external data, so it might be appropriate to only review a sample.

The review process should also consider the relevance of losses for a firm. For instance, LIBOR fixing by an investment bank may not be relevant to a small building society, while unit pricing errors may not be relevant to a life insurer which only writes non-linked business. However, some insurance losses may be relevant to banks and vice versa – external losses should not be automatically discarded just because they arise in a different industry.

6.3. Limitations of ELD

External data will not be suitable for modelling operational risk: if the data cannot be mapped to a firm’s own risk categories, if it does not cover the loss impacts assessed as part of internal data, if it does not split out insurance and other recoveries, if different firms use different loss data thresholds and/or if external data is not scalable. It may still be useful, however, for informing scenario analysis and aiding BAU operational risk management in terms of learning from other organisations’ mistakes.

Even if external data can be used in modelling, it is worth noting that it shares a lot of the weaknesses of ILD. Even though it covers a greater number of firms, this does not guarantee that it will capture low frequency risks. What often happens with high impact operational risks is that they hit multiple companies around the same time rather than being spread out over a prolonged period.

So, for instance, banks were hit around the same time with PPI mis-selling and LIBOR fines, while pensions mis-selling crystallised for life insurance companies at the same time as part of an industry-wide review mandated by the regulator.

This is not to say that a high impact loss cannot crystallise elsewhere before it affects one’s own firm, but extending data to other companies may still miss out on low frequency, high impact risks. Using the general insurance analogy from section 5.3 above, extending windstorm claims data to cover other companies back to say 2005 still misses out the 1987 event.

Like ILD there are also issues with the relevance of historic ELD. This is exacerbated by the multiple companies involved who will be adjusting their product mix, controls, risk appetite and operational risk profile over time. As noted, while we may have sight of changes in profile in one’s own firm, changes in other companies’ profiles may be obscure.

In summary, even if external data could be incorporated into modelling, there will still be an issue with historic data not capturing low frequency, high impact risks though it would capture a greater range of loss events. Like ILD, there will still be issues around the relevance of historic data to the firm, with the added question as to whether historic losses in other firms are relevant.

8.1. Preparation

Preparation is essential for scenario analysis and involves identifying key SMEs, ensuring they are properly briefed on requirements, and providing supporting information to help them with the scenario assessment.

As wide a range as possible of SMEs with relevant experience should contribute to scenario analysis. When considering scenarios by risk type, there is a need to think widely about which functions could be affected by sub-types of a risk. Taking processing risks, e.g., a firm should consult not just customer service SMEs but also those involved in payroll processing and asset dealing.

Once SMEs have been identified, they need to be properly briefed on requirements. They should understand the importance of scenario analysis to capital modelling and financial strength, as well as the wider management of operational risks. Detailed instructions should be providing covering what impacts need to be considered and also what should not be considered (see sections 5.1 and 5.1.1. above). Where scenario analysis is carried out by risk type, the underlying (Level 3) sub-types should be supplied so SMEs understand what risks come under the category to be considered – a common failing in scenario analysis is where the same risk is considered under two separate categories, or where a material risk is missed because SMEs assume it is covered under a different category. Where scenario analysis is carried out by function, every effort should be made to identify potential risks faced by that function, going beyond what the function may have on its risk register. Supplying this information on risks to be considered helps ensure no material risks are omitted from scenario analysis.

SMEs should also be supplied with background material to help them with the exercise, including

• ∙ ILD, including gains and near misses;

• ∙ ELD – consideration should be given not just to losses from external loss databases such as ORX and ORIC, but also any other relevant examples outside financial services, e.g., recent theft of data from Target in the United States;

• ∙ details of current risks and issues;

• ∙ result of control assessments and KRI values; and

• ∙ strategy details which may affect risk profile, e.g., new products and channels which may affect conduct risks, or new systems which may affect processing errors.

8.2. Assessment

There are a number of ways scenario analysis can be conducted. Some conduct a series of 1:1 interview with SMEs, others carry out workshops with SMEs, while others elicit views through survey. A variation of the survey approach is the Delphi technique where follow-up surveys are issued based on survey responses until a consensus emerges. While more difficult to arrange and manage, workshops can lead to useful interaction between SMEs.

Whichever approach is adopted, a key challenge is to avoid biases in what is a subjective exercise. One such bias is the “anchoring” of scenarios on past experience. This extends not just to past events, but to the results of past scenario analysis, and there may be a bias towards scenarios already identified as opposed to new scenarios. SMEs should be encouraged to think of new risks that could crystallise. Another problem is that SMEs may be too focussed on future control failures and may not properly consider legacy failings which could emerge.

SMEs may also be reluctant to identify large losses for fear this would reflect negatively on them. This may be particularly true of losses that exceed minimum capital requirements such as Basel II TSA or Solvency II SF requirements for operational risk. Facilitators should ensure that the SMEs can speak freely without fear of recrimination, and that scenario analysis should focus on understanding the risks faced rather than producing any particular capital figure. This ties in with a broader risk culture of openness and willingness to acknowledge and learn from mistakes, which should be promoted by the Board and senior management.

Note some firms provide SMEs with estimates of the operational risk capital arising from their scenario outputs. This can help guard against ad hoc loss estimates and outputs giving rise to unfeasibly large capital figures. However, the Working Party is not convinced of the merits of this, as there is a risk that SMEs may “game” their estimates to produce capital figures they feel comfortable with, rather than assess potential losses in a neutral manner. It would probably be better to validate outputs against such capital estimates as part of the review and challenge of results.

There is an issue with the quality of ad hoc loss estimates and other parameters derived as part of the scenario process. Rather than trying to finalise figures as part of a workshop or interview, time should be built in to the scenario analysis process to firm up loss estimates, perhaps in consultation with Finance and Actuarial. This time can also be used for further investigation of issues raised as part of the scenario analysis. There may be a need for follow-up workshops, interviews, etc., which should be allowed for in planning the exercise.

As for ILD, scenario analysis impacts need to split out insurance and other recoveries from gross losses. There may be a problem of bias towards scenarios involving uninsured risks, away from insured risks which may be more significant in gross terms. Consideration should also be given to pension scheme risks and impacts.Footnote ³¹

8.3. Documentation

It is essential that points raised during the assessment process are captured and documented. This should include not just the scenarios ultimately chosen but also those risks and scenarios considered but rejected. This can help demonstrate to regulators the breadth of coverage of the scenario analysis exercise.

Ideally, the scenario discussion would address all risks in a category/function and assess which are material before selecting a subset as the representative scenarios to be quantified. If this discussion were captured and documented, it would provide evidence that all risks have been considered as part of the scenario process.

Documentation should also capture the impacts by type and the rationale for frequency parameters and loss parameters chosen. Scenario analysis results should be collated in a standard template which can help ensure that no item is missed and aid comparison and validation of results. This should be updated as loss estimates are refined and issues identified are followed up, along with an audit trail of adjustments to results.

Finally, the minutes of the scenario workshops should also be kept, to show who was present and to demonstrate that there was effective challenge during the workshop.

8.4. Review, Challenge and Governance

Scenario analysis results will be inherently subjective so should be subject to extensive review and challenge. BCBS AMA guidelines suggest this review and challenge may be carried out by the third-line Internal Audit function and/or an external reviewer. This may be because of the role of the second-line risk management function in facilitating scenario analysis. However, the Working Party believes there may be a role for risk management to review and challenge scenario analysis provided those reviewing and challenging the results are sufficiently independent. Involving Risk Management in the review of operational risk scenario analysis results also helps to feed these results into emerging risk analysis, reverse stress testing and wider risk management.

Aside from independence, the other key attribute of the review function is knowledge of operational risks, which they should use to challenge whether all material risks have been considered. They should also consider whether the scenario has been properly categorised. Loss estimates and other parameters need to be challenged and validated, which may include estimating operational risk capital arising from scenario outputs as described in section 8.2 above, but which should consider other information sources, e.g., regulatory fines could be assessed against historic fines while market movements could be validated against economic capital models of market risk.

Having being through review and challenge, scenario results should be reviewed and approved by senior executives and ideally the Board or its risk committee. One approach would be to assign final responsibility for scenario results to the executive director responsible for the business area most closely aligned to managing category of risk, who would then be required to present to his peers and the Board.

Requiring senior management and the Board to endorse scenario analysis results in this way helps to ensure focus on the quality of results while also eliciting the perspective of those at the top of the organisation which can prove invaluable. It will also bring key operational risk exposures to their attention and may prompt consideration of mitigation strategies.

10.1. Insurance

As well as the amount of cover, which is a function of sum insured and policy excesses, there is also a need to consider

• a. Term of policy and any corporate policy on rolling over cover – without the latter the benefit of cover may need to be pro-rated by outstanding term.

• b. Counterparty default risk of the insurer and hence claim payment credit ratings and potential recoveries on default.

• c. Gaps in cover, e.g., policy may cover fire damage but not damage from civil unrest, or policies which only cover claims made and not claims IBNR in the policy year.

• d. Linked to (c), there may be ambiguity over whether a particular loss is covered and that the insurer may refuse to pay the claim.Footnote ³⁴

With regard to (c) and (d), in theory a firm could look to split frequency assessment into the frequency of insured and uninsured events and simulate recoveries accordingly. In practice, this may be too complex and either a subjective “haircut” may be applied or no credit allowed for insurance at all.

10.2. Other Risk Mitigation

Another source of recoveries relates to third-party indemnification, e.g., by outsourcers in respect of administration errors, but the modelling of such recoveries needs to consider

• ∙ the terms of the outsource agreement and what exactly can be recovered under the indemnity;

• ∙ any weaknesses in wording which could lead to dispute over indemnification;

• ∙ the financial strength of the outsourcer and whether they would be able to honour indemnities; and

• ∙ the process for identifying errors and recovering under indemnities – the greater the delays, the greater the exposure to outsourcer default and there may be time bars on indemnification, e.g., the outsourcer may only indemnify losses which have arisen in the past 5 years.

Operational risk modelling could also assume recoveries through legal action but consideration needs to be given to whether such action would be successful – a legal opinion would probably be required to support this – and whether the counterparty to the litigation would have the means to redress even if successful.

Lastly, insurance firms may model operational losses charged to the WPF and hence to policyholders. However, there is a need to consider whether losses charged to the WPF are consistent with its Principles and Practices of Financial Management (PPFM) and also with Treating Customers Fairly (TCF) principles and other FCA rules. Assumptions as to which losses could be charged to the WPF should be reviewed and approved by the With-Profit Actuary and the With-Profit Committee.

11.1. Correlations Derived from Data

Correlation assumptions could be derived for internal and ELD, e.g., by considering correlations between quarterly loss totals for each category (Cope & Antonini, Reference de Jongh, de Wet, Raubenheimer and Venter2008). However, loss data and hence empirical correlations may be driven mainly by low impact, high frequency losses. These may give a misleading picture of correlations at the tail.

For instance, there may be little correlation between low impact, high frequency losses under two categories (e.g. manual processing errors and card fraud) which may be correlated at the tail (e.g. weak IT system implementation leads to systemic processing errors and also exposes a firm to large-scale cyber theft). Alternatively, while there may be strong correlations between low impact, high frequency losses in two categories (e.g. weakness in customer service recruitment leads to manual processing errors and petty theft), there may be less connection between high impact, low frequency events (e.g. large-scale system processing errors and deliberate under-reserving).

Another issue with correlations derived from data is that even if there is a strong underlying correlation between two low frequency risks, the empirical correlation may still be low due to the infrequent occurrences of loss events. For instance, if we were to simulate five operational risks {X1, … , X5}, each distributed binomially with p=0.1 probability of a loss and assuming a Gaussian copula with a medium 50% correlation between variables, the empirical correlation between the number of losses would be ≈25%.

11.2. The Role of Expert Judgement

Given the limitations of correlations derived from data, the Working Party believes that expert judgement is essential to determine correlation assumptions. However it would probably be impractical to ask SMEs to identify correlations for each pair of risks. If one had 20 Level 2 risks, this would require 190 separate correlation assumptions which would be too much to ask for and difficult to validate. The resulting correlation matrix is also unlikely to satisfy the positive semi-definite property required to be valid.

To address this, one approach may be to group risks and assume a common correlation between groups of risks. For instance, splitting 20 Level 2 risks into four groups of five, we would need ten correlation assumptions within each group (40 in total) plus six correlation assumptions between groups, which may be more manageable. If one assumed a uniform correlation within groups the task of setting correlations becomes easier still.

Risks could be grouped by some common factor, for instance, by

• ∙ high-level risk type, e.g., Basel Level 1 categories;

• ∙ type of risk, e.g., people, process, system or external event; and

• ∙ function, e.g., conduct risks mapped to sales, processing risks to operations, people-related risks to Human Resources (HR), etc.

Given the heterogeneity of operational risk categories, however, it is likely there will be a loss of granularity, e.g., two sub-types of risks in two otherwise uncorrelated groups could be strongly correlated. (It should be noted that this applies not just to groups but also within Level 2 categories given the broad range of sub-risks in each category.)

Another approach to determining correlation may be to consider the results of broader stress and scenario testing work, e.g., flu pandemic scenario testing could highlight common dependencies between processing risks (due to backlogs arising) and mis-selling (due to falling markets giving rise to customer losses). This could also help with setting correlations between operational risks and market and other non-operational risks.

Whichever approach is adopted, expert judgements on correlation assumptions should be subject to rigorous review and challenge given their subjectivity. Key correlation assumptions should be identified through sensitivity analysis which should be subject to particular scrutiny. As for scenario analysis, this review should be independent and performed by those with an understanding of operational risks and how these interlink.