Discrimination is the taboo of our times. We often feel that it is unfair for a person’s treatment to be determined by personal characteristics which have historically been used in oppressive ways, which are not perfectly predictive and which we do not choose. This sentiment is vaguely specified and inconsistently applied, but it is also pervasive, and now deeply embedded in our law and culture.
The insurance industry has traditionally claimed – and to a large extent, received – a degree of exemption from this sentiment. Insurance pricing is largely based on statistical discrimination, one of the practices which anti-discrimination laws often seek to restrain. So when first introduced, these laws typically included comprehensive exemptions for insurers. Over the years, some of these exemptions have been whittled away: many countries now limit insurers’ use of genetic test results, and in Europe they can no longer charge different prices to men and women. Actuarial and economic orthodoxy generally derides such restrictions, and predicts that unfortunate consequences will flow from them.
A succinct statement of this orthodoxy is given in the policy document Insurance & superannuation risk classification policy published by the Institute of Actuaries of Australia, which explains:
In the absence of a system that allows for distinguishing by price between individuals with different risk profiles, insurers would provide an insurance or annuity product at a subsidy to some while overcharging others. In an open market, basic economics dictates that individuals with low risk relative to price would conclude that the product is overpriced and thus reduce or possibly forgo their insurance. Those individuals with a high level of risk relative to price would view the price as attractive and therefore retain or increase their insurance. As a result the average cost of the insurance would increase, thus pushing prices up. Then, individuals with lower loss potential would continue to leave the marketplace, contributing to a further price spiral. Eventually the majority of consumers, or the majority of providers of insurance, would withdraw from the marketplace and the remaining products would become financially unsound.Footnote 1
The pattern of people with lower risks buying less insurance and people with higher risks buying more insurance is called ‘adverse selection’. This is said to lead to a rise in insurance prices and fall in numbers insured, followed by a further rise in prices and fall in numbers insured, and so on; the sequence is often labelled as an ‘adverse selection spiral’ or ‘death spiral’. The lurid vocabulary reflects the fact that in the typical usage, adverse selection is not a quantified phenomenon, but rather a persuasive fable.
This fable has the seductive property of trivial sophistication: it is sufficiently simple for non-specialists to understand, and yet sufficiently sophisticated to make them feel a little smart and insightful for understanding it. But the fable is often an exaggerated and unnecessarily negative description of reality, and hence a distorting and malign influence on public policy. This book seeks to promote a more nuanced understanding of adverse selection in insurance.
Three Main Differences
Insurance textbooks typically represent adverse selection as a severe problem in all insurance markets, which should always be avoided, minimised or deprecated. Some typical examples are as follows:
The business of insurance inherently involves discrimination; otherwise adverse selection would make insurance unavailable.
Adverse selection … is an ever present fact in the insurance market.
Adverse selection … plagues insurers worldwide.
This book is critical of the orthodoxy illustrated by these quotations. Those who are indignant at my perceived heresies may tend to read into them a more extreme ‘straw-man’ version of the claims I actually make. I therefore state succinctly here the three main differences from orthodoxy in this book – one empirical and two normative – and some important limits on those differences:
(a) Adverse selection in insurance is usually weaker than most commentary suggests. (But I do not say that adverse selection is always unimportant, or that insurers should not be vigilant about it.)
(b) From a public policy perspective, ‘weak’ adverse selection in insurance is a good thing. This is because a degree of adverse selection is needed to maximise loss coverage, the expected losses compensated by insurance for the whole population. (But I do not say that adverse selection of any severity is always good.)
(c) To induce the degree of adverse selection which maximises loss coverage, some restrictions on risk classification are a good thing in some insurance markets. (But I do not say that all restrictions are always a good thing, or that all risk classification should be banned.)
The remainder of this book emphasises the differences, and does not dwell on their limits. My presentational style leans towards what the economist Paul Romer recently labelled ‘Stigler conviction’, following George Stigler’s advice 60 years ago that ‘new economic theories are introduced by the technique of the huckster … [but] … they are not the work of mere hucksters’.Footnote 2
Scope and Focus
The term ‘insurance’ can encompass many transactions and arrangements between parties of varying status and sophistication. My focus is on personal insurances, particularly those contingent in some way on the insured’s life or health – life insurance, annuities, income protection insurance, critical illness insurance and health insurance (medical expenses insurance). For these insurances, higher risks often face not only prospective disadvantage, but also some degree of current disadvantage (e.g. some degree of current ill-health). To a lesser extent, I also have in mind other personal insurances, such as travel, home and car insurance. I do not focus on insurances where the insured is a corporation of comparable strategic sophistication to the insurer, or where the insured views the contract as part of a speculative investment portfolio, rather than as protection against some unlikely and undesirable contingency.Footnote 3
Intuitions about public policy, and particularly perceptions of fairness in risk classification, are highly sensitive to this scope and focus. A common but unimpressive response to advocacy of restrictions on risk classification in (say) life insurance is a question along the lines, ‘Ah, but would you say the same for property insurance? Or marine, aviation or spacecraft insurance?’ The insinuation is that if one would not say the same for all classes of insurance, then this indicates some inconsistency in one’s thinking. But there is usually no inconsistency; the comparison is merely capricious. I see no reason why a preference for some restrictions of risk classification in one social context (say life insurance) should necessarily be accompanied by a preference for identical restrictions in some other very different commercial context (say aviation insurance). In this book, I usually have in mind personal insurances contingent in some way on the insured’s life or health.
Adverse Selection and Loss Coverage
While this book critiques orthodoxy about adverse selection from a number of perspectives, the main innovation is the concept of loss coverage. The following paragraphs sketch the argument; more detail is given from Chapter 3 onwards.
Consider an insurance market where individuals can be divided into two risk-groups, one higher risk and one lower risk, based on information which is fully observable by insurers. Assume that all losses and insurance are of unit amount (this simplifies the discussion, but it is not necessary). Also assume that an individual’s risk is unaffected by the purchase of insurance, i.e. there is no moral hazard.
If insurers can, they will charge risk-differentiated prices to reflect the different risks. If instead insurers are banned from differentiating between higher and lower risks, and have to charge a single ‘pooled’ price for all risks, a pooled price equal to the simple average of the risk-differentiated prices will seem cheap to higher risks and expensive to lower risks. Higher risks will buy more insurance, and lower risks will buy less.
To break even, insurers will then need to raise the pooled price above the simple average of the prices. Also, since the number of higher risks is typically smaller than the number of lower (or ‘standard’) risks, higher risks buying more and lower risks buying less implies that the total number of people insured usually falls. This combination of a rise in price and a fall in demand is usually portrayed as a bad outcome, for both insurers and society.
However, from a social perspective, it is arguable that higher risks are those more in need of insurance. Also, the compensation of many types of loss by insurance appears to be widely regarded as a desirable objective, which public policymakers often seek to promote, by public education, by exhortation and sometimes by incentives such as tax relief on premiums. Insurance of one higher risk contributes more in expectation to this objective than insurance of one lower risk. This suggests that public policymakers might welcome increased purchasing by higher risks, except for the usual story about adverse selection.
The usual story about adverse selection overlooks one point: with a pooled premium and adverse selection, expected losses compensated by insurance can still be higher than with fully risk-differentiated premiums and no adverse selection. Although pooling leads to a fall in numbers insured, it also leads to a shift in coverage towards higher risks. From a public policymaker’s viewpoint, this means that more of the ‘right’ risks – those more likely to suffer loss – buy insurance. If the shift in coverage is large enough, it can more than outweigh the fall in numbers insured. This result of higher expected losses compensated by insurance – higher ‘loss coverage’ – can be seen as a better outcome for society than that obtained with no adverse selection.
Toy Example
The argument above can be illustrated by a toy example, in the same spirit as dice-rolling examples to illustrate probability rules.
Consider a population of just ten risks (say lives), with two alternative scenarios for adverse selection. First, risk-differentiated prices are charged, and a subset of the population buys insurance. Second, risk classification is banned, leading to adverse selection: a different (smaller) subset of the population buys insurance. The two scenarios are represented in the upper and lower parts of Figure 1.1.
Two scenarios for risk classification
In Figure 1.1, each ‘H’ represents one high risk and each ‘L’ represents one low risk. The population has the typical predominance of lower risks: eight lower risks each with probability of loss 0.01, and two higher risks each with probability of loss 0.04. In each scenario, the shaded ‘cover’ above some of the ‘H’ and ‘L’ denotes the risks covered by insurance.
In Scenario 1, in the upper part of Figure 1.1, risk-differentiated premiums are charged. Higher and lower risk-groups each face a price equivalent to their probability of loss (an actuarially fair price). The demand response of each risk-group to an actuarially fair price is the same: exactly half the members of each risk-group buy insurance. The shading shows that a total of five risks are covered. Note that the equal areas of shading over one ‘H’ and four ‘L’ represent equal expected losses.
The weighted average of the premiums paid in Scenario 1 is (4 × 0.01 + 1 × 0.04)/5 = 0.016. Since higher and lower risks are insured in the same proportion as they exist in the population, there is no adverse selection. The expected losses compensated by insurance for the whole population can be indexed by:
In Scenario 2, in the lower part of Figure 1.1, risk classification has been banned, and so insurers have to charge a common ‘pooled’ premium to both higher and lower risks. Higher risks buy more insurance, and lower risks buy less. The shading shows that three risks (compared with five previously) are now covered. The pooled premium is set as the weighted average of the true risks, so that expected profits on low risks exactly offset expected losses on high risks. This weighted average premium is (1 × 0.01 +2 × 0.04)/3 = 0.03.
Note that the weighted average premium is higher in Scenario 2, and the number of risks insured is smaller. These are the essential features of adverse selection, which Scenario 2 accurately and completely represents. But there is a surprise: despite the adverse selection in Scenario 2, the expected losses compensated by insurance for the whole population are now larger. Visually, this is represented by the larger area of shading in Scenario 2. Arithmetically, the loss coverage in Scenario 2 is:
This book argues that Scenario 2, with a higher expected fraction of the population’s losses compensated by insurance, is superior from a social viewpoint to Scenario 1. The superiority of Scenario 2 arises not despite adverse selection, but because of adverse selection.Footnote 4
This very simple example may not have wholly convinced the reader. But I hope that Scenario 2 – where something good seems to be happening with adverse selection – has at least intrigued you. The key idea is that loss coverage – the expected losses compensated by insurance for the whole population – is increased by a degree of adverse selection. Loss coverage seems a reasonable metric for the social efficacy of insurance, and for comparing alternative risk classification schemes. Hence this book argues that insurance works better with some adverse selection.
Some readers may already have noticed that if the adverse selection progresses beyond Scenario 2 to its logical extreme, so that only a single higher risk remains insured, then loss coverage will be lower than with no adverse selection. This point will be addressed in Chapters 3–6. For now, I merely reiterate that this book’s message is one of moderation: it says that insurance works better with some adverse selection, not with any amount of adverse selection.Footnote 5
Outline
Part I: Introduction
In the remainder of Part I of this book, Chapter 2 describes the history of exaggerated predictions in policy debates about adverse selection, with particular reference to HIV testing, genetic testing, gender classification and racial classification. The documented history of exaggeration motivates the development of a more nuanced account.
Part II: Loss Coverage
Part II covers loss coverage. Chapter 3 uses only elementary arithmetic; Chapters 4–6 are more technical. Readers who are not concerned with technical details may prefer to skip Chapters 4–6, or perhaps just skim the graphs and end-of-chapter summaries.
Chapter 3 gives more detailed and realistic numerical examples than the toy example in this chapter, showing that while loss coverage is increased by the ‘right amount’ of adverse selection, it can be reduced if there is ‘too much’ adverse selection. This chapter also places loss coverage in context, relating it to other concepts in insurance theory.
Chapter 4 gives mathematical definitions for loss coverage, adverse selection and related quantities. It also presents graphical plots of loss coverage against adverse selection. The inverted U-shape of these graphs (indicating that loss coverage is maximised by an intermediate degree of adverse selection) is probably the most important visual image in this book.
Chapter 5 covers mathematical models of insurance markets. These facilitate an exploration of how loss coverage varies with changes in the low-risk and high-risk population sizes, probabilities of loss and demand elasticities.
Chapter 6 considers loss coverage under ‘partial risk classification’, that is with intermediate classification schemes in between complete pooling and fully risk-differentiated premiums. For comparing different risk classification schemes, I propose a novel metric, the ‘separation’ of a risk classification scheme, and its complement, the ‘inclusivity’ of a risk classification scheme.
Part III: Further Aspects of Risk Classification
Part III covers a variety of further aspects of risk classification. This material is more opinionated and less technically dense than Part II, and may be more readable and engaging to casual readers.
Chapter 7 gives a taxonomy of reasons why particular methods of risk classification may be considered objectionable. There are two broad groups of reasons. First, insufficient inclusivity – too much or too little pooling of dissimilar risks, resulting in suboptimal loss coverage. Second, misguided methods – even if the overall degree of pooling of dissimilar risks produces the highest feasible loss coverage, particular classification methods may be objectionable on various ethical, technical or practical grounds.
Chapter 8 examines empirical evidence for the existence of adverse selection in various insurance contexts, which turns out to be surprisingly weak compared with the strong predictions of orthodox insurance theory. From the perspective of this book’s main argument, this is a hopeful state of affairs, since ‘weak’ adverse selection is needed to maximise loss coverage.
Chapter 9, on myths of insurance rhetoric, explains ways in which adverse selection is often exaggerated and excessively maligned in policy debates. There are four main groups of ways: genuine misperceptions, strategic misrepresentations, rhetorical affectations and cognitive capture.
Chapter 10, on myths of insurance economics, appraises some themes which are pervasive in accounts of adverse selection and risk classification in insurance economics, but for which real-world evidence is lacking. These include the concepts that different levels of deductibles are used by insurers to separate high and low risks; that insurance is ‘rationed’ for low risks, while full cover is available for high risks; and that tax-and-subsidy schemes are always better from society’s viewpoint than risk classification bans.
Chapter 11 considers contexts where one of the main points of this book – that adverse selection is usually weaker than most commentary suggests – may be less valid. These contexts are of two types: (a) the customer has a large ‘information edge’, defined as the difference (not the ratio) of a true probability based on the customer’s private information, and a probability used by the insurer to set premiums or (b) the customer can engage in multiple independent transactions, either in parallel (diversification) or in sequence (repetition).
Chapter 12 discusses the concept of moral hazard in insurance. As with adverse selection, moral hazard is typically deprecated, but I argue that there is an optimal non-zero level of moral hazard.
Chapter 13 considers new surveillance technologies and the big data they can collect. To the extent that big data allows insurers to reduce the inclusivity of risk classification, this might reduce loss coverage, if adverse selection is reduced too much. Hence technical advances in the collection and analysis of big data are possible reasons for considering new controls on insurance risk classification.
Part IV: Conclusion
Chapter 14 concludes this book with a summary of the main points, and suggestions for public policymakers, economists and actuaries.
In the examples in Chapter 1, two regimes for risk classification were considered: either insurers charged risk-differentiated prices for high and low risks; or risk classification was banned, so that insurers had to charge a common pooled price to all risks. The latter regime induced some adverse selection, but also higher loss coverage (expected losses compensated by insurance); I argued that this was a better outcome for society as a whole. This surprising phenomenon of ‘beneficial adverse selection’ does not hold without limit. We shall see from Chapter 3 onwards that maximising loss coverage requires not unlimited adverse selection, but the right amount of adverse selection. This in turn requires the right amount of risk classification. This book treats adverse selection and risk classification as matters of degree.
Many commentators do not treat risk classification and adverse selection as matters of degree, but as matters of dogma. They say that any degree of adverse selection is always detrimental; that more risk classification is always better than less; and that any restriction on risk classification – for example, banning classification on a particular variable, such as gender or genetic tests – will always have dire consequences. Some grain of truth underlies each of these assertions, insofar as too little risk classification leading to too much adverse selection does indeed produce a bad outcome for society. But each of these assertions is often made in an unqualified and exaggerated form.
This chapter sets the context for the rest of this book by documenting the long history of such exaggerations, which have collectively created the myth of insurance adverse selection as an overwhelming force of darkness. I focus on four risk variablesFootnote 1 which have often attracted particular controversy: HIV tests, genetic tests, gender classification and racial classification.
Because this chapter is concerned with history, it necessarily quotes from a range of articles, press releases and other policy documents, some as recent as 2014 but others up to two decades old. In many cases the individuals and organisations quoted might no longer endorse their past views, or might today express more nuanced opinions. My purpose in such extensive quotation is not to call for contemporary censure of, or justification from, the authors whom I quote, but rather to substantiate the history of exaggeration which provides the context for this book.
Background: Principles of Insurance Pricing
This section gives a recap on basic principles of insurance pricing and risk classification. Readers with good prior knowledge may prefer to skip this section.
Assume a population of 100 individuals, each facing the same potential loss amount, and each with the same probability of say 15% over 1 year. Insurers can break even by offering insurance over 1 year at a premium of 15% of the potential loss.Footnote 2 Because all individuals have the same probability of loss, insurance can be offered to all comers at the same price. Each insurer pools all premiums it collects in single fund, from which payouts are made to 15% of customers who actually incur a loss. Because all individuals have the same probability of loss, insurers are not concerned that those who choose to buy insurance might have different characteristics from those who do not.
Now suppose instead that although all individuals face the same potential loss amount, there are five different probabilities of loss, 5%, 10%, 15%, 20% and 25%, with 20 individuals having each probability of loss. The average probability of loss is still 15%. If insurers offer insurance at a premium of 15%, the insurance may be seen as cheap by individuals whose probability of loss is 20% or 25%, and expensive by individuals whose probability of loss is 5% or 10%. Those who choose to buy may therefore be skewed towards those with higher probabilities of loss. This skew in insurance buyers towards individuals with higher probabilities of loss is called adverse selection.
If insurers facing adverse selection pool the skewed group of individuals with five different probabilities of loss in a single fund and price insurance at the 15% average probability of loss over the whole population of 100 individuals, the insurers will make losses. For a pooled fund to break even, the insurance price needs to correspond to the average probability of loss not for all 100 individuals, but rather for those who choose to buy insurance at that price. This is likely to be rather higher than 15%. For example, if all the 25% and 20% risks, and half of the 15% risks, but none of the 5% or 10% risks, choose to buy insurance when a single price is charged, the break-even price is (20 × 25% + 20 × 20% + 10 × 15%)/50 = 21%.
There may be an alternative to this pooling of different risks in a single fund. If insurers can distinguish the higher risks from the lower risks, they can pool approximately similar risks in separate sub-funds (which we shall call risk-groups), and charge a different price to each risk-group, set to achieve break-even for the buyers in that risk-group. For example, suppose the 20% and 25% risks are all men under age 30; the 5% and 10% risks are all women over age 50 and the 15% risks are men or women between the ages of 30 and 50. Then if insurers can observe prospective customers’ age and gender, they can allocate the risks to three risk-groups for ‘low’, ‘medium’ and ‘high’ risks, and charge different prices to individuals in each risk-group. This classification process is called risk classification (or sometimes underwriting).
What premiums do insurers need to charge to the ‘low’, ‘medium’ and ‘high’ risk-groups to break even? The break-even premium for the ‘medium’ risk-group with probability of loss 15% is obviously 15%. For the ‘low’ and ‘high’ risk-groups pooling (5% and 10%) and (20% and 25%) risks, respectively, the break-even premiums may be skewed towards the loss probability at the upper end of each risk-group, reflecting some adverse selection within the group. Suppose only two-thirds as many of the 5% risks compared with the 10% risks buy insurance, and the same for the 20% risks compared with the 25% risks. The break-even premiums for the low and high risk-groups would then be 8% and 23% (i.e. slightly above the midpoints of each risk-group, 7½% and 22½%).Footnote 3
Where risks differ very substantially between individuals, insurance may not work well without some degree of risk classification. To give an extreme example, life insurance is unlikely to work well in a completely age-blind regime, where the same premium rate is offered to a 20-year-old and a 90-year-old. Under such an age-blind regime, the pooled price which insurers need to charge to break even may be very high, with mainly very elderly people buying insurance. Life insurance may then be largely ineffectual in providing financial security to younger families. So some degree of risk classification generally makes insurance work better.
But how much risk classification? In the scenario above, the insurer was able to observe customers’ age and gender, and thereby separate the risks into three risk-groups: low (5% and 10% risks), medium (15% risks) and high (20% and 25% risks). Now suppose that a new classification technology is developed which enables insurers to separate the risks into five risk-groups, one for each of the five levels of risk from 5% to 25%. Would this be better or worse than using three risk-groups?
From the perspective of a single insurer competing with other insurers in a market where risk classification is unregulated, using five risk-groups is likely to be better than using three risk-groups – or if not better, at least prudently defensive of its competitive position. This is because once some insurers adopt the new classification, their lower prices for the 5% risks and 20% risks, compared with the pooled prices of 8% and 23% which other insurers charge to those risks under the three risk-group regime, may attract most of the 5% and 20% risks. Insurers still using the crude ‘low’, ‘medium’ and ‘high’ risk-group regime will then be left with a higher proportion of 10% and 25% risks in their ‘low’ and ‘high’ risk-groups than they anticipated, and may therefore make losses in those risk-groups. Thus once some insurers adopt the new five risk-group regime, competitive pressures encourage all insurers to do so.Footnote 4
From a public policy perspective, however, it is not obvious that the new classification technology facilitating five risk-groups rather than three risk-groups gives a better result for society as a whole. The use of five risk-groups will eliminate the adverse selection which previously existed in the ‘low’ and ‘high’ risk-groups. But the example in Chapter 1 suggested that some degree of adverse selection can make insurance work better, in the sense that the expected losses compensated by insurance for the whole population is increased (loss coverage is increased). It is possible that moving from three risk-groups to five risk-groups reduces loss coverage. Whether this is in fact so depends on the responses of the various risk-groups to changes in the prices they face (technically, the demand elasticities of the various risk-groups).
The principles of insurance pricing and risk classification as sketched above suggest that from society’s viewpoint, optimal risk classification is a question of degree. We saw in Chapter 1 that some adverse selection gives a better outcome for society as a whole; we shall see in Chapter 3 that too much adverse selection can make things worse. A common distortion of these principles is to assert that more risk classification is always better than less, and to predict that any limits on risk classification will have dire consequences for society as a whole. As one lawyer has put it, ‘actuaries are sometimes like the boy who cried wolf when it comes to adverse selection’.Footnote 5 The rest of this chapter presents examples of such exaggerated predictions.
HIV Tests
When David Hurlbert, a San Francisco business consultant, sought to renew his health insuranceFootnote 6 in 1986, Great Republic Life Insurance Company asked a few extra questions. In the past six months, had he suffered from a sexually transmitted disease? Or an immune disorder? Or had he lost weight recently? The company required its sales force to ask these extra questions of ‘single males without dependents … in occupations that do not require physical exertion’. Hurlbert didn’t answer the questions; he sued Great Republic for discriminating against gay men. In a settlement of the litigation in 1990, Great Republic paid a total of $85,000 to Hurlbert and two law firms representing him, and agreed to stop attempting to identity customers’ sexual orientation as part of its insurance underwriting.Footnote 7
As can be seen from this anecdote, in the 1980s the insurance industry worldwide was alarmed by the rise of HIV and AIDS. Starting in about 1985, premium rates for single men were substantially increased, and HIV tests were required for all policies above a modest size. In health-related insurances, any AIDS-related illness was subject to exclusions (meaning that no benefit would be paid for such illnesses). For many years, most insurers declined to provide any life insurance to men whom they suspected might be gay. Insurers who did offer cover charged premiums at twice or more standard rates.
Insurers attempted to make inferences about applicants’ sexual orientation not only from direct questions, but also from circumstantial evidence such as joint house purchase and casual stereotyping of occupations. The company in the anecdote above, Great Republic, directed its agents to highlight applications from ‘restaurant employees, antique dealers, interior decorators, consultants, florists, and people in the jewelry or fashion business’.Footnote 8 All applicants were asked whether they had ever taken a HIV test. Negative as well as positive HIV test results were used as justification for refusing insurance, on the grounds that where the test was negative, the fact that the individual had sought testing indicated higher risk. The catch-22 nature of this rationale attracted a great deal of criticism, because it seemed likely to discourage people from taking HIV tests, and so promote transmission of the virus. Eventually, in 1994, the Association of British Insurers banned the practice in the UK; since then, questions have referred only to ‘testing positive’ or ‘awaiting the results of a test’.
For justification of these policies, insurers pointed to projections produced by actuarial professional associations. Figure 2.1 shows the projections produced by an Institute and Faculty of ActuariesFootnote 9 working party in 1987 for the numbers of deaths per annum from AIDS in the UK. Six projections from A (highest) to F (lowest) were given, with peak deaths ranging from 15,000 per annum up to almost 60,000 per annum. The actual numbers of deaths from AIDS in the UK in each year are shown by the lowest line barely visible at the bottom of Figure 2.1. Deaths from AIDS in the UK peaked at less than 2,000 per annum in the mid-1990s – around one-tenth the lowest projection (projection F) – and have fluctuated in the range 400–600 per annum since around 2000.
The 1987 projections of numbers of deaths per annum from AIDS (A highest, F lowest) versus actual deaths (lowest solid line, barely visible above x-axis)
The lowest prediction for deaths from AIDS in the UK was too high by a factor of 10, and the highest prediction was too high by a factor of 30. This in itself is not necessarily worthy of criticism. In its early stages in the mid-1980s, the ultimate spread of the AIDS epidemic was extremely unpredictable. Based on experience at the time, survival periods after diagnosis with AIDS were believed to be very short, with perhaps 10% surviving three years.Footnote 10 Although the predicted deaths turned out to be much too high with hindsight, the exaggerated early response to AIDS may have been more justifiable than the response a decade later to the introduction of a small number of genetic tests, which from an early stage were clearly likely to be immaterial to insurance.
However, what may be more justifiably criticised in relation to AIDS is the reluctance to acknowledge in the 1990s that earlier predictions had been overstated, and to update insurance practices and recommendations accordingly. Questions about sexual orientation were still being used to reject or rate customers in the early 2000s, by which time it was clear that AIDS was wholly insignificant for insurance in the UK. The Association of British Insurers eventually issued a recommendation against questions about sexual orientation only in 2004. Even today, many insurers continue to apply exclusions so that no payment is made for any HIV-related illness under critical illness and income protection policies, which the Association of British Insurers continues to promote as ‘best practice’.Footnote 11 There is no justification for this in the pattern or prevalence of HIV-related illness, and no reason why HIV-related illness should not now be treated like any other illness.
In summary, most predictions of the effects of HIV and AIDS on insurance prices in the UK were substantially overstated. The exceptional (and now quite unjustified) treatment of HIV-related illnesses continues to this day. The range of predictions promulgated by the Institute and Faculty of Actuaries was between 10 and 30 times too high. HIV and AIDS did not have the large adverse selection effects which were widely predicted.Footnote 12
Genetic Tests
After HIV and AIDS, the next phenomenon to attract adverse selection concerns from the mid-1990s onwards was the perceived threat to insurance companies from not knowing the results of any genetic tests which had been taken by insurance applicants. Figure 2.2 shows the annual number of mentions of the phrase ‘genetics and insurance’ in English language news items in one online news database. The peak in the early 2000s reflects excessive adverse selection concerns, which later experience has largely negated.
Annual numbers of mentions of ‘genetics and insurance’ in English-language news items worldwide
The decline in comment since the early 2000s reflects a consensus in many countries around legislative or quasi-legislative bans on insurers asking questions about any pre-symptomatic genetic tests insurance applicants may have taken. In the UK, a voluntary restriction by insurers has been extended several times, most recently (at the time of writing) to 2019.Footnote 13 Laws restricting the ability of insurers to use genetic tests have been enacted in many European countries, including Austria, Belgium, Denmark, France, Norway, Sweden and the Netherlands. The prediction by many insurance experts around the millennium that use of genetic test results would imminently become essential for insurance companies’ survival now seems exaggerated or misconceived.
The argument often put forward around the millennium was that people with some knowledge of genetic predisposition to illness would routinely seek to buy large amounts of life insurance. For example, a press release from the Institute and Faculty of Actuaries in July 1999 said:
Bans [on insurers using genetic test results] have already been imposed in some countries, such as Denmark, Netherlands, Norway and a number of states of the USA (e.g. California), leaving the way open for people with knowledge of their genetic condition to take advantage of insurance companies, thus pushing up the cost for everyone.Footnote 14
Actuarial associations in the USA made similar claims. In a letter dated 16 February 2007 lobbying the then Speaker of the House, Nancy Pelosi, against the Genetic Information Non-discrimination Act,Footnote 15 the American Academy of Actuaries wrote:
A ban on the use of genetic information … would have a direct effect on premium rates, ultimately raising the cost of insurance to everyone.Footnote 16
For people with knowledge of their genetic condition to ‘take advantage’ of insurance companies, they must pay less than they would pay if the insurance companies knew of their condition; the cost of insurance is not raised ‘for everyone’. The assertion that insurance becomes more expensive ‘for everyone’ makes no sense – until one understands that for many actuaries, the ontological concept of ‘everyone’ excludes persons with genetic disadvantages.
Despite the absurdity of such claims, they found a receptive audience among policymakers around the millennium. Indeed the UK government seemed to accept fully actuaries’ claims that the use of genetic tests would imminently become indispensable in insurance. Popular opinion was, however, quite hostile to this concept.Footnote 17 To bridge the gulf between popular and expert opinion, in 1998 the government established a Genetics and Insurance Committee (GAIC). The GAIC was to consider applications from insurers supported by actuarial evidence for the permission to use particular tests. It was said to be predicated on the following rationales (as expressed by the Department of Health in response to a Parliamentary Committee report in 2001):
The Government recognises the legitimate concerns of insurance companies over the consequences of ‘adverse selection’ where individuals with adverse test results take out larger than usual amounts of insurance cover in the expectation of an early claim and a substantial payout.
The Government is aware of insurers’ concerns about the risk of adverse selection and supports the continued assessment by GAIC of applications for the use of genetic test results.Footnote 18
The concerns of insurance companies were not in fact legitimate, and the raison d’être of GAIC – that genetic discrimination in insurance could be justified by extant actuarial evidence – was wholly misconceived. This was starkly demonstrated by the fact that after a single much-criticised approval for a test for Huntington’s disease in life insurance in October 2000, GAIC never received any further applications for test approval from insurers. The actuarial evidence which GAIC’s terms of reference envisaged would be presented by insurers simply did not exist, and could not be produced to order, despite very strong ideological motivations for insurers to do so. As a result GAIC became moribund, and was eventually disbanded in 2009.
Even if GAIC had succeeded in approving more tests, it might not have succeeded in its underlying purpose of promoting public acceptance of genetic discrimination. In announcements by or about GAIC, it was never explained why the State’s endorsement of genetic discrimination should reassure those who would thereby be disadvantaged. On the contrary, the concept of genetic discrimination endorsed by a State body seemed ominously evocative of eugenic projects such as Nazi Germany’s obsession with ‘genetically defective’ individuals, and analogous sterilisation programmes in the USA and Scandinavia. As an official arbiter of genetic discrimination, GAIC itself was an uncomfortably close analogue of the ‘Genetic Health Courts’ established to provide a scientific imprimatur to genetic discrimination in Germany in the 1930s (albeit GAIC’s initial powers were fortunately less wide-ranging). In these senses, the concept of State approval via a body such as GAIC might increase rather than reduce alarm about discrimination.Footnote 19
The concept of GAIC may not have originated in government; something like it seems to have originally been proposed to the Department of Health by the Association of British Insurers.Footnote 20 I surmise that the concept had its origins in contemporary understanding about genetics and adverse selection in the insurance industry. To assess this understanding, it is useful to examine contemporary articles written for discussion among semiprivate insider groups of experts. Many such articles were even more alarmist than public pronouncements. A portentously titled discussion paper on ‘The freedom to underwrite’ discussed before a large audience at the headquarters of the Institute and Faculty of Actuaries in 1996 included the following remarks:
If genetic history is banned on proposals, adverse selection becomes a major issue.
I would agree but it could herald the end of life assurance as we know it.
My own feeling is that the government should outlaw all forms of home and postal testing … The industry will have terrible problems if legislation does not take place.
Recently, some preserved samples from the body of the US politician, Hubert Humphrey, have been tested. It has been discovered that his bladder cancer could have been diagnosed with genetic testing over ten years before he had any symptoms. Think of the damage individuals could inflict on the industry with such knowledge.
We have to protect ourselves and our existing policyholders from proposers who know the results of genetic or other tests which reveal a poor outlook.
Because of this, I believe that from a theoretical viewpoint and provided offices act responsibly, we are invariably right.Footnote 21
The florid tone of these quotations exemplifies actuaries’ paranoia in the mid-1990s about a supposedly existential threat to the insurance industry from people affected by genetic conditions. But questions about genetic tests have now been banned for over 15 years, with no discernible impact on the health of the insurance industry. And in my view, comments such as those quoted above were patently absurd even at the time. It was always clear that the non-disclosure of a small number of genetic tests never represented any realistic threat to the operation of insurance. This is not just hindsight. Here is what I wrote in response to a consultation by the Human Genetics Commission in early 2001:
Recent actuarial studies suggest that even under the most unfavourable assumptions, ignoring genetic tests and family history would have little effect on most insurance markets … There should be a presumption against insurers’ right of access to genetic tests, except for exceptionally large policies, or where insurers can show that ignoring tests would seriously disrupt the financial stability of insurance markets. All evidence to date suggests that under this criterion, little or no access to test results would be justified.Footnote 22
By the middle of the 2000s, insurers’ paranoia concerning genetic tests and insurance around the turn of the decade had substantially subsided. The predictions made a few years earlier could now be seen by all to be grossly overstated. In 2001, the Institute and Faculty of Actuaries had predicted:
There could be a substantial reduction in consumer choice though products becoming dearer or even with the withdrawal of some types of product.Footnote 23
In fact, the lowest premiums for critical illness insurance decreased by about 20% over 10 years up to 2012. The lowest premiums for life insurance decreased by around 25% over the same period.Footnote 24 An exhaustive programme of bottom-up research (that is, considering various genetic conditions and aggregating the results) completed by 2011 in the Genetics and Insurance Centre at Heriot-Watt University suggested that the impact of banning genetic tests in life insurance can be expected to be less than 1% of premiums.Footnote 25
Research and practical experience of genetics and insurance in the UK has not deterred actuaries in other jurisdictions from continuing to make fanciful claims. In 2014, the Canadian senate was considering a proposal whereby insurers would be banned from asking for genetic test results for life insurance cover of up to 1 million Canadian dollars, or benefits payable annually up to 75,000 Canadian dollars (these are similar limits to those which are applied in the UK). The Canadian Institute of Actuaries issued a press release commenting as follows:
A ban on using genetic tests is a bad idea. It hurts the vast majority of Canadians. The research shows that term insurance premium rates could go up by 30% for males and 50% for females.Footnote 26
When considered against the experience in the UK, where life insurance premiums decreased by around 25% in the decade after the ban was introduced, the figures of 30% and 50% increases appear absurd. Such exaggerations can have real policy consequences: in this case, they were enough to dissuade the Canadian senate from passing the proposed legislation in December 2014.
Gender
Controversy about gender-based pricing predates both HIV testing and pre-symptomatic genetic testing. Differences between insurance risks for men and women are statistically well-evidenced, but generally much smaller than the differences attributable to some genetic predispositions or to HIV infection. Some restrictions on the use of gender in pricing have applied for many years, particularly in pensions. In the USA, higher pension contributions for women (reflecting their longer life expectancy) were outlawed by the Supreme Court as long ago as 1978. In the UK, a unisex requirement applied from 1988 onwards to pensions bought from insurers under ‘contracting out’ from the State pension scheme.Footnote 27 These rules did not apply for non-pension products.
However following a 2011 decision of European Court of Justice, insurers throughout the European Union were banned from charging different prices to men and women on all new policies issued after 21 December 2012. Prior to the ban, different countries adopted different approaches, with several already mandating unisex prices for at least some types of insurance.Footnote 28
The wide variation in pre-existing rules across different European states, all with substantial insurance industries, made it easy to see that banning gender pricing would not create any insuperable difficulty for insurers. Nevertheless prior to the ban, insurers in European states where gender pricing was the norm made many florid predictions of the effects from a ban.
Some examples of such predictions were contained in a document The use of gender in insurance pricing published by the trade association Insurance Europe. Sample claims from this document are as follows:
There are no reasonable alternatives to the use of gender in insurance pricing.
This … [unisex pricing] would be likely to jeopardise the future of numerous small and medium-sized enterprises (SMEs).Footnote 29
Insurance companies are almost invariably substantial institutions; they are not SMEs. The reference to SMEs appears to be an ‘applause light’: a largely meaningless allusion which is made only because it is thought policymakers might be more sympathetic to SMEs than to the monolithic reality of most insurance companies. Nearly five years after the ban came into effect, I am not aware of any evidence that SMEs throughout Europe have been devastated by gender-neutral insurance pricing.
In a press release, the Institute & Faculty of Actuaries made the following predictions:
If gender cannot be taken into account when setting premiums, young female drivers could face premiums of up to 50% higher while young male drivers may benefit from reductions of around 25%.Footnote 30
The Actuarial Association of Europe (previously known as the Groupe Consultatif), a pan-European group of actuarial professional associations, issued a press release claiming that banning gender pricing would increase prices for both men and women:
The Groupe Consultatif Actuarial Européen has highlighted that gender is an important risk factor for insurance and that a prohibition on its use could increase costs for both men and women overall.
The Groupe Consultatif believes that the impact of self-selection could be even greater for pensions business, leading to an increase in annuity costs for men but no discernible reduction in costs for women.Footnote 31
The claim that prices would increase for both genders was similar in character to claims that restriction on access to genetic tests would raise prices ‘for everyone’. In a competitive market, such claims are incoherent. This can be seen in detail from a simple thought experiment involving equalisation of annuity prices for men and women; analogous arguments apply for equalisation of insurance prices.
Suppose that before the gender ban, the price of an annuity of £1 per annum from age 65 is £10 for men and £12 for women, and equal numbers of men and women buy annuities. Assume all annuities are of the same amount (this simplifies the discussion, but is not necessary). After the ban, a first approximation of the pooled price is the simple average of £11. This is cheaper than before for women and dearer than before for men.
Suppose that as a result of this relative cheapness and dearness, more women and fewer men now buy annuities. This is adverse selection. Then to cover the total cost of annuity payments, the pooled price will need to be weighted away from the simple average of £11 and towards the previous price for women of £12.
How far towards £12 the price needs to be weighted depends on how far the male/female mix of annuity purchasers shifts from its original 50:50 proportion. But even if most men now decline to buy annuities, the weighted average price, reflecting a mix of predominantly females and fewer males, will still be less than the previous female price.
The claim that the pooled price will be higher than the previous price for the higher-cost gender can be true only if the ban is followed by a substantial increase in premiums which is unrelated to what insurers pay out in benefits. This is not plausible in a competitive market: if an insurer tries to enforce such an increase, it will be undercut by rivals. As far as I am aware, the ‘more expensive for everyone’ outcome has not been documented in any real market, anywhere in the world.
The actual effect of the gender ban can be seen in surveys conducted a few months after the ban. For car insurance, an index produced by aggregator website confused.com and actuarial consultants Towers Watson in October 2013 showed that after the ban, car insurance premiums for 17–20-year-olds had risen by 9% for women, but fallen by 29% for men.Footnote 32 Since very much more than half of aggregate premiums in this age group relate to men, this suggests that the average premium over both genders had fallen significantly. As with genetics, I do not suggest that the ban caused the average premium to fall, but only that any effect of the ban on the average premium over both genders was imperceptibly small, and swamped by other influences on premiums.
Race
Much less needs to be said about pricing differences by racial origin than the other categories discussed in this chapter. There is evidence of racial discrimination in insurance in the past, but probably no more than would be expected from contemporary social attitudes.Footnote 33 In the UK, the Race Relations Act 1968 outlawed racial discrimination in the supply of all goods and services, and the Race Relations Act 1976 outlawed indirect discrimination; neither contained exemptions for insurance. In the USA, the statutory position is more ambiguous: insurance regulation is largely devolved to state level, and around half of states have no specific ban on racial classification in underwriting, at least for some classes of personal insurance.Footnote 34
Nevertheless, in the USA as well as in the UK, nobody now argues that racial classification is important to the operation of insurance. When evidence of past racial classification has come to light in recent years, it appears to have been perceived by insurers as a source of embarrassment, rather than as a practice to be defended with arguments about adverse selection. For example, in the early 2000s a series of lawsuits in the USA alleged that blacks who had bought life insurance policies in the 1960s through 1980s were continuing to be charged 25% more than whites under those policies in the 2000s. Insurers appeared anxious to settle these cases with a minimum of publicity, rather than to defend the practices of the past.Footnote 35
Although insurers no longer overtly defend racial classification, some insurers may continue it covertly. In 2010, the UK consumer organisation Which? obtained quotations from 19 insurers for two car insurance scenarios. The only difference between the scenarios was whether the 50-year-old applicant had been born in the UK, or born overseas and moved to the UK at the age of 15. Most insurers quoted the same prices for both scenarios. But four brands owned by the FTSE100 insurer Admiral quoted prices an average of 18% higher for the applicant who had moved to the UK as a teenager.Footnote 36
Differentiating premiums on this question is ‘indirect’ racial discrimination, that is, it is likely that a higher proportion of the white population can satisfy the requirement for a lower premium than ethnic minority populations. Under the Equality Act 2010, indirect racial discrimination could be lawfully justified only if the discriminatory act is a ‘proportionate means of achieving a legitimate aim’. This seems unlikely in this example, particularly given the ‘immigrated age 15, now age 50’ detail of the test scenario, which precludes any argument that a recent immigrant might have lower familiarity with UK driving conditions. As far as I am aware, the point has not been tested in court.
Insurance Discrimination Is Gradually Increasing
Insurers and their advocates often assert that only a very few life insurance applications are not accepted at standard rates. For example in a position statement on genetics and insurance in 1999, the Institute and Faculty of Actuaries said:
Notwithstanding the underwriting requirements of life insurance companies, it is estimated that over 95% of insurance applications are accepted at standard rates.Footnote 37
Similarly in 2014, the Canadian Institute of Actuaries asserted:
A significant proportion of policyholders – over 90% in Canada, the US and the UK – are issued insurance at standard rates.Footnote 38
Figures collated by reinsurer Swiss Re in surveys of its client companies in the UK suggest that the actual average percentage of life insurance applicants accepted at standard rates in 1999 was around 92%. By 2011, this had fallen to 79%.Footnote 39 More recently, an informal report from one Australian intermediary indicates an acceptance rate of around 79% of life insurance applications.Footnote 40
Legislative and regulatory developments which purport to limit discrimination by disability, gender and genetics may give the impression of discrimination in insurance gradually reducing over time. The trend in the Swiss Re figures just quoted suggests that this impression is probably wrong. Although discrimination is nominally circumscribed in a few areas, the bigger picture is that in the absence of regulatory limits on risk classification, ‘competitive adverse selection’ forces each insurer to seek to discriminate more finely than its rivals. The result is that insurers today probably exclude or surcharge a substantially higher proportion of applicants than they did 20 years ago. This process of ‘competitive adverse selection’ will be discussed further in Chapter 8.
Hindsight Bias?
The preceding sections present examples of overstated predictions of the likely effects of partial restrictions on risk classification. A possible objection is that my critique is informed by hindsight, and that much of the commentary I criticise was more reasonable at the time it was made. To some extent, this must be true: the future is always less known than the past. But many of the more egregious exaggerations of past commentary were contemporaneously identifiable, and indeed were publicly highlighted at the time. For genetics, this is illustrated by the quotation earlier in this chapter from Thomas (Reference Thomas2001), my response dated February 2001 to the Human Genetics Commission’s public consultation on genetics and insurance.Footnote 41
Summary
The examples on the last few pages have shown that strong claims about the negative effects of any restriction on insurance risk classification – whether related to HIV tests, genetic tests or gender classification – have a long history. This book questions these claims on a variety of grounds. The main point of the book is that some restrictions on risk classification, far from having adverse effects, can actually make insurance work better, in the sense of increasing loss coverage. This is the subject of Chapters 3–6.
