Summarizing Insurance Scores with the Gini Index

As we go about building our predictive models for ratemaking, many have found that the usual goodness-of-fit diagnostics (i.e., F-statistics, t-statistics, etc.) are uninformative, since their scale does not correspond to what most feel is the economic value of the model. David Cummings and I have attempted to deal with this in previous “Brainstorms” columns by introducing a statistic we call the “Value of Lift” or VoL.1 We have been comfortable with using this statistic on large samples, but we get progressively uncomfortable with using it as the sample size decreases. Another problem we have encountered with the VoL is that many object to the fact that it depends upon an arbitrary cutoff point where we select the unprofitable risks.

To address these problems, Dave and I enlisted the help of Edward T. (Jed) Frees. Jed is a professor at the business school of the University of Wisconsin, a Fellow of the Society of Actuaries, and has a Ph.D. in Statistics. We have jointly submitted a paper for publication that addresses these problems.2 This column summarizes that paper.

Let Pi be the measure of the current exposure for the ith risk. It could represent the current premium or it could represent an exposure base such as car years. Let ai and bi represent two different sets of independent variables related to the loss, Li for the ith risk. Let Pa,i = E[Li|Pi,ai] and Pab,i = E[Li|Pi,ai,bi] where the expectation, E, is determined by a predictive model. Define the relativity Ra,i = Pa,i/Pi. For a given relativity, r, let’s now define the coordinates (x(r), y(r)) of what we call an ordered Lorenz curve by as follows:

In words, the coordinates of the ordered Lorenz curve plots the percentage of the predicted losses on the x-axis against the percentage of the actual losses on the y-axis, as ordered by the relativity, Ra,i. Similarly, define the relativity Rab,i and its corresponding ordered Lorenz curve.

Let’s consider an example. Consider a portfolio of 250,000 personal lines risks Pi = 1, Pa,i = ai , and Pab,i = 0.5 ai + 0.5 bi . The losses, Li , have a Tweedie distribution with mean Pab,i . Pa,i represents our best estimate of the expected loss given that we only observe the factor ai . The exact algorithm that produced this example is in the R code accompanying the Web version of this article.

The Lorenz curves ordered by Ra,i and Rab,i are on Figures 1 and 2. The dotted lines in Figure 1 below highlight the point (60.0, 39.7) on the Lorenz curve ordered by Rab,i. The dotted lines in Figure 2 also highlight the points (60.0, 44.8) on the Lorenz curve ordered by Ra,i. The Lorenz curve has significant meaning in an insurance context. It shows that including both the factors ai and bi in the risk selection criteria allows an insurer to select a portfolio consisting of 60% of the risks, yet having 5.1% fewer losses than a risk selection criteria based only on the factor ai. Of course, the 60% cutoff is arbitrary, but the Lorenz curve includes all possible cutoffs.

The Gini index is a single number that summarizes the information in the Lorenz curve. It should be easy to see that if the losses were random relative to the factors ai and bi, the Lorenz curve would be a straight 45° line. The Gini index is equal to twice the area bounded by the 45° line and the Lorenz curve. In the example, the Gini indices for Ra,i and Rab,i are 19.1% and 29.2%, respectively.

Here are some theoretical properties of the Gini index and the corresponding ordered Lorenz curves:

• The Lorenz curves, Ra,i and Rab,i are concave up and lie at or beneath the 45° line.
• The Gini index for Ra,i is nonnegative, and the Gini index for Rab,i is no smaller than that of Ra,i.
The proof of these properties assumes that Pa,i and Pab,i are equal to the conditional expected values as specified above. In practice, the conditional expected value and the Gini index must be estimated from data and hence are subject to estimation error. In the paper we derive estimators for the standard error of the Gini index and for the difference between two Gini indices. These estimators depend on the first two moments (including covariances) of the underlying Pi, Pa,i and Pab,i, and thus are sensitive to the underlying frequency and severity distributions. Table 1 provides estimates of the Gini indices and their standard errors for our simulated example, with varying portfolio sizes.

This table illustrates what we have seen with real data—in general, one needs a fairly large sample to reliably conclude that adding new information improves the Gini index.

In principle, the Gini index could be applied to situations less structured than the example above. For example, Pi could be the current premium that was influenced by competitive and regulatory pressures. In the example above, the information used in calculating Pa,i was contained within the information used in calculating Pab,i. While the economic interpretation of the Lorenz curve and the Gini index makes sense in less structured settings, the nice theoretical properties listed above may not hold. Using real data we have seen that when comparing two different estimators of the expected loss, based on nonoverlapping information, the Lorenz curves may cross. This indicates that there are pockets of risks within the portfolio where one estimator outperforms the other, and vice versa. The lesson to be learned here is that if your goal is to improve risk segmentation, make sure that your “improved” model contains all the information used in your existing model.

To summarize, this article proposes a new measure of the effectiveness of predictive models that is specifically tailored to the problems of insurance underwriting and ratemaking. The article also explores some of the statistical properties of this measure.