COTOR Announces Results of Round 3 Challenge
Louise Francis, Chairperson, and Steven Visner, Member, CAS Committee on Theory of Risk
Since 2003, the CAS Committee on Theory of Risk (COTOR) has challenged practitioners to flex their actuarial muscles to solve intricate exercises in risk. Respondents to COTOR's third challenge were faced with an even more complex exercise that built on the previous challenges to include more real-world aspects.
Round 3 of the COTOR challenge addressed some of the real-world issues identified in Round 2, one of which is trend. That is, when forecasting the severity of a future period, the actual severity is likely to be different from those in the historic data used to estimate claim severity due to the impact of claim severity inflation. Thus, the claims in the sample distributed to submitters reflected trend. Stuart Klugman drew seven consecutive years of claims at random from a heavy-tailed distribution. The 490 claims were sampled and split equally among seven years. Each year of claims was drawn from the same distribution, except the scale parameter changed due to inflation. The challenge was to estimate mean severity and a 95 percent confidence interval for the layer $500,000 xs $500,000 for the next (eighth) year. The challenge was submitted to both the "practitioner" community as well as the "student" community.
The distribution selected was a 30 percent/70 percent mixture of an exponential and a Pareto. Year-to-year inflation increased dramatically over time.
Eleven practitioners submitted results to the Round 3 challenge. Results of these submissions were as follows:
Whereas most submitters estimated the average trend of approximately 20 percent fairly accurately, most results assumed consistent trend over time, which the review committee believes accounted for the majority of submitters underestimating the true severity. Many submitters used empirical data to estimate the layer severity. Given that this was a heavy-tailed distribution, COTOR was surprised that most submitters did not fit loss distributions.
- The range between the lowest and highest severity estimate was 75 percent (of the low estimate)
- 36 percent of the submitters produced estimates within five percent of the true result.
- The majority (9 of 11) of results underestimated the true mean.
- The average result was 11 percent lower as compared to the true average severity.
|The submitters used a number of interesting and novel trend estimation methods... trimmed means, percentile matching, chi-squared statistic, bootstrapping, and maximum likelihood estimation. |
A committee composed of Philip Heckman, Stuart Klugman, and the authors of this article, Francis and Visner, selected winners. The winners chosen to present their solutions during the November 2005 CAS meeting in Baltimore were Glenn Meyers, Stephen Fiete, and Thomas Wright. The criteria for choosing the winners was new and creative ways to solve the problem, clarity of exposition, accuracy of result, and establishment of a methodology that practicing actuaries can use.
The submitters used a number of interesting and novel trend estimation methods. These included using trimmed means (where high and low sample values are eliminated), percentile matching, use of the chi-squared statistic, bootstrapping, and maximum likelihood estimation where the scale parameter was allowed to vary by year for trend.
The Pareto distribution, a distribution found frequently in extreme value literature was one of the distributions favored by submitters who fit distributions. Another distribution appearing frequently in the literature on heavy-tailed distributions, the Student t/log-t, was used for the first time in this challenge.
All three winners of round three of the challenge fit several probability distributions to the data. Two of the three winners used a Bayesian approach to determine the confidence intervals. That is, it was assumed that the sample could be from one of a number of possible probability distributions. Each of the possible probability distributions was assigned a weight and this weight was used to generate samples from that distribution in a simulation or apply numerical techniques to compute an aggregate probability distribution.
This challenge was also issued for the first time to the student community. The deadline for student submissions was December 31, 2005. We expect that the student with the best solution will be invited to a CAS meeting to present their results.
COTOR encourages CAS members and practitioners to further pursue some of the interesting and useful techniques that were provided by people responding to the challenge. The papers and PowerPoint presentations can be found on the COTOR Web Site.