Fail Forward: Using the Exam Analysis Report

Every spring and fall, actuarial candidates wrap up an arduous journey when they put their best foot forward in hopes of passing an exam. While everyone wants to pass, there are many who end up failing during any particular sitting. I know I have been on the receiving end of a failing score more times than I would like!

Upon receipt of a failing grade for Exams 5-9, the CAS will send each candidate an "Examination Analysis." This report provides the candidate a small glimpse into how well they performed on each question. The report has a row for each question that identifies the following items:

1) The total point value available for the question.

2) The quartile range in which the candidate scored in comparison to the total point value for that question.

3) A breakdown by question subpart. This breakdown indicates if the candidate received from 0-49% or from 50%-100% of the points available for that subpart.

4) A percentage indicating how the candidate scored in comparison to the average score of all candidates. This percentage is in 25% increments from 0-25% up to >175%.

The Examination Committee also publishes an "Examiners' Report" within a few weeks of the exam results release that includes narratives for each question describing where points were achieved and commonly missed by the candidates. Reviewing the "Examination Analysis" report in conjunction with the "Examiners' Report" can help candidates focus on areas that may need improvements before the next sitting. Below are some considerations for ways to accomplish this task.

Tactics for Improvement

First, it may be wise to consider questions on which a low percentage of the available points was scored. Since the pass mark is generally in the 67%-75% range, it is appropriate to consider any questions where the score falls in or below the 50%-74% quartile. Certainly, all questions in the 0-24% and 25%-49% quartiles should be sections of the syllabus that are revisited.

The next area to consider includes any questions receiving less than 75% of the average score of all candidates. Achieving a passing score on an exam means exceeding the "minimally qualified" candidate threshold. Therefore, it is imperative that answers approach or exceed what an average candidate scores. Consistently scoring lower than 75% of the average score shows areas where other candidates are more thoroughly prepared.

Finally, the subpart breakdown section of the "Examination Analysis" is the last place to focus. Each question subpart will indicate a "0-49" or "50-100" to show what percentage of points a candidate received. If a low-scoring question (as a percentage of the total) has several parts and only one of them is "0-49," then further study is needed. The subparts analysis has limited usefulness, however, because of the limited number of subparts or scoring in the same zone for all parts.

Focus on Weaknesses; Maintain Strengths

After identifying the questions and subparts that need review, it is time to zero in on the material that must be learned to pass on the next attempt. Consult the "Examiners' Report" to determine what syllabus objectives are represented by each of these low-scoring questions. Review the question and attempt to isolate the section of the text reference that is applicable. The "Examiners' Report" should also provide details the graders were looking for in a candidate's answer. Use these details as a reference and guide for learning the material. The end goal should be to maintain knowledge in all other areas of the syllabus but improve on areas of weakness.

In my experience, studying for exams is a layering experience. Building up enough knowledge to cover each of the syllabus objectives takes time and repetition. Certainly, candidates will have their own ways of learning the material and preparing for exams. Make every effort to find the best method for you. Hopefully this article will put one more tool in your toolbox to be successful on your next exam attempt!