Actual Candidate Papers as a Study Aid
The CAS Examination Committee is releasing four actual candidate papers, in their entirety, from the Fall 2008 exam sitting for review and discussion. The purpose of this exercise is to give candidates and members alike greater insight into how papers are scored by the graders so that they can better prepare for future examinations. It is our hope that candidates will use this exercise to learn how better to craft examination responses that maximize their credit with minimal effort. It is also our hope that this exercise will increase confidence among candidates in the fairness of the grading process.
For this exercise, volunteers were solicited on the Actuarial Outpost.
From among those volunteers, one passing and one failing candidate were drawn at random for each of Exams 6 and 9. The selected volunteers have agreed to allow their exam responses and scores to be published and discussed for the benefit of all candidates. These volunteers have further agreed that they may not use the publication of their entire papers for personal benefit beyond that available to all candidates through this publication. That is, they may not use their papers to reopen appeals or file new appeals, as that would represent an unfair advantage over candidates whose papers are not published. Please note that the window for appeals for the Fall 2008 exam sitting is closed and will not be reopened under any circumstances.
The Examination Committee invites discussion of the published papers on the AO in the discussion thread: Candidate Paper Discussion. Registration for this discussion forum is free to all candidates and members alike, with participation subject to AO guidelines. The Examination Committee will monitor and participate in this discussion process and responses will be coordinated through the Chair. Two to three weeks into that discussion, the Examination Committee will solicit question numbers from the participants where there is interest in the actual grading rubrics. For a sample of those questions, the detailed rubrics will be acquired from the grading pairs and published here along with the candidate papers.
Attached to the top of each candidate paper is a summary of their scores by question. The scores represent the average of the two graders for each question. To interpret these scores, it is important to understand how the grading process works. Each grading pair is assigned a question and is given a preliminary grading rubric that specifies the correct answer(s) to the question and the partial credit scheme to be applied in the grading. Using this rubric, the graders independently mark approximately 30 papers and then stop to compare notes. If additional specifications are necessary in the rubric to make their scoring consistent, those are added and the grading process starts over on the entire stack of candidate papers. If, at any point in the process, the graders discover a novel answer not considered by the rubric that they believe should be accepted, it is added to the rubric and all previously graded papers are reviewed to determine if additional credit is appropriate based on the newly accepted response.
Once the grading pair has completed all candidate papers, they compare their grades for each candidate prior to the grading session in Las Vegas. For all papers where the grades differ by more than a specified amount (normally .25 or .50 point), the graders review the scores and reconcile their grades to within the tolerance specified by the Part Chair. In Las Vegas, the scores by question are aggregated and candidates with total scores within a specified tolerance of the tentative pass mark (the “regrade” band) are noted for further reconciliation. The grading pairs review the papers of all candidates within the regrade band and precisely reconcile their scores. In this way, all candidates within the regrade band will have a precise score that is a multiple of 0.25. For candidate papers outside the regrade band, there will remain differences among the grading pairs of up to .50 point. For this reason, the scores published here will sometimes appear in 1/8th point increments, even though the grading rubrics always specify that grading is in .25 point increments.
The Examination Committee hopes that candidates find the material presented here and the ensuing discussion helpful. We invite feedback on the usefulness of this exercise, as we are always looking for ways to improve the examination process. We believe we have a world class process and we want to make sure that it remains that way through continual improvement.
The Examination Committee also asks that you be patient during the period in which we collect the rubrics from grading teams. These documents are not always electronic, meaning that standard mail or fax will be used. Furthermore, we may experience situations where both members of a grading pair are away from the office for a period of time in the collection window.
The Examination Committee
Selected Exam 6 Responses and Grading Rubrics, (Updated 13 October 2009)