Examining the Process—Part I
Creating the Examination
by Arlene F. Woodruff, FCAS
Part 1: July—Summertime. Swimming.
Boating. Vacation. Spring Exams.
Spring Exams? Yes, for the CAS Examination Committee work begins about 10 months before the exam. By this time, the CAS Syllabus Committee is well on its way to finalizing what will be on each exam. As each exam part of the CAS Syllabus of Examinations is finalized, guidelines for creating the exam, including the various topics and the types of questions (application, analysis, recall, etc.), are given to the CAS Examination Committee. Committee members are assigned a particular segment of the Syllabus and are expected to become an expert on their particular topic.
A question writer (1) not only creates questions from their portion of the Syllabus, they are also responsible for documenting the source material, proposing model solutions, and estimating what point values should be assigned. Since two or more committee members are assigned to each section, the members (2) also review questions from the other question writers in their section. The committee usually needs a pool of about 250 points worth of questions to be able to select 100 points for an exam. Each exam part committee has various target dates for submitting and reviewing questions so that by November they can begin to put an exam together.
Proctoring Exams. CAS Annual Meeting.
Teleconferences for Spring Exams.
By this time a pool of questions has been compiled, and the process of selecting questions for the exam begins. Teleconferences abound between the part chair for the exam and the committee members. The part chair is responsible for creating the exam and making sure that is follows the guidelines set forth by the CAS Syllabus Committee. Questions are selected for the exam, and a pool of supplemental questions from each section is held in reserve. The part chair (3) will review all the questions follow editorial conventions and the format actually looks like what the students will see on the CAS Spring Examinations.
Year End—Fall Exam Results. Annual
Statement Work. Spring Exam First Review.
In the beginning of the year the first draft of the exam is sent to the vice chair-exam series, and exam consultant and in some cases an academic consultant. All the exams for the Spring series are reviewed by the Spring vice chair (4) for technical content, clarity, balance, length and difficulty. The exam consultant is usually an actuary with expertise in the material covered by the exam. The consultant (5) looks for technical content while keeping an eye out for questions which conflict with real-life situations. An academic consultant is not necessarily an actuary but rather an independent expert in the field. Teleconferences between the vice chair, part chair and consultant can take several hours as each question is discussed in detail. During this review, any questions that are faulty or ambiguous will be fixed, or thrown out and one of the supplemental questions will be used instead. The part chair will then prepare a second draft for the next level of review.
February—Annual Statement Deadlines.
Spring exam Second Review.
It is time for the second draft of the exam to be given to the chairperson of the CAS Examination Committee (6), another exam consultant (7) and a proofreader (8) for review in addition to the vice chair and first draft consultant for review of any changes. The chairperson of the CAS Examination Committee reviews all the exams. The proofreader concentrates on uniformity and grammar, yet there have been times when the proofreader found a common sense question on a point that the members of the Examination Committee had taken for granted. Again, teleconferences of several hours in length are needed to discuss these questions. If any questions still are deemed to be faulty or ambiguous, they will be thrown out and supplemental questions will be substituted.
Each question on the exam has been reviewed by a minimum of eight people. For some exams, additional consultants may be called upon for advice. By the second round of review, faulty questions are not fixed because not enough people would have the chance to review them, and the chances of making additional errors are too great. If students still find a question that they believe is faulty, they should write to the Casualty Actuarial Society within tow weeks after the examination date.
March—Spring Exam Finalized.
The exam must be ready about six weeks before the exam date to allow for printing and distribution to the exam centers. All the committee members mentioned except the academic consultant and the proofreader are volunteers. In many cases the number of hours to produce a quality exam can run in to the hundreds for each member of the committee. The Committee can now take a breather until grading time.