Examining the Process - Part VI
Examining the Pass Ratios
by Arlene Woodruff, FCAS
CAS Examination Committee Assistant Vice Chairperson
"I was one of the lucky few to have passed Part 7 this time, but I still agree the 7 and 4B ratios were too low. I'm sure plenty of people who got fives knew the material as well as I did. I'm looking for other opinions.
"Do you think the pass ratio was so low because the 4B and 7 exams were so difficult, or because they had the most people taking them, or both?..."
-ACAS Wannabe, Students' Corner, CAS Web Site
While not wanting to intrude upon the Students' Corner, the CAS Examination Committee tries to keep in touch with students' concerns. The results of the Fall 1997 Parts 4B and 7 exams have spurred considerable discussion not only in student circles, but within the Admissions Committees, Executive Council, Board of Directors, and many other actuarial circles. From a student's point of view the simple question is "Why was the pass ratio so low?" But the CAS leadership believes the students should be brought into the bigger discussion of "What is the ultimate goal of the CAS and how is the Examination Committee's performance relative to this goal?"
The CAS Task Force on Education outlined the direction of our new educational process. In moving toward a new exam structure, the Examination Committee is beginning to introduce some of the guidelines that will be fully in place by the year 2000. One of the most common complaints from students has always been the amount of strict memorization that has been required to pass exams. One of the goals that we are trying to achieve is to reduce the number of strict memorization questions and to incorporate more synthesis questions that require additional thought. With the Fall 1997 exams, the Examination Committee started to change the overall thought process and tried to move in this direction. There were fewer "standard" questions and more "thought" questions. (Yes, the Examination Committee is tracking how many points on each exam are strictly memorization questions.) This was not intended to be a radical change. Rather, it was supposed to be a gradual transition to a new level of thought and discernment.
In looking at the fall results for Parts 4B and 7, it is not difficult to see that the results were not what the committee expected. The committee realized that thought and understanding questions might be more time-consuming than strict memorization questions. Unfortunately, the committee misjudged what the relative time weights should be or how well the students would respond. For Part 7 in particular, this resulted in extremely low scores. While Part 4B does not involve strict memorization, the relative number of synthesis questions also produced low scores.
If the Examination Committee had used fewer thought questions...
If there had been fewer overall questions...
If more students had been able to adapt...
If the exam time were longer...
If wishes were horses....
Regardless of the intent of those creating the exams, the results were in and the committees had to determine a passing score. The conflict of passing a "reasonable" number of candidates versus having a "reasonable" passing grade became an issue of much discussion within the Examination Committee. The statistics were analyzed with "a fine-tooth comb." Benchmark questions were evaluated. Exam consultants were called upon for their analysis of the situation. In keeping with our current procedures, many people were involved in the passing grade deliberations. In the final determination, no one was fully satisfied. The passing scores were considerably below historic levels in recognition of the difficulty and/or length of the exams. (In fact, the pass marks for these two exams were well below any other pass marks that the committees had seen in the 1990s.) The statistics on how well the candidates demonstrated their knowledge of the material were also poor and the committees felt that the students needed to demonstrate a minimum overall knowledge of the material in order to pass. The result was a lower number of passing candidates as well.
Where do we go from here? Having learned from the experience of the Fall 1997 exams, the Examination Committee will strive to apply those lessons to create a proper balance between "thought" questions and standard questions that will produce an exam with proper overall length and difficulty. Change is never an easy task and the Examination Committee, as well as the students, will continue on our journey toward the year 2000.