Letter to the Editor
To Whom it May Concern:
Regarding the article "Was the Examination Too Long?" which appeared in the June 1997 issue of Future Fellows, the Society blew it. Badly.
First of all, if the people who took the exam say it was too long (and the examination survey seems to indicate as much) then the exam WAS too long, period. The candidates are the only ones who are in an appropriate position to make the call. The Exam Committee has as much time as they want to devise and grade the questions, as many glances at the footnotes as they want, and no pressure to perform since, as we see, no one is allowed to second-guess them. The exam in question being Part 9, the candidates are not without experience and sophistication in their exam preparation techniques. These people know when an exam is too long.
But more importantly, the response of the Society to the exam survey was disturbing, to say the least. If the Society is not going to take the concerns of exam candidates seriously, if the survey is not going to have any effect on the exam process, then don't waste the candidates' time. We have enough to worry about on exam day. I can't think of a better way to discourage filling out the survey than circulating the captioned article.
The authors take the position that since "only 26 percent" of the surveys were submitted the results don't count. Look for that percentage to go down, now that a Society publication has voiced what candidates must view as a more-or-less official position that nobody's going to take the surveys seriously anyway.
Thank you for your letter. You have touched on significant points and I appreciate the opportunity to discuss them.
Your assertion that well-prepared candidates who took the exam are in the best position to comment on its length is correct. The Examination Committee goes to great pains to make each exam consistent from one sitting to the next, including review by examination consultants for length as well as for clarity, question design, etc. However, the Examination Committee has not found a way to evaluate exams on a significant sample of prepared pseudocandidates to see if it meets the time requirement standards to which you would like them to perform. The best approximation that is available are the committee members who once were candidates. I encourage you to join them as soon as you obtain your Fellowship.
Your assertion that no one is allowed to second-guess the Examination Committee is simply not true. Questions are prepared by Examination Committee members and submitted to the Part Chair who edits, picks, and chooses. Questions are thrown out or rewritten for many reasons, including concern about the length of time it will take candidates to respond. The Part Chair must then take the critique and respond with changes to all issues raised by the consultants who review the exam.
Your assertion that Examination Committee members have as much time as they want is not true. It takes a lot of time to write questions and a lot longer time to convince others that your questions are indeed of high quality. Grading exams requires less creativity than writing questions, but is brutal from a short-term high hour per day time standpoint. It is also unforgiving in that someone else independently grades the questions and each grader must justify all the scores he or she assigns.
Your assertion that the CAS does not take seriously the survey and other information provided by candidates is not true. All information is used to help understand how the exams can be improved. As you have pointed out, the linkage is not always easy because, in many ways, the candidates have the best perspective on the exam itself.
The authors did observe that "since only 26 percent of the Part 9 Examination-takers submitted their surveys, the results may be biased." The authors did not conclude or intend to suggest that the survey results do not count. The authors' observation that the results may be biased is much different than taking a position that the results do not count.