Getting the Most out of the Examiners’ Report
Beginning with the May 2012 examinations, the Examination Committee started publishing “Examiners’ Reports” for Exams 5 through 9. These reports include narratives for each question describing where points were achieved and where points were commonly missed by the candidates. The Examiners’ Report is an invaluable tool when preparing for an upcoming exam or consequently assessing missed opportunities on a prior exam. All candidates should read the reports from prior versions of the exam for which they are sitting. However, when incorporating the Examiners’ Report into your studying routine, it is important to keep in mind what the report is and, most importantly, what it is not.
What the Examiners’ Report Is:
The report provides insight into what the graders for each question were looking for in responses that received full or nearly-full credit. This includes an explanation of common mistakes and oversights among candidates. Though candidates can lose points on any given question for a variety of reasons, common themes tend to emerge across the range of responses.
A common complaint among candidates is something like “We have no idea what you want from us!” This criticism is perhaps further magnified by the move toward testing at higher levels of Bloom’s Taxonomy, which tends to create more open-ended questions. While the Examiners’ Report will likely not completely address the open-ended nature of some exam questions, it does provide a valuable insight into the differences between responses that are comprehensive and those that are lacking in some way.
What the Examiner’s Report Is Not:
The report is not a guide to providing the minimally acceptable answer to each question. As stated above, the report describes what graders were looking for in perfect responses, not what they expected from the minimally qualified candidate (MQC). In fact, the report provides no insight into what the MQC score was for each question. For example, the report may include a sentence such as, “A very small number of candidates received full credit,” but the majority of candidates may have still scored above the MQC standard for that question. Likewise, just because a report says “Candidates scored well on this question,” doesn’t mean that the majority scored the maximum points.
Additionally, the report may not be helpful in the appeals process. In order for an appeal to be valid, it must demonstrate that a particular response, which was initially considered to be incorrect, should instead be given full or partial credit. Since the report is confined to the response(s) that were considered correct in the first place, it is unlikely to help in crafting an appeal, though it may provide guidance on whether a particular question is worth appealing. So in summary, you should think of the Examiners’ Report as a tool to help you master the material, rather than as an aid to changing the outcome of a prior sitting or as a means of gleaning information about the MQC standards set by the Examination Committee.
The primary mission of the Candidate Liaison Committee is to communicate the concerns of candidates to CAS leadership. If you have any questions or comments on the examination process, or on any other topic, please feel free to share your thoughts with us by filling out the Candidate Liaison Committee Feedback Form.
