2026 Request for Proposals: Evaluating LLMs for Actuarial Perception Tasks for an LLM Benchmark and Re-Evaluation Suite

• Develop a rigorous set of actuarially relevant perception and classification tasks, as well as drawing on existing open insurance benchmark datasets.
• Example (including but not limited to): Claims triage/classification, underwriting decision judgments, policy segmentation, and fraud or litigation likelihood flagging.
• Ideally, broad actuarial competencies are decomposed into granular, well-defined evaluation sub-tasks. For example, breaking "reserving" into loss development pattern recognition or claim severity classification.

• Proposals should find, assemble, or simulate their own datasets.
• All data must be fully legally usable and publishable as part of the CAS benchmark suite.
• Example papers and potential data sources:
  • CUFE Insurance Evaluation Suite:
    • https://arxiv.org/pdf/2511.07794
    • https://github.com/CUFEInse/CUFEInse
  • INS-MMBench:
    • https://arxiv.org/pdf/2406.09105
    • https://github.com/FDU-INS/INS-MMBench
    • FDU-INS/INS-MMBench
  • Benchmarking Agents in Insurance Underwriting Environments:
    • https://arxiv.org/pdf/2602.00456

• Example industry benchmarking initiatives:
  • InsureBench:
    • https://www.insurebench.com/

• Provide all task definitions, datasets, scoring methodologies, and evaluation scripts.
• Scoring must rely on reproducible, objective metrics (e.g., accuracy, F1, Brier score, log loss).

• Implement standardized evaluation protocols for the top current commercial models from Google, Anthropic, and OpenAI.
• Similarly evaluate at least 3 of the top open-source models.
• The evaluation framework must be built for repeatable re-testing. As new frontier models are released, the CAS should be able to run the existing benchmark suite against them with minimal additional engineering effort.

• Deliver an interactive, user-friendly public platform that compares model performance across tasks.
• Build the platform as an extensible framework so new models and new benchmark tasks can be added over time.
• Enable actuaries and researchers to propose additional evaluation tasks and datasets.
• Allow model ratings to update dynamically based on newly submitted data and defined performance criteria.

• A detailed written report summarizing methodology, model performance comparisons, and limitations.
• The CAS will own and maintain the benchmark suite and comparison platform after delivery, including re-testing new models as they are released, adding new benchmark tasks over time, and platform upkeep.
• Proposals must include documentation sufficient for the CAS to run and maintain the system independently, without requiring the contractor's continued involvement. This would include (but isn’t limited to) instructions, code, and configuration files.
• Clear recommendations on maintenance, governance, and update cadence.

To ensure openness and reproducibility all datasets, code, benchmark definitions, scoring scripts, and documentation must be published to a CAS GitHub repository.

The CAS AI Working Group will evaluate proposals using the following criteria:

• Understanding of actuarial perception and classification tasks
• Technical competency in AI and benchmarking
• Feasibility and clarity of approach
• Quality of benchmark design
• Strength of team qualifications
• Cost effectiveness
• Ability to provide reproducible, transparent results

If no proposal meets the requirements, the CAS may decline to award a contract.

The award amount will most likely fall in the $25,000 to $65,000 range, although requests outside this range may be considered if supported by exceptional scientific merit and a well-justified budget.  Applicants are encouraged to request only the funding necessary to complete the proposed work.  We will consider budget appropriateness and the efficient use of CAS resources during proposal evaluation.

• The maximum budget for this project is $75,000.
• Proposers should include detailed pricing for the work as specified, along with an estimate of any ongoing costs the CAS should expect to bear when maintaining the system independently. Note this is separate from the funding covered by this RFP.
• A portion of the funding should be used to cover usual and customary travel expenses to present the paper at a CAS-sponsored seminar or meeting.
• CAS research funding does not cover overhead expenses but can cover salary and fringe benefits.
• Compensation will be commensurate with the time required to carry out the work.

• As a condition of selection, the CAS requires that all rights, title, and interest, including copyright and patent, in and to the report be owned by the CAS. The selected researcher(s) must sign a formal research agreement that assigns all such rights to the CAS.
• In any publication of the report, the researcher(s) will receive appropriate authorship credit. The CAS may publish the report in its entirety, or any sections thereof, in any format and medium as it finds fit, including, but not limited to CAS publications, and electronic versions on its website or physical storage media. Publishing outside the CAS requires permission from the CAS, and the authors are to acknowledge previous publication.
• If the author publishes the final draft paper on a pre-print platform such are ArXiv, the CAS must be acknowledged as the funder of the research.
• The researcher(s) should make every effort to be available to present their findings and share their expertise at a CAS meeting or seminar.
• The project will be overseen by the CAS AI Working Group with regular check‑ins.

• To aid research adoption, the final work product’s code and data will also be placed in the CAS’s GitHub repository, under the MPL2.0 license. The author should include a link in the body of the final paper to the CAS Research Paper Repo here: https://github.com/casact/research-papers.
• Authors must report specific use of artificial intelligence, if any, while producing research.
• All tables and figures must be numbered and include an in-text callout.
• Manuscripts must include an abstract with a maximum word count of 150 words.
• References must use the author-date system from Chicago Manual of Style, 18th edition.
• Manuscripts should include a list of 2–10 relevant keywords to help readers discover the content online.