Very Large Calculation Systems: A Specialized Solution for the Complex Needs of Advanced Knowledge Workers

Abstract
Motivation: Advanced calculations on large data sets provide important business insights. Such calculations must be flexible enough for the dynamic nature of advanced analytics done by actuaries and other high-skill users, yet must also leverage the power and stability of large-scale IT systems. The system design detailed in this paper balances these concerns, delivering the optimal combination of both.

Method: The design is based on a pattern the author observed repeatedly during a decade of building systems for actuaries and researchers: data is provided using traditional data warehousing, the calculations are implemented by IT in a manner that solidifies stable elements yet externalizes change-prone elements, system operation is exposed through self-service interfaces that allow users to configure and run the calculations against large data volumes, and the output is delivered by standard business intelligence tools and customized approaches.

Results: Systems with this design: optimize the users’ time by moving the majority of the lower-value work to IT, have solid auditability and legal compliance, and provide high computing power and storage capacity, but do require users to give up some amount of control and flexibility.

Conclusions: Organizations should use this design to give their most advanced knowledge workers high power, localized control, and optimal efficiency.

Keywords: Data warehousing, exploratory data analysis, actuarial systems, ratemaking, modeling, simulation.

Volume
Spring
Page
1-40
Year
2010
Categories
Actuarial Applications and Methodologies
Data Management and Information
Actuarial Systems
Actuarial Applications and Methodologies
Data Management and Information
Data Administration, Warehousing and Design
Financial and Statistical Methods
Statistical Models and Methods
Data Mining
Financial and Statistical Methods
Statistical Models and Methods
Exploratory Data Analysis
Actuarial Applications and Methodologies
Ratemaking
Financial and Statistical Methods
Simulation
Publications
Casualty Actuarial Society E-Forum