Precision In Forecasting

Precision in Forecasting The SG-2 Philosophy of forecasting encompasses four major goals, in approximate order of priority. 1. Comprehensive (forecasts should incorporate a broad range of signficant drivers of utilization; e.g. a forecast that only focuses on one driver cannot reliably forecast total utilization) 2. Timely (forecasts should incorporate the latest information available about emerging technologies, population trends, etc. 3. Locally specific (forecasts should reflect local conditions as the practice of health care is highly regional) 4. Precise (Precision in the input data and empirical timings and impact factors is critical) Many may be surprised that precision is last in priority. The reasoning behind this is as follows: First, if a forecast is not comprehensive (e.g. it ignores or is unaware of major drivers of change), then it is by definition systematically wrong. No degree of precision will correct the overall error. Lack of timeliness (e.g. not incorporating the latest understanding of technologies, etc.) and inappropriate local applicability are other such systematic sources of error for which no degree of precision can compensate. Second, it is intrinsic to the prospective nature of forecasting, that the will be an unavoidable imprecision to judgements required to estimate the timings and impacts of future drivers of change. When an industry report predicts that a certain drug will most likely be approved by 2005, then to say that it will be approved in June of that year (as opposed to July) is not only probably wrong, but also misleading. Third, and perhaps most significantly, the comprehensive nature of our forecasts – spanning economic cycles, hundreds of technologies, population trends across many dimensions results in enormous amounts of data that are produced. For example, a dataset run that tabulates technologies, component analysis across several geographies, age groups and institutions may very well result in over 5GB (gigabytes) of data. Hence, a comprehensive run may necessarily imply (in order to reduce the computational complexity) that the overall forecast be built up from pared-down versions of the complete forecast. As the following example will show, the necessary reduction in precision may result, for example, in a less than exact "footing" of totals across different "pared-down" runs. For example, the economic effect (more specifically, the effect that unemployment has on reducing utilization) affects patients who are largely covered by commercial (e.g. non Medicare) health insurance. If a dataset broken down by age-group cohorts is run, then (to insure increased precision), the unemployment effect is only

applied to non-Medicare (e.g. < age 65) age cohorts. On the other hand, if a dataset (for the sake of being able to run other simultaneous dimensions such as institution and geography) is run without specific age groups (e.g. the age groups are "rolled-up" into a generic category), the nationally-specified Medicare fractions will be used for each DRG instead of the more precise specification at run-time. The tradeoff in precision will necessarily result in a slightly different Economic forecast for these two, ostensibly similar, datasets. The solution to this is to present a single forecast run with as many dimensions as possible. Because of the raw runs are fairly efficient, these calculations can proceed. However, when it becomes necessary to actually view the individual technologies, components, etc., the space requirements increase exponentially.