Chapter 13 Forecasting

Chapter 13 Forecasting

It is difficult to forecast, especially in regards to the future.

MGS3100 Julie Liggett De Jong

It isn’t difficult to forecast, just to forecast correctly.

Economic Forecasts Influence:

Numbers, if tortured enough, will confess to just about anything.

Government policies & business decisions

1

Insurance companies’ investment decisions in mortgages and bonds

Service industries’ forecasts of demand as input for revenue management

Excel Features & Functions FEATURES • Regression • Solver • Sorting FUNCTIONS • SUMPRODUCT( ) • SUMXMY2( ) • YEAR( ) • MONTH( ) • RIGHT( )

Quantitative Forecasting Models

Quantitative vs Qualitative Forecasting Models

Expressed in mathematical notation

2

Quantitative Forecasting Models 1. Causal (Curve Fitting) a. Linear b. Quadratic

2. Moving Averages (Naive) a. Simple n-Period Moving Average b. Weighted n-Period Moving Average

3. Exponential Smoothing a. Basic model b. Holt’s Model (exponential smoothing with trend)

Based on an amazing quantity of data 4. Seasonality

Important Variables

X

Independent variable(s)

Y

True value of dependent variable

Yˆ

Predicted or forecasted dependent variable (Y hat)

Y

Average value of dependent variable (Y bar)

Causal vs Time Series Models

Causal Forecasting Models

Requirements

3

Independent and dependent variables must share a relationship

Curve Fitting

Oil company wants to expand its network of self-service gas stations

We must know the values of the independent variables when we make the forecast

Self Service Gas Stations

We’ll use historical data for five stations to calculate average traffic flow and sales

4

Sales & Traffic Data

Plot the averages in a scatter plot.

Traffic flow: average # of cars / hour Sales: average dollar sales / hour

Figure 1, p274

Method of Least Squares

300

300

250

250

200

200

S ales /hour ($)

S ales /hour ($)

Scatter Plot of Sales & Traffic Data

150 100

100

50

50

-

y = a + bx

150

0

50

100

150

200

250

0

50

100

150

Cars/hour

200

250

Cars/hour Figure 2, p274

Use Regression to fit a Linear Function

Figure 3, p275

Regression computes three types of errors n

^

–

RSS =

(Yi – Y )2 Σ i=1

ESS =

(Yi – Yi )2 Σ i=1

TSS =

– (Yi – Y )2 Σ i=1

n

^

Regression Residual

n

Total

TSS = ESS + RSS

R2 = Figure 5, p277

RSS TSS

5

Should we build a gas station at Buffalo Grove where traffic is 183 cars/hour?

How confident are we in this forecast?

^ y

=

+ -

2 * Standard Error (Se)

+

b

*

x

Sales/hour = 57.104 + 0.92997 * 183 cars/hr

= $227.29

Confidence intervals use the following statistics: 1.00 =68%

1.96 = 95.0%

n

Yˆ

a

Se =

^

(Yi – Yi )2 Σ i=1 n – k -1

=

3.00 = 99.7%

ESS n – k -1

6

Excel calculates the Standard Error for us.

The 95% confidence interval is: [227.29 – 2(44.18); 227.29 + 2(44.18)] [$138.93; $315.65]

Fitting a Quadratic Function Other important information: 9T-statistic and its p-value 9Upper & Lower 95% 9F significance 9R2 and Adjusted R2

Figure 5, p277

Fitting a Quadratic Function

Use Solver to fit a Quadratic Function

300 250

Sales/hour ($)

200

y = a0 + a1x + a2x2 150 100 50 0

50

100

150

200

250

Cars/hour

Figure 10, p283

y = a0 + a1x + a2x2

Figure 7, p281

7

We could create a formula that exactly passes through every data point…..

Which curve to fit?

n

SSE =

^ Σ (Yi – Yi )2

i=1

But, why wouldn’t we want to do that?

Goodness of fit statistics: Sum of Squared Errors (SSE)

Goodness of fit statistics: Mean Squared Error (MSE)

8

MSE =

Sum of Squared Errors

Regression: SSE = 5854 Quadratic: SSE = 4954

(# of points – # of parameters)

Causal Forecasting Models Negative Slope indicates downward trend

Positive Slope indicates upward trend

140

140

120

120

y = 5x + 5

y = -5x + 135 Customers

100

80 60

80 60

40

40

20

20

Time

Time-Series Forecasting Models

21

25

23

15

19

17

13

11

5

9

3

7

25

21

23

15

19

13

17

11

5

9

7

1

1

0

0

3

Regression: MSE = 5854 / (5 - 2) = 1951.3 Quadratic: MSE = 4954 / (5 - 3) = 2477.0

Profit

100

Time

1. Curve Fitting: a) Linear b) Quadratic

Time is the independent variable

9

2. Moving Averages (Naive) a) Simple n-Period Moving Avg b) Weighted n-Period Moving Avg

3. Exponential Smoothing a) Basic model b) Holt’s Model (trend)

Curve Fitting Seasonality

Plot historical values as function of time and draw a linear “trend line”. Use trend line to predict future value.

Moving Averages (Naïve): Use previous period’s actual value to forecast the current period (i.e., use 12th value to predict 13th value). The Bank of Laramie

10

Moving Averages:

Simple n-Period Moving Averages:

Use average of past 12 values as best forecast for 13th value.

Use average of the most recent 6 values to predict 13th value.

Steco: a simple nperiod moving averages forecasting model

MONTH Jan. Feb. Mar. Apr. May June July Aug. Sep. Oct. Nov. Dec.

ACTUAL SALES ($000s) 20 24 27 31 37 47 53 62 54 36 32 29

THREE-MONTH SIMPLE MOVING AVERAGE FORECAST

(20 (24 (27 (31 (37 (47 (53 (62 (54

+ + + + + + + + +

24 27 31 37 47 53 62 54 36

+ + + + + + + + +

27)/3 31)/3 37)/3 47)/3 53)/3 62)/3 54)/3 36)/3 32)/3

= = = = = = = = =

FOUR-MONTH SIMPLE MOVING AVERAGE FORECAST

23.67 27.33 31.67 38.33 45.67 54.00 56.33 50.67 40.67

(20 + (24 + (27 + (31 + (37 + (47 + (53 + (62 +

24 27 31 37 47 53 62 54

+ + + + + + + +

27 + 31 + 37 + 47 + 53 + 62 + 54 + 36 +

31)/4 37)/4 47)/4 53)/4 62)/4 54)/4 36)/4 32)/4

= = = = = = = =

25.50 29.75 35.50 42.00 49.75 54.00 51.25 46.00

Three- and Four- Month Simple Moving Averages

yˆ

16

Goodness of fit statistics

MAD =

MAPE =

∑

∑

actual sales − forecast sales

all forecasts

all forecasts

number of forecasts

actual sales − forecast sales actual sales number of forecasts

∗ 100 %

=

y

15

+

y

14

+

y

13

+

y

12

4

Table 1, p290

STECO: Simple n-Period Moving Average

MONTH Jan. Feb. Mar. Apr. May June July Aug. Sep. Oct. Nov. Dec.

ACTUAL SALES ($000s) 20 24 27 31 37 47 53 62 54 36 32 29

THREE-MONTH FOUR-MONTH SIMPLE MOVING SIMPLE MOVING AVERAGE ABSOLUTE AVERAGE FORECAST ERROR FORECAST

$ $ $ $ $ $ $ $ $

23.67 27.33 31.67 38.33 45.67 54.00 56.33 50.67 40.67

7.33 9.67 15.33 14.67 16.33 0.00 20.33 18.67 11.67

SUM = MAD =

114.00 12.67

$ $ $ $ $ $ $ $

ABSOLUTE ERROR

25.50 29.75 35.50 42.00 49.75 54.00 51.25 46.00

11.50 17.25 17.50 20.00 4.25 18.00 19.25 17.00

SUM = MAD =

124.75 15.59

Figure 14, p291

11

Simple n-Period Moving Average forecasting models have two shortcomings

Philosophical Shortcoming Most recent observations receive no more weight or importance than older observations.

Operational Shortcoming All historical data used to make forecast must be stored in some way to calculate the forecast.

Weighted n-Period Moving Averages:

Weighted n-Period Moving Averages:

Recent data is more important than old data

Use weighted average of previous values & assign higher weights to more recent observations

Resolves philosophical shortcoming of simple period moving average forecasting

yˆ = α 0 y6 + α1 y5 + α 2 y4

12

Constraints:

Constraints:

The α' s (weights) are positive numbers

Smaller weights are assigned to older data

alpha2 = alpha1 = alpha0 = SUM OF WTS=

Constraints:

0.167 0.333 0.500 1.00

Month Actual Sales (000) 3month WMA Fcst Absolute Error January 20 February 24 March 27 April 31 24.83 6.17 May 37 28.50 8.50 June 47 33.33 13.67 July 53 41.00 12.00 August 62 48.33 13.67 September 54 56.50 2.50 October 36 56.50 20.50 November 32 46.34 14.34 December 29 37.01 8.01 99.35 11.04

Sum = MAD =

All the weights sum to 1 Use Solver to find the optimal weights Figure 16, p293

Weighted n-Period Moving Averages resolves philosophical shortcoming of simple period moving average forecasting

Exponential Smoothing resolves operational shortcoming of simple period moving average forecasting

13

Exponential Smoothing

Exponential Smoothing Forecast for t + 1

Observed in t

Forecast for t

yˆ t +1 = αy t + (1 − α )yˆ t

α

Where

Resolves operational shortcoming of the Moving Averages Model: Number of 8-period Moving Inventory Items Average Model to Forecast

5,000

0.500

alpha =

Actual Sales (000) Fcst Sales Absolute Error Month January 20 20.00 February 24 20.00 4.00 March 27 22.00 5.00 April 31 24.50 6.50 May 37 27.75 9.25 June 47 32.38 14.63 July 53 39.69 13.31 August 62 46.34 15.66 September 54 54.17 0.17 October 36 54.09 18.09 November 32 45.04 13.04 December 29 38.52 9.52

Exponential Smoothing Model

40,000: 5,000 * 8

is a user-specified constant

10,001: 5,000 yˆ t 5,000 y t 1α

Saving alpha and the last forecasts stores all the previous forecasts. When t = 1,

the expression becomes:

yˆ t +1 = αy t + (1 − α )yˆ t

VARIABLE yt

Does exponential smoothing produce a better forecast?

yˆ 2 = αy t + (1 − α )yˆ t

COEFFICIENT

y t-1

α α(1-α)

y t-2

α(1-α)

2 3

y t-3

α(1-α)

y t-4

α(1-α)

y t-5

α(1-α)

y t-6

α(1-α)

4 5 6 7

y t-7

α(1-α)

y t-8

α(1-α)

y t-9

α(1-α)

y t-10 Sum of the Weights

109.17 9.92

Sum = MAD =

8 9

10

α(1-α)

α = 0.1 0.1

α = 0.3 0.3

α = 0.5 0.5

0.09

0.21

0.25

0.081

0.147

0.125

0.07290

0.10290

0.06250

0.06561

0.07203

0.03125

0.05905

0.05042

0.01563

0.05314

0.03529

0.00781

0.04783

0.02471

0.00391

0.04305

0.01729

0.00195

0.03874

0.01211

0.00098

0.03487 0.68619

0.00847 0.98023

0.00049 0.99951

The value of alpha affects the performance of the model

Case 1: Response to Sudden Change System Change when t = 100 2 1

yt

Response to a Unit Change in y t

1 0 -1 94

95

96

97

98

99

t

100

101

102

103

1.2 1.1 1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0 100

105

110

115

120

125

t

A forecasting system with alpha = 0.5 responds quickly to changes in the data.

14

Case 2: Response to Steady Change

Case 2: Response to Steady Change

Steadily Increasing Values of yt (Linear Ramp)

Steadily Increasing Values of yt (Linear Ramp)

yt

6 5 4 3 2 1 0

yt

0

2

4

6

8

10

6 5 4 3 2 1 0 0

t

2

4

6

8

10

t

Exponential smoothing is not a good forecasting tool in a rapidly growing or a declining market.

But the model can be adjusted (Holt’s model / exponential smoothing w/trend)

Case 3: Response to Seasonal Change Seasonal Pattern in y t 1.5 1.3 1.1 0.9

yt

0.7 0.5 0.3 0.1 -0.1 0

1

2

3

4

5

6

7

8

9

10 11 12 13 14 15 16 17

t

Seasonality Exponential smoothing is not a good model to use here because it ignores the seasonal pattern.

Takes into consideration and adjusts for the seasonal patterns in data

1. Look at original data to see seasonal pattern. Examine the data & hypothesize an m-period seasonal pattern.

15

2. Deseasonalize the Data

3. Forecast using deseasonalized data

4. Seasonalize the forecast to account for the seasonal pattern

Gillett Coal Mine

Coal Receipts Over a Nine-Year Period

Deseasonalized Data

3,000

3,000.0 2,500.0 Coal (000 Tons)

Coal (000 Tons)

2,500

2,000

1,500

1,000

2,000.0 1,500.0 1,000.0 500.0 -

500

1- 1- 1- 1- 2- 2- 2- 2- 3- 3- 3- 3- 4- 4- 4- 4- 5- 5- 5- 5- 6- 6- 6- 6- 7- 7- 7- 7- 8- 8- 8- 8- 9- 9- 9- 91 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 9-3

9-1

8-3

8-1

7-3

7-1

6-3

6-1

5-3

5-1

4-3

4-1

3-3

3-1

2-3

2-1

1-3

1-1

0

Tim e (Year & Qtr)

Tim e (Ye ar and Quarte r)

1. Look at original data to see seasonal pattern. Examine the data & hypothesize an m-period seasonal pattern. Figure 27, p303

2. Deseasonalize the Data

Figure 32, p306

16

2.Deseasonalize the Data

Time Coal 4 Period Year-Qtr Receipts Moving Average 1-1 2,159 ----1-2 1,203 ----1-3 1,094 1,613 1-4 1,996 1,594 2-1 2,081 1,626 2-2 1,332 1,721 2-3 1,476 1,856 (2,159+1,203+1,094+1,996)/4 2-4 2, 533 1,898 3-1 2,249 1,948 3-2 1,533 2,063 3-3 1,935 2,060 3-4 2,523 2,050 4-1 2,208 2,066

a) Calculate a series of m-period moving averages, where m is the length of the seasonal pattern. b) Center the moving average in the middle of the data from which it was calculated. c) Divide the actual data at a given point in the series by the centered moving average corresponding to the same point. d) Develop seasonal index e) Divide actual data by the seasonal index

= 1,613

a) Calculate a series of m-period moving averages, where m is the length of the seasonal pattern. Figure 28, p304

4 Period Moving Average

Data & Centered Moving Average

Centered Moving Average 3,000

1,613 1,594 1,626 1,721 1,856 1,898 1,948 2,063 2,060 2,050 2,066 2,061 2,112 2,213

1,603 1,610 1,674 1,788 1,877 1,923 2,005 2,061 2,055 2,058 2,064 2,087 2,163 2,255

Receipts

2,500

(1613 + 1594)/2 = 1603

Coal (000 Tons)

Time Coal Year-Qtr Receipts 1-1 2,159 1-2 1,203 1-3 1,094 1-4 1,996 2-1 2,081 2-2 1,332 2-3 1,476 2-4 2,533 3-1 2,249 3-2 1,533 3-3 1,935 3-4 2,523 4-1 2,208 4-2 1,597 4-3 1,917 4-4 2,726

Centered Moving Average

2,000

1,500

1,000

500

0 1- 1- 1- 1- 2- 2- 2- 2- 3- 3- 3- 3- 4- 4- 4- 4- 5- 5- 5- 5- 6- 6- 6- 6- 7- 7- 7- 7- 8- 8- 8- 8- 9- 9- 9- 91 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4

Time (Ye ar & Qtr)

b) Center the moving average in the middle of the data from which it was calculated.

b) Center the moving average in the middle of the data from which it was calculated. Figure 29, p305

Figure 28, p304

Time Coal Year-Qtr Receipts 1-1 2,159 1-2 1,203 1-3 1,094 1-4 1,996 2-1 2,081 2-2 1,332 2-3 1,476 2-4 2,533 3-1 2,249 3-2 1,533 3-3 1,935 3-4 2,523

4 Period Moving Average

Centered Moving Average

Ratio of Coal Receipts to Centered Moving Average

1,613 1,594 1,626 1,721 1,856 1,898 1,948 2,063 2,060 2,050

1,603 1,610 1,674 1,788 1,877 1,923 2,005 2,061 2,055 2,058

0.682 1.240 1.244 0.745 0.787 1.317 1.122 0.744 0.942 1.226

1,094 / 1,603 = 0.682

c) Divide the actual data at a given point in the series by the centered moving average corresponding to the same point. Figure 28, p304

Ti m e C o al Y e ar - Q tr R ec e i pt s 1-1 2,159 1-2 1,203 1-3 1,094 1-4 1,996 2-1 2,081 2-2 1,332 2-3 1,476 2-4 2,533 3-1 2,249 3-2 1,533 3-3 1,935 3-4 2,523

4 P e ri o d Mo v i ng A ve r a ge

1,613 1,594 1,626 1,721 1,856 1,898 1,948 2,063 2,060 2,050

C e n te r ed Ra t i o o f C o al R ec e ip t s t o S e as o n al M o vi n g A v er a g e C e n te r ed M ov i n g A v er a ge I n di c e s 1.112 0.786 1,603 0.682 0.863 1,610 1.240 1.238 1,674 1.244 1.112 1,788 0.745 0.786 1,877 0.787 0.863 1,923 1.317 1.238 2,005 1.122 1.112 2,061 0.744 0.786 2,055 0.942 0.863 2,058 1.226 1.238

d) Develop seasonal index for each quarter • Group ratios by quarter • Average all of the ratios to moving averages quarter by quarter • Add Seasonal Indices data to table • Normalize the seasonal index

17

Deseasonalized Data 3,000.0

2,000.0 1,500.0 Deseasonalized Data

1,000.0 500.0

93

83 91

71 73 81

61 63

51 53

41 43

-

31 33

1,603 1,610 1,674 1,788 1,877 1,923 2,005 2,061 2,055 2,058

Deseasonalized Data 1,941.0 1,529.8 1,267.7 1,611.9 1,870.9 1,693.8 1,710.3 2,045.6 2,021.9 1,949.4 2,242.2 2,037.5

21 23

1,613 1,594 1,626 1,721 1,856 1,898 1,948 2,063 2,060 2,050

Ratio of Coal Receipts to Seasonal Centered Moving Average Indices 1.112 0.786 0.682 0.863 1.240 1.238 1.244 1.112 0.745 0.786 0.787 0.863 1.317 1.238 1.122 1.112 0.744 0.786 0.942 0.863 1.226 1.238

13

Centered Moving Average

11

4 Period Moving Average

Coal (000 Tons)

2,500.0 Time Coal Year-Qtr Receipts 1-1 2,159 1-2 1,203 1-3 1,094 1-4 1,996 2-1 2,081 2-2 1,332 2-3 1,476 2-4 2,533 3-1 2,249 3-2 1,533 3-3 1,935 3-4 2,523

Tim e (Ye ar & Qtr)

e) Divide actual data by the seasonal index

e) Divide actual data by the seasonal index

Figure 31, p306

Time Coal Year-Qtr Receipts 1-1 2,159 1-2 1,203 1-3 1,094 1-4 1,996 2-1 2,081 2-2 1,332 2-3 1,476 2-4 2,533 3-1 2,249 3-2 1,533 3-3 1,935 3-4 2,523

4 Period Moving Average

Centered Moving Average

1,613 1,594 1,626 1,721 1,856 1,898 1,948 2,063 2,060 2,050

1,603 1,610 1,674 1,788 1,877 1,923 2,005 2,061 2,055 2,058

Ratio of Coal Receipts to Seasonal Centered Moving Average Indices 1.108 0.784 0.682 0.860 1.240 1.234 1.244 1.108 0.745 0.784 0.787 0.860 1.317 1.234 1.122 1.108 0.744 0.784 0.942 0.860 1.226 1.234

Deseasonalized Data Forecast 1,948.1 1,948.1 1,535.4 1,948.1 1,272.3 1,678.5 1,617.8 1,413.1 1,877.8 1,546.8 1,700.0 1,763.0 1,716.6 1,721.9 2,053.1 1,718.4 2,029.3 1,937.1 1,956.5 1,997.4 2,250.4 1,970.7 2,045.0 2,153.4

3. Forecast method in deseasonalized terms • Review the graphed deseasonalized data to reveal pattern • Use forecasting method that accounts for the pattern in the deseasonalized data • Use Excel’s Solver to minimize the error

Figure 32, p306

Time Coal Year-Qtr Receipts 1-1 2,159 1-2 1,203 1-3 1,094 1-4 1,996 2-1 2,081 2-2 1,332 2-3 1,476 2-4 2,533 3-1 2,249 3-2 1,533 3-3 1,935 3-4 2,523

4 Period Moving Average --------1,613 1,594 1,626 1,721 1,856 1,898 1,948 2,063 2,060 2,050

Centered Moving Average --------1,603 1,610 1,674 1,788 1,877 1,923 2,005 2,061 2,055 2,058

Ratio of Coal Receipts to Seasonal Centered Moving Average Indices ----1.108 ----0.784 0.682 0.860 1.240 1.234 1.244 1.108 0.745 0.784 0.787 0.860 1.317 1.234 1.122 1.108 0.744 0.784 0.942 0.860 1.226 1.234

Deseasonalized Seasonalize Data Forecast Forecast 1,948.1 1,948.1 2,159.000 1,535.4 1,948.1 1,526.409 1,272.3 1,678.5 1,443.212 1,617.8 1,413.1 1,743.439 1,877.8 1,546.8 1,714.276 1,700.0 1,763.0 1,381.390 1,716.6 1,721.9 1,480.540 2,053.1 1,718.4 2,120.128 2,029.3 1,937.1 2,146.723 1,956.5 1,997.4 1,564.974 2,250.4 1,970.7 1,694.495 2,045.0 2,153.4 2,656.854

4. Reseasonalize the forecast to account for the seasonal pattern • Multiply the deseasonalized forecast by the seasonal index for the appropriate period. • Graph the actual Coal Receipts and Seasonalized Forecast

Figure 33, p307

1. Look at original data to see seasonal pattern. Examine the data & hypothesize an m-period seasonal pattern.

Actual & Forecast

Coal (000 Tons)

3,500 3,000

2. Deseasonalize the data.

2,500 2,000 1,500 1,000 500 0

Coal Receipts Seasonalized Forecast 1- 1- 1- 1- 2- 2- 2- 2- 3- 3- 3- 3- 4- 4- 4- 4- 5- 5- 5- 5- 6- 6- 6- 6- 7- 7- 7- 7- 8- 8- 8- 8- 9- 9- 9- 9- 1 1 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 0- 01 2

Year-Quarter

4. Reseasonalize the forecast to account for the seasonal pattern

a) Calculate a series of m-period moving averages, where m is the length of the seasonal pattern. b) Center the moving average in the middle of the data from which it was calculated. c) Divide the actual data at a given point in the series by the centered moving average corresponding to the same point. d) Develop seasonal index e) Divide actual data by the seasonal index 3. Forecast method in deseasonalized terms. 4. Reseasonalize the forecast to account for the seasonal pattern.

18

SUMXMY2( ) Returns the sum of squares of differences of corresponding values in two arrays. Syntax: SUMXMY2(array_x,array_y), where Array_x is the first array or range of values. Array_y is the second array or range of values. The equation for the sum of squared differences is:

SUMXMY2 =

∑ (x − y )2

Measures of Comparison MAD =

MAPE =

∑

number of forecasts

∑

all forecasts

n

MSE =

actual sales − forecast sales

all forecasts

actual sales − forecast sales actual sales number of forecasts

∑ ( actual

∗ 100 %

sales − forecast sales ) 2

t =1

number of forecasts

Model Validation

Create experience by simulating the past.

Create the model with a portion of the historical data.

Use remaining data to see how well the model would have performed.

19

Qualitative Forecasting Models

Expert Judgment

Consensus Panel

Delphi Method

Coordinator requests forecasts

Delphi Method

Coordinator receives Individual forecasts Coordinator determines (a) Median response (b) Range of middle 50% of answers Coordinator sends to all experts (a) Median response (b) Range of middle 50% (c) Explanations

Coordinator requests explanations from any expert whose estimate is not in the middle 50%

Grassroots Forecasting

20

Market Research

21