New Approch For Stlf With Autoregressive And Ann Models

International Journal of Computational Intelligence Research. ISSN 0973-1873 Vol.3, No.1 (2007), pp. 66–71 c Research India Publications http://www.ijcir.info °

A New Approach for the Short-Term Load Forecasting with Autoregressive and Artificial Neural Network Models Ummuhan Basaran Filik1 , Mehmet Kurban2 1 2

, Anadolu University, Department of Electric Engineering, Eskisehir,Turkey ubasaran,[email protected]

Abstract: In this paper, a new approach to the short-term load forecasting using autoregressive (AR) and artificial neural network (ANN) models is introduced and applied to the power system of Turkey by using the consumption values of electrical energy for three months in 2002, including January, February, and March. The load forecasting for the next day using AR and ANN models is performed separately, and the results of the AR analysis is used for the input of different ANN models, which are Feed Forward Back Propagation and Cascade Forward Back Propagation Models. The performance of these models was compared with each other. When the energy consumption is examined for the whole week, it was observed that Sundays are different from other six days in the weeks. Because of this, the values for the past six days except Sunday are used for the load forecasting of the next day. For the system that uses ANN models only, the network is composed of a 6-neuron input layer and a 1-neuron output layer; for the systems that use AR and ANN models, there are 7 neurons in the input layer and one neuron in the output layer. It was found that systems that use both AR and ANN models can achieve a higher forecasting accuracy.

I. Introduction Load forecasting is very important for the power system planning and security. The main problem for the planning is the determination of load demand in the future. Because electrical energy cannot be stored appropriately, correct load forecasting is very important for the correct investments. There are three types of load forecasting: Short-term, middle-term and long-term load forecasting. Short-term load forecasting is to predict the hourly loads, one day or even one week. Short-term load forecaster calculates the estimated load for each hours of the day, the daily peak load, or the daily or weekly energy generation. Short term load forecasting is important for the economic and secure operation of

power systems [1]. Many methods have been developed for the forecasting. They are based on various statistical methods such as regression [2-3], Box Jenkins model [4], exponential smoothing [5] and Kalman filters [6]. However, these methods cannot represent the complex nonlinear relationships [4]. Many electric power companies have adopted conventional prediction methods for load forecasting. However, these methods can not properly represent the complex nonlinear relationships that exist between the load, and series of factors that influence it [7]. Recently, artificial neural networks (ANN) have been successfully applied to short term load forecasting [814]. In this paper, a new approach in which both autoregressive (AR) and artificial neural network (ANN) models are used for the short-term load forecasting is introduced. The results of the AR analysis are used for the input of different ANN models which are Feed Forward Back Propagation and Cascade Forward Back Propagation Models. All of the methods are compared with each other. Firstly, the whole models are examined one by one. Then the applications and simulations for the system is given.

II. Auto Regressive Models These are the models that explain a time series value in a time period and its relationship with the previous values and error term. AR models are named with the number of previous time period terms. An autoregressive process AR(p) with order p is defined as Xt =

p X

φr Xt−r + ²t

(1)

r=1

where φ1 , φ2 ...φr are fixed constants and ²t is a sequence of independent (or uncorrelated) random variables with zero

Short-Term Load Forecasting

67

mean and variance σ 2 [15]. It contains only one past observation value, called as AR(1). The AR(1) process is defined by (2) Xt = φr Xt−1 + ²t

III. Artificial Neural Network Models A neural network is massively parallel-distributed processor made up of simple processing units called neurons, which have a natural propensity for storing experimental knowledge [16]. The motivation for the development of neural network technology stems from the desire to develop an artificial system that could perform “intelligent” tasks similar to those performed by the human brain. Neural networks resemble the human brain in the following two ways:

Figure. 1: The structure of AR-input ANN 4

2

x 10

1.9

1. A neural network acquires knowledge through learning.

1.8

2. A neural network’s knowledge is stored within interneuron connection strength known as synaptic weights.

1.7

MW

1.6

The true power and advantage of neural networks lies in their ability to represent both linear and non-linear relationships and in their ability to learn these relationships directly from the data being modeled. Traditional linear models are simply inadequate when it comes to modeling data that contains non-linear characteristics. The most common neural network model is the multilayer perceptions (MLP). This type of neural network is known as supervised networks because they require a desired output in order to learn. The goal of this type of network is to create a model that correctly maps the input to the output using historical data so that the model can then be used to produce the output when the desired output is unknown [17].

IV. A New Approach for the ANN Models A new approach is to use the results of the AR analysis for the input of different ANN models which are Feed Forward Back Propagation and Cascade Forward Back Propagation Models. The structure of the new ANN model is given in Figure 1.

V. Applications and Simulations In the application, the load forecasting for the next day using AR and ANN models is performed separately; then results of the AR analysis is used for the input of different ANN models which are Feed Forward Back Propagation and Cascade Forward Back Propagation Models. All of them are compared with each other. When the the energy consumption of the whole week is examined, it was observed that Sundays are different from other six days in the weeks. Because of this, the values for the past six days except Sunday are used for the load forecasting of the next day. Thus the correction of the neural network is provided. Firstly, hourly load values

1.5 1.4 1.3 1.2 January February March

1.1 1

0

20

40

60

80

100

120

140

160

180

Hours

Figure. 2: The hourly load values of a week for January, February and March are determined. The hourly load values for January, February, and March are shown in Figure 2. The filter coefficient is set to 6 for this analysis. If we analyze the pole zero plot of the AR(6) filter, one of the pole is near the unit circle as shown in Figure 3. This state shows that AR model can not be very suitable for this analysis. In Figure 2, hourly load values (MW) for one week are plotted for January, February and March. The data sequence has some periodicity but the statistical values such as bias and variance are changing from one month to the other. The daily data are similar from one day to the other also. This introduces the periodicity. But some unexpected events such as holidays, failure on power plants, whether condition changing, etc. effect the loads. Therefore, if we make load forecasting by linear prediction with AR, these unexpected changes decrease the prediction performance. In this model (based on the parametric methods), it is assumed that the data sequence is stationary. But the data is not stationary as can be observed in Figure 4 where the statistical characteristics are changing rapidly. A. Forecasting with AR The AR models explain time series values in a time period, their relationship with the previous values, and an error term.

Ummuhan Basaran Filik and Mehmet Kurban

68 4

x 10

2

1

1.9

0.8 1.8

0.4

1.7

0.2

1.6

MW

Imaginary Part

0.6

0

1.5

−0.2 1.4

−0.4 −0.6

1.3

−0.8

1.2 Predicted Load Values Actual Load Values

−1 1.1

−1

−0.5

0 Real Part

0.5

1

0

50

100

150

200

250

300

350

400

450

500

Hours

Figure. 5: Actual and predicted load values for January

Figure. 3: Pole zero plot of the AR (6) filter 20

Performance is 267759, Goal is 0

10

January February March

10

15 9

10

10 8

Training−Blue

Power(dB)

10

5

0

7

10

6

10

−5

5

−10

−15

10

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

4

10

1

Normalized Frequency

0

20

40

60

80

100 199 Epochs

120

140

160

180

Figure. 4: Pseudo spectrum estimate via music (multiple signal classification method) (Order=6)

Figure. 6: Curve showing the training process of the feed forward backpropagation network.

AR models are named with the number of previous time period terms. Denote

B. Forecasting with Artificial Neural Network Models

Y486×6 as data matrix y486×1 as target vector. We have,  y(1)  y(2)   ..  . y(480)



... ... .. .

y(6) y(7) .. .

...

y(485)

   

a(1) a(2) .. . a(6)





    =  

y(7) y(8) .. .

    (3) 

y(486)

Ya=y AR coeffients = a ≈ (Y H Y )−1 Y H y We can find AR coefficients by using the least squares method. AR model is applied to the system. Actual load values and predicted load values are shown in Figure 5 for January 1-20. For this analysis, the hourly load values for January are used. The AR(6) filter is constructed. The program is written in MATLAB for AR analysis. Using the results of the AR analysis, actual and predicted load values for January are shown in Figure 5.

ANN is trained such that a particular input leads to a specific target output. There are generally four steps in the training process: (1) assemble the training data, (2) create the network object, (3) train the network, and (4) compute the network response to new inputs. Different neural network structures are tested for this system and results of this analysis are compared. All the neural network structures have input layer composed of 6 neurons and output layer with 1 neurons. Feed Forward Backpropagation network is selected as the network type. The network is implemented by using the MATLAB Neural Networks Toolbox. Size of the input vector is 6 × 480, and the size of the target vector is 1 × 480 in this structure. Neural Network is trained for 199 epochs, as illustrated in the Figure 6. The actual and predicted load values of the feed forward backpropagation network is shown in Figure 7. The Cascade Forward Back Propagation has 5 layers. The first layer has 6 neurons; the second, third, and fourth layers have 4 neurons, and the last layer has 1 neuron. Neural Network is trained for 50 epochs, as illustrated in Figure 8. The actual and predicted load values of the Cascade Forward Back Propagation network is shown in Figure 9.

Short-Term Load Forecasting

69

4

4

x 10

2

x 10

2

1.8

1.8

1.7

1.7

1.6

1.6

MW

1.9

MW

1.9

1.5

1.5

1.4

1.4

1.3

1.3

1.2

1.2 Predicted Load Values Actual Load Values

1.1

0

50

100

150

200

250

300

350

400

450

Predicted Load Values Actual Load Values 1.1

500

0

50

100

150

200

250

Hours

350

400

450

500

Hours

Figure. 7: Actual load values and predicted load values for January for the feed forward backpropagation network.

Figure. 9: Actual and predicted load values for January for the Cascade Forward Back Propagation Performance is 240985, Goal is 0

10

Performance is 175621, Goal is 0

9

300

10

10

9

10 8

10

8

Training−Blue

Training−Blue

10 7

10

7

10

6

10

6

10

5

10

5

10

4

10

4

10

0

5

10

15

20

25

30

35

40

45

50

0

2

4

Figure. 8: Curve showing the training of the cascade forward backpropagation network.

C. Forecasting with a New Approach In this part, the correction of neural networks by giving the results of AR model as the input to the ANN model is applied to the system for January as an example application. The structure has 2 layers; the first layer is composed of 6 neurons and the output layer is composed of one neuron. Feed Forward Back propagation is chosen as the network type. In this case, the size of the input vector is 7 × 480 and the size of the target vector is 1 × 480. The network is trained for 18 epochs. The curve of the epoch number and training for the feed forward back propagation structure is shown in Figure 10. Actual and predicted load values are shown in Figure 11. As can be observed in this figure, Feed Forward Back propagation with AR analysis has the less performance value than the one without AR. The number of the epochs required is also less than that of the previous ones. Neural network structure for Cascade Forward Back Propagation has 5 layers. The first layer has 7 neurons; each of second, third and forth layers has 4 neurons, and the output layer has 1 neuron. The size of the input vector is 7×480 and size of the target vector is 1 × 480 in this structure. The correction of ANNs and the actual and predicted load values for the cascade forward back propagation with AR for January

6

8

10

12

14

16

18

18 Epochs

50 Epochs

Figure. 10: Curve showing the training process of the second structure. are shown in Figures 12 and 13, respectively.

Mean square error is calculated according to the following formula: N 1 X (xi − x ˆ i )2 (4) MSE = N i=1 where N is the number of data points, xi is the actual Value, and x ˆi is the predicted Value. MSE values of the all analysis are given in Table 1. As can be observed from the results, models with Cascade Feed Forward Back Propagation neural network structure gives the best results because mean square error value is less than those of others.

VI. Conclusion AR and multilayer artificial neural networks are proposed for short-term load forecasting in this paper. Past six days were used except Sunday, and the next day was forecasted. In the first approach, an AR model is used for load forecasting. Then, different neural network structures were tested. Network types that have been tested include Feed Forward Back

Ummuhan Basaran Filik and Mehmet Kurban

70 4

4

x 10

2

2

1.8

1.8

1.7

1.7

1.6

1.6

MW

1.9

MW

1.9

x 10

1.5

1.5

1.4

1.4

1.3

1.3 Predicted Load Values

1.2

1.1

Predicted Load Values

Actual Load Values

0

50

100

150

200

250

300

350

1.2

400

450

1.1

500

Actual Load Values

0

50

100

150

200

Hours

250

300

350

400

450

500

Hours

Figure. 11: The correction of ANNs for January

Figure. 13: Actual and predicted load values for January for the cascade forward back propagation with AR.

Performance is 158650, Goal is 0

10

10

Table 1: MSE values of the analysis

9

10

Method 8

Training−Blue

10

Forecasting with AR Forecasting with ANN Models BP Alghm. Forecasting with ANN Models Cascade Forward BP Alghm. Forecasting with a New Approach Feed Forward BP Alghm. Forecasting with a New Approach Cascade Forward BP Alghm.

7

10

6

10

5

10

4

10

0

1

2

3

4

5 10 Epochs

6

7

8

9

10

MSE values (GW) 39.862 26.776 21.975 20.078 18.085

Figure. 12: Correction neural networks for January.

Propagation and Cascade Forward Back Propagation. All of the neural network structures comprise six neurons in the input and one neuron in the output. Cascade Forward Back Propagations was found to be more efficient than Feed Forward Back Propagation Method. Lastly, the neural networks were corrected by applying the result of the AR model to their inputs. The best result is obtained by the Cascade Feed Forward Back Propagation neural networks. Results show that ANNs are feasible solutions and valuable approaches to short-term load forecasting.

References

[5]

Huang, H., Hwang, R.,Hsieh, J.: A new artificial intelligent peak power load forecaster based on non-fixed neural networks, Electr. Power Energy Sys., (2002) 245250

[6]

Irisarri, G.D., Widergren, S.E. Yehsakul, P.D: Online load forecasting for energy control center application, IEEE Transactions on Power Apparatus and Systems,PAS 101 (1971)900-911

[7]

Christiaanse, W.R.: Short-term load forecasting using general exponential smoothing, IEEE Transactions on Power Apparatus and Systems, PAS-90, (1971) 900911

[8]

Park, D.C., El-Sharkawi, M.A., Marks R.J., Atlas L.E., M.J. Damborg: Electric Load Forecasting Using and Artificial Neural Network, IEEE Transactions on Power Systems, Vol. 6, No 2 (1991), 442-449

[9]

Peng,T.M., Hubele, N.F., Karady, G.G. :Advancement in the Application of Neural Networks for Short Term Load Forecasting, IEEE Transactions on Power Systems, Vol. 7(1992) 250-258

[1] Gross, G., Galiana F.D.: Short Term load forecasting. Proc. IEEE, Vol. 75 (1987) 1558-1573 [2] Asbury, C. : Weather Load Model for Electric Demand Energy Forecasting, IEEE Transactions on Power Apparatus and Systems, Vol. PAS-94,(1975) 1111-1116 [3] Papalexopoulos, A.D., Hesterberg, T.C.: A regression based approach to short-term system load forecasting, Proceedings of PICA Conference (1989) 414-423 [4] Hill, T., OConnor, M., Remus, W., :Neural Networks Models for Time Series Forecasts, Management Sciences, (1996) 1082-1092

[10] Lu, C.N., Wu, H.T., Vemuri, S.: ”Neural Network Based Short Term Load Forecasting, IEEE Transactions on Power Systems, Vol. 8 (1993) 336-342 [11] Peng, T.M., Hubele, N.F., Karady ,G.G.: An Adaptive Neural Network Approach to One-Week ahead

Short-Term Load Forecasting Load Forecasting IEEE Transactions on Power Systems, Vol.8 (1993)1195-1203 [12] Papalexopoulos, A.D., How, S., Peng, T.M.: An Implementation of a Neural Network based Load Forecasting Model for the EMS, Paper 94 WM 209-7 PWRS presented at the IEEHES (1994) Winter Meeting [13] Mohammed, 0, Park, D., Merchant, R., Dinh, T., Tong, C., Azeem, A., Farah J., Drake C.: Practical Experiences with an Adaptive Neural Network Short Term Load Forecasting System, Paper 94 WM 210-5 PWRS presented at the IEEE PES (1994) Winter Meting [14] Senjyu, T., Mandal, P., Uezato, K., Funabashi, T. :Next day load curve forecasting using recurrent neural network structure, IEE Proc.Gener. Transm. Distrib., Vol. 151 (2004) [15] http://www.statistics.com/content/glossary /a/armamodel.php [16] Chauhan, B.K., Sharma, A., Hanmandlu, M.: Neuro Fuzzy Approach Based Short Term Electric Load Forecasting, IEEE/PES Transmission and Distribution Conference & Exhibition: Asia and Pacific Dalian, China (2005 ) [17] http://www.nd.com/neurosolutions/products/ns /whatisNN.html

71