Machine Learning - The Application Of Radial Basis Functions

1

THE APPLICATION OF RADIAL BASIS FUNCTIONS AND SUPPORT VECTOR MACHINES TO THE FOETAL WEIGHT PREDICTION

FERNANDO SERENO FEUP – Faculdade de Engenharia da Universidade do Porto and INEB Instituto de Engenharia Biomédica, Porto, Portugal ([email protected])

JOAQUIM P. MARQUES DE SÁ FEUP – Faculdade de Engenharia da Universidade do Porto and INEB Instituto de Engenharia Biomédica, Porto, Portugal

ANA MATOS HSJ – Hospital de S. João, Dep. Ginecologia e Obstectrícia, Porto, Portugal

JOÃO BERNARDES FMUP – Faculdade de Medicina da Universidade do Porto and INEB Instituto de Engenharia Biomédica, Porto, Portugal

ABSTRACT Foetal weight prediction based on echographic features is an important procedure in perinatal medicine. Classical methods of foetal weight prediction have serious shortcomings in current clinical practice. We investigated the application of Radial Basis Functions (RBF) and Support Vectors Machines (SVM) neural networks in order to predict foetal weights in a reliable way. A RBF was trained using a set of 220 input vectors of echographic features spanning a foetal weight range from 1500 to 4500 grams and was tested in a separate set of 55 cases with similar distribution. The overall absolute relative error attained a reasonable 6.2%. However, for foetal weights greater than 4000 grams the relative error was of the order of minus 10%, underestimating foetal weights, a problem we tried to solve using a SVM classifier. Keywords: Artificial Neural Networks, Pattern Recognition, Bio-Medical Engineering Applications, Prediction, Radial Basis Functions, RBF, Support Vector Machines, SVM.

INTRODUCTION

Pre-natal foetal weight prediction is an important part of obstetric and neonatal management, since foetuses who have not grown properly may have a higher perinatal mortality rate and are namely at high risk for neurological problems. Traditional formulas to estimate the foetal weight take at least two echographic measurements: the abdominal circumference (AC) and the femur length (FL) or the biparietal diameter (BPD). These formulas were derived from linear generalized models by Hadlock and Shepard (Farmer et al., 1992). The need for a quick and easy method for estimating foetal weight has been clearly established. Results of a statistical analysis and multiple multivariable linear regressions showed that: (i) the birth weight is a logarithmic function of foetal body parameters and that the abdominal circumference has the single best correlation with the log10 of the birth weight; (ii) linear regression with the use of two foetal dimensions (abdominal circumference and biparietal diameter) had

2

a standard deviation of 106 grams per kilogram foetal weight (Warsof et al., 1977). Farmer et al. (1992) used a neural network model for the ultrasonic estimation of foetal weight in the macrosomic foetus and obtained an average error of 4.7% from actual birth weight, statistically better (p=0.001) than the results obtained from regression models. Chauhan et al. (1998) compared the accuracies of ultrasonographic estimates of birth weights among infants born between 24 and 34 weeks of gestation at three tertiary centers and concluded that “ultrasonographic estimates for preterm infants, as obtained from 26 equations are characterized by a rather wide range of accuracy, [and] for most of the equations the accuracies of estimates differ markedly among centers.” The objective of the present prospective study is to investigate the application of Radial Basis Functions (RBF) and Support Vector Machines (SVM) neural networks in order to predict foetal weights. METHODS AND DATA

Echographic features taken on 414 pregnant voluntary women, within 7 days before birth, were collected by obstetricians in four Portuguese hospitals during 1998-2000. A mean gestational age at delivery of 39 (± 2) weeks was reported originating data that seems to be representative of the population in study. The frequencies in the birth weight categories 3000-3499 g (grams) and 3500-3999 g are bigger than other categories. Table 1 represents the birth weight frequency distribution and Table 2 depicts summary statistics of echographic measurements taken from the foetus, within 7 days before labour, and at birth measurements of weight, length and cephalic circumference. We selected 275 cases as appropriate for neural network training by eliminating example patterns deteriorated by missing data, outliers, erroneous or inconsistent data. We used a RBF algorithm for supervised learning of an approximation function over the foetal weight range 1500-4500 grams, and a SVM algorithm for supervised learning classification in the two-category of foetal weights 3500-4000g and greater than 4000g, and evaluated the performance by the sensitivity, specificity and overall accuracy. Birth Weight

1500200025001999 2499 2999 Frequency % 1.9 3.4 4.1 17.9 Table 1. Frequency distribution of 414 Portuguese birth weights < 1500

30003499 39.6

35003999 24.9

> 4000 8.2

Our verification of the accuracy and appropriateness of the echographic measurements for training neural networks, as is suggested by Hudson & Cohen_(2000; p.122), made necessary cases exclusion. APPROPRIATENESS OF RBF ALGORITHM AND PERFORMANCE EVALUATION

We used an RBF algorithm, developed by Bishop (1997) & Nabney (1999) for use within the MATLAB Neural Networks Tools (Hagan et al., 1996), that

3

uses a combination of unsupervised learning in a hidden layer, and a supervised learning technique in the output layer, schematically represented in Table 3, as a variant of Hudson & Cohen (2000; p.42) using the EM algorithm for the inputs to hidden layer weights determination (McLachlan & Krishnan,1997).

Echographic Measurements

Birth Measurements

Biparietal Diameter (BPD) Cephalic Circumference (CC) Abdominal Circumference (AC) Femur Length (FL) Umbilical Artery Resistance Index (URI) Weight

Mean 9.20 32.8 33.1 7.1 0.6 3,225

Stand. Dev. 0.6 1.8 3.1 0.5 0.1 628

Units cm cm cm cm

Length

48.6

2.9

cm

Cephalic Circumference

34.3

2.0

cm

g

Table 2. Descriptive statistics of 414 Portuguese infants

To improve performance we used pre-processing to reduce noise and inconsistent data: (i) the input space consisted of only four echographic measurements as far as our prior knowledge about fetal weight association and umbilical resistence index (URI) was uncertain; (ii) we filtered cases that were not in the neighbourhood of the centres of each category of foetal weights. Each case was characterised by a vector of three normalised features, AC, BPD and FL. The centre of each category was defined by a vector whose components were the averages of these normalised features of all cases belonging to the category. If in a given category a case had a distance to this centre greater than 1,5 standard deviations, then it was excluded. The RBF NNs had a hidden layer with 10 units and Gaussian basis functions, were trained by 220 cases and tested in a separated set of 55 cases, and gave the results that are summarised in the Table 5. Radial Basis Function (RBF) Algorithm Assign Connections Weights • Output layer weights assigned to small random numbers Initialise For the hidden layer • Use a small number of iterations of the k-means algorithm; • Determine hidden layer centres by fitting a Gaussian mixture model with circular covariances using the EM algorithm; • Set Gaussian activation functions widths to the maximum inter-centre squared distance. Iterate until convergence For the output layer • wij(t+1) = wij(t) + ∆wij where • ∆wij = η δi µ j where η is the learning rate, and • δi = Ti - µ i where Ti is the target output activation and µ i is the actual output activation at unit i. Repeat until convergence Table 3

4

USING SVM TO PREDICT HIGH FOETAL WEIGHTS

Support Vector Machines (SVM) have been successfully applied to a number of classification and regression tasks, similar to the foetal weight prediction (Odone et al., 1998; Veropoulos et al., 1999). The SVM algorithm is based on the theory of finite sample statistics, which shows that the critical quantity is not the number of parameters in the system, but the VapnikChervonenkis (Vapnik, 1995) dimension of the set of functions available to the system. Since its introduction the SVM theory has been developing and gaining popularity due to many attractive features and promising empirical performance. A detailed explanation of SVM would be beyond the scope of this paper, and can be found, for example, in Cristianiani (2000) or Haykin (1999; pp. 318-350). We used an SVC algorithm developed by Gunn (1998) for use within the MATLAB Neural Networks Tools. The main steps in the algorithm are summarised in the Table 4. The SVC algorithm • Construct the kernel matrix • Add a small amount of zero order regularization to avoid problems when Hessian is badly conditioned • Initialize optimization parameters • Solve the optimization problem by quadratic programming • Compute de number of Support Vectors Table 4

SVC classifiers were trained by a set T = {x i , d }iN=1 , where N = 72 inputs vectors, xi consisting of two echographic features, the abdominal circumference (AC) and another echographic measurement, FL, BPD, CC or URI. The target d is -1 if the corresponding foetal weight category is greater than 4000, and +1 otherwise. We experimented, among several possible kernels, an order 20 spline kernel matrix, and used in the Lagrangian a related regularization parameter C equal to 105 for optimisation and selection of the support vectors. The SVC classifiers were tested by a set with the number of cases N=37, separate from the learning one, and whose input vectors xi have the same subset of echographic features. SVM classifiers performance is summarised in Table 5. CONCLUSIONS

Whereas the foetal weight relative absolute error using Hadlock and Shepard formulas were 7.8% and 7.5%, respectively, using RBF NN we got the lowest relative absolute error, 6.2 %, as can be seen in Table 5. The sensitivities, specificities and accuracies provided in Table 5 were estimated for five 500 g foetal weight classes in order to give a detailed picture of the statistical validity of neural nets regarding birth weight prediction. This clearly underestimates the clinical validity, which would improve if calculated for the more relevant clinical classes of foetal weights higher or inferior to 3500 g and 1500 g, respectively.

5

RBF PREDICTED FETAL WEIGHT 4500

•-Real o - Predicted

4000

3500

3000

2500

2000

1500 0

10

20

30 # CASE

40

50

Fig. 1 Graphical representation of 55 real and estimated weights by an RBF with ten hidden Gaussian units and one output linear unit. Shows the real foetal weights ordered increasingly and represented by dots, and the corresponding estimated foetal weights represented by circles.

SVM Classification (3500-4500 g)

RBF Estimation and Classification (1000-4500 g)

Model Hadlock Shepard

Estimation Error 0.078 0.075 0.076 0.067

Sensitivity 0.60 0.60 0.73 0.64

Classification Specificity 0.52 0.55 0.61 0.60

Accuracy 0.55 0.56 0.64 0.57

AC, FL, BPD

0.062

0.69

0.53

0.61

AC, FL, BPD, CC

0.067

0.75

0.58

0.69

AC, FL, BPD, CC, URI AC, FL

0.068 N.A.

0.74 0.78

0.74 0.45

0.60 0.70

AC, BPD

N.A.

0.85

0.45

0.86

AC, CC

N.A.

0.85

0.27

0.68

AC, URI

N.A.

0.81

0.27

0.65

Inputs AC, FL AC, BPD AC, FL AC, BPD

Table 5 Estimation and classification Errors. AC – Abdominal Circumference. BPD – Biparietal Diameter. CC – Cephalic Circumference. FL – Femur Length. URI – Umbilical Resistance Index. RBF – Radial Basis Function. SVM – Support Vector Machine (Classifier). N.A. - Not Appropriate.

6

RBF neural nets performance gave in these experiments lower error ratios than multi-layer perceptron (MLP) solutions that we have experimented with a different training set and reported previously in Sereno et al. (2000). Nevertheless, as can be seen in Fig. 1, the RBF prediction sub-estimates the larger foetal weights, probably because this range is poorly represented in the data available. The RBF NN trained with 1000-4500 grams patterns may become biased either by the higher frequency of the mean foetal weights, or by noise that may be present in the big foetal weights range. Therefore, in this range, the false negatives could be an influence to a wrong prediction of the risk of emergency Cesarian section for a pregnant woman. We are currently improving the generalization performance of an RBFSVM-heuristic rules approach by re-scaling the input variables in proportion to their relative importance in the output, using prior knowledge in training strategies applied to RBF and SVM function approximation architectures, and reducing the variance by combining the outputs of MLP, RBF, SVM and knowledge-based artificial NN (Mitchell, 1997) to form committees that could effectively improve the accuracy of the foetal weight approximation. REFERENCES Bishop, C. M., 1997, Neural Networks for Pattern Recognition, Oxford, Oxford University Press. Chauhan S. P. et al., 1998, Ultrasonographic estimate of birth weight at 24 to 34 weeks: A multicenter study, Am J Obstet Gynecol October 1998. Cristianini N. & Shawe-Taylor J., 2000, An Introduction to Support Vector Machines: And Other Kernel-Based Learning Methods, Cambridge University Press. Farmer R.M., Medearis A.L., Hirata G.I., Platt L.D., 1992, The Use of a Neural Network for the Ultrasonographic Estimation of Foetal Weight in Macrosomic Fetus, Am J Obstet Gynecol May 1992. Gunn S., 1998, Support Vector Machines for Classification and Regression, Image Speech & Intelligent Systems Group, University of Southampton, United Kingdom. Hagan M.T., Demuth H.B., Beale M., 1996, Neural Network Design, Boston, PWS Publishing Company. Haykin S., 1999, Neural Networks - A Comprehensive Foundation (2d Edition), Prentice Hall. Hudson D.L. & Cohen M.E., 2000, Neural Networks and Artificial Intelligence for Biomedical Engineering, IEEE Press in Biomedical Engineering. McLachlan J. & Krishnan T., 1997, The EM Algorithm and Extensions, John Wiley & Sons, Inc. Mitchell T., 1997, Machine Learning, New York, McGraw Hill. Nabney I.T., 1999, Efficient Training of RBF Networks for Classification, Birmingham, Aston University. Odone F., Trucco E., Verri A., 1998, Visual Learning of Weight from Shape Using Support Vector Machines, British Machine Vision Conference. Sereno F., Marques de Sá J.P., Matos A., Bernardes J., 2000, A Comparative Study of MLP and RBF Neural Nets in the Estimation of the Foetal Weight and Length, in Campilho A., Mendonça A., 2000, Proceedings of RECPAD 2000 - 11th Portuguese Conference on Pattern Recognition, University of Porto. Vapnik V.N., 1995, The Nature of Statistical Learning Theory, Springer. Veropoulos K., Cristianini N., Campbell C., 1999, The Application of Support Vector Machines to Medical Decision Support : A Case Study. Department of Engineering Mathematics, Bristol University, United Kingdom. Warsof S.L. et al., 1977, The Estimation of fetal weight by computer-assisted analysis. Am J Obstet Gynecol Aug 15, 128:8, 881-92.