Malhotra Mr05 Ppt 19

Ch apter Ni nete en

Factor Analysis

© 2007 Prentice Hall

19-1

Cha pter Ou tl ine 1) Overview 2) Basic Concept 3) Factor Analysis Model 4) Statistics Associated with Factor Analysis

© 2007 Prentice Hall

19-2

Cha pter Ou tl ine 5) Conducting Factor Analysis i.

Problem Formulation

ii. Construction of the Correlation Matrix iii. Method of Factor Analysis iv. Number of of Factors v. Rotation of Factors vi. Interpretation of Factors vii. Factor Scores viii. Selection of Surrogate Variables © 2007 Prentice Hall

ix. Model Fit

19-3

Cha pter Ou tl ine 6) Applications of Common Factor Analysis 7) Summary

© 2007 Prentice Hall

19-4

Fac to r Ana lys is 

Fac tor anal ysi s is a general name denoting a class of procedures primarily used for data reduction and summarization.



Factor analysis is an int erd epe nden ce te chnique in that an entire set of interdependent relationships is examined without making the distinction between dependent and independent variables.



Factor analysis is used in the following circumstances: 

To identify underlying dimensions, or fac tors, that explain the correlations among a set of variables.



To identify a new, smaller, set of uncorrelated variables to replace the original set of correlated variables in subsequent multivariate analysis (regression or discriminant analysis).



To identify a smaller set of salient variables from a larger set 19-5 for use in subsequent multivariate analysis.

© 2007 Prentice Hall

Fac to rs Under lying S el ecte d Psy chogra phi cs a nd L ifestyl es Fi g. 19.1

Factor 2 Football

Baseball Evening at home Factor 1

Go to a party

Home is best place Plays

© 2007 Prentice Hall

Movies

19-6

Fac to r Ana lys is Mo de l

Mathematically, each variable is expressed as a linear combination of underlying factors. The covariation among the variables is described in terms of a small number of common factors plus a unique factor for each variable. If the variables are standardized, the factor model may be represented as:

Xi = Ai 1 F1 + Ai 2 F2 + Ai 3 F3 + . . . + Aim Fm + Vi Ui where

Xi Aij

= =

F Vi

= =

Ui = © 2007 Prentice m Hall =

i th standardized variable standardized multiple regression coefficient of variable i on common factor j common factor standardized regression coefficient of variable i on unique factor i the unique factor for variable i number of common factors

19-7

Fac to r An al ysi s Mo de l The unique factors are uncorrelated with each other and with the common factors. The common factors themselves can be expressed as linear combinations of the observed variables. Fi = Wi1X1 + Wi2X2 + Wi3X3 + . . . + WikXk Where:

© 2007 Prentice Hall

Fi

=

estimate of i th factor

Wi

=

weight or factor score coefficient

k

=

number of variables

19-8

Fac to r Ana lys is Mo de l 

It is possible to select weights or factor score coefficients so that the first factor explains the largest portion of the total variance.



Then a second set of weights can be selected, so that the second factor accounts for most of the residual variance, subject to being uncorrelated with the first factor.



This same principle could be applied to selecting additional weights for the additional factors.

© 2007 Prentice Hall

19-9

St at is tic s Asso cia ted with Facto r Analys is 

Bar tle tt's test of sp her icity. Bartlett's test of sphericity is a test statistic used to examine the hypothesis that the variables are uncorrelated in the population. In other words, the population correlation matrix is an identity matrix; each variable correlates perfectly with itself (r = 1) but has no correlation with the other variables (r = 0).

Co rrel ati on mat rix. A correlation matrix is a lower triangle matrix showing the simple correlations, r, between all possible pairs of variables included in the analysis. The diagonal elements, which are all 1, are usually omitted. © 2007 Prentice Hall 

19-10

St at is tic s Ass ocia ted with Facto r Analys is 

Co mmun al ity . Communality is the amount of variance a variable shares with all the other variables being considered. This is also the proportion of variance explained by the common factors.



Eig en val ue . The eigenvalue represents the total variance explained by each factor.



Fa ctor load ing s. Factor loadings are simple correlations between the variables and the factors.



Fa ctor load ing pl ot. A factor loading plot is a plot of the original variables using the factor loadings as coordinates.



Fa ctor mat rix. A factor matrix contains the factor loadings of all the variables on all the factors extracted.

© 2007 Prentice Hall

19-11

St at is tic s Ass ocia ted with Facto r Analys is 

Fac tor sc ores. Factor scores are composite scores estimated for each respondent on the derived factors.



Kais er- Mey er- Olkin (K MO) me asure of samp lin g ad equ ac y. The Kaiser-Meyer-Olkin (KMO) measure of sampling adequacy is an index used to examine the appropriateness of factor analysis. High values (between 0.5 and 1.0) indicate factor analysis is appropriate. Values below 0.5 imply that factor analysis may not be appropriate.



Pe rc ent ag e of v ar ian ce. The percentage of the total variance attributed to each factor.



Re si dual s are the differences between the observed correlations, as given in the input correlation matrix, and the reproduced correlations, as estimated from the factor matrix.

Scre e plot. A scree plot is a plot of the Eigenvalues against the number of factors in order of extraction. 19-12 © 2007 Prentice Hall 

Co nduc ti ng Fa ctor An alysi s Table 19.1

© 2007 Prentice Hall

RESPONDENT NUMBER 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30

V1 7.00 1.00 6.00 4.00 1.00 6.00 5.00 6.00 3.00 2.00 6.00 2.00 7.00 4.00 1.00 6.00 5.00 7.00 2.00 3.00 1.00 5.00 2.00 4.00 6.00 3.00 4.00 3.00 4.00 2.00

V2 3.00 3.00 2.00 5.00 2.00 3.00 3.00 4.00 4.00 6.00 4.00 3.00 2.00 6.00 3.00 4.00 3.00 3.00 4.00 5.00 3.00 4.00 2.00 6.00 5.00 5.00 4.00 7.00 6.00 3.00

V3 6.00 2.00 7.00 4.00 2.00 6.00 6.00 7.00 2.00 2.00 7.00 1.00 6.00 4.00 2.00 6.00 6.00 7.00 3.00 3.00 2.00 5.00 1.00 4.00 4.00 4.00 7.00 2.00 3.00 2.00

V4 4.00 4.00 4.00 6.00 3.00 4.00 3.00 4.00 3.00 6.00 3.00 4.00 4.00 5.00 2.00 3.00 3.00 4.00 3.00 6.00 3.00 4.00 5.00 6.00 2.00 6.00 2.00 6.00 7.00 4.00

V5 2.00 5.00 1.00 2.00 6.00 2.00 4.00 1.00 6.00 7.00 2.00 5.00 1.00 3.00 6.00 3.00 3.00 1.00 6.00 4.00 5.00 2.00 4.00 4.00 1.00 4.00 2.00 4.00 2.00 7.00

V6 4.00 4.00 3.00 5.00 2.00 4.00 3.00 4.00 3.00 6.00 3.00 4.00 3.00 6.00 4.00 4.00 4.00 4.00 3.00 6.00 3.00 4.00 4.00 7.00 4.00 7.00 5.00 3.00 7.00 2.00

19-13

Co nduc ti ng Fa ctor An alysi s Fig. 19.2

Problem formulation Construction of the Correlation Matrix Method of Factor Analysis Determination of Number of Factors Rotation of Factors Interpretation of Factors Selection of Surrogate Variables

Calculation of Factor Scores © 2007 Prentice Hall

Determination of Model Fit

19-14

Conduc ting Fac tor Analys is For mulat e the Pro blem 

The objectives of factor analysis should be identified.



The variables to be included in the factor analysis should be specified based on past research, theory, and judgment of the researcher. It is important that the variables be appropriately measured on an interval or ratio scale.



An appropriate sample size should be used. As a rough guideline, there should be at least four or five times as many observations (sample size) as there are variables.

© 2007 Prentice Hall

19-15

Co rre latio n Matr ix Table 19.2

© 2007 Prentice Hall

19-16

Conduc ti ng Fa ctor Ana lys is Cons truc t t he Co rrela tio n Ma trix 

The analytical process is based on a matrix of correlations between the variables.



Bartlett's test of sphericity can be used to test the null hypothesis that the variables are uncorrelated in the population: in other words, the population correlation matrix is an identity matrix. If this hypothesis cannot be rejected, then the appropriateness of factor analysis should be questioned.

Another useful statistic is the Kaiser-Meyer-Olkin (KMO) measure of sampling adequacy. Small values of the KMO statistic indicate that the correlations between pairs of variables cannot be explained by other variables and that factor analysis may not be appropriate. © 2007 Prentice Hall 

19-17

Conduc ting Fac tor Analys is Determ ine the Metho d of Fact or Analys is 

In princ ipal c om pone nts analysi s, the total variance in the data is considered. The diagonal of the correlation matrix consists of unities, and full variance is brought into the factor matrix. Principal components analysis is recommended when the primary concern is to determine the minimum number of factors that will account for maximum variance in the data for use in subsequent multivariate analysis. The factors are called principal components.



In commo n fac tor anal ysis , the factors are estimated based only on the common variance. Communalities are inserted in the diagonal of the correlation matrix. This method is appropriate when the primary concern is to identify the underlying dimensions and the common variance is of interest. This method is also known as principal axis factoring.

© 2007 Prentice Hall

19-18

Resul ts of P ri ncipal Com po nents Analysi s Table 19.3 Com m una lities Variabl es V1 V2 V3 V4 V5 V6

I niti al 1.00 0 1.00 0 1.00 0 1.00 0 1.00 0 1.00 0

Ex tracti on 0.926 0.723 0.894 0.739 0.878 0.790

I nitial Eig en values Factor 1 2 3 4 5 6 © 2007 Prentice Hall

Eigen va lue 2.731 2.218 0.442 0.341 0.183 0.085

% of varianc e 45.52 0 36.96 9 7.360 5.688 3.044 1.420

Cum ulat. % 4 5.520 8 2.488 8 9.848 9 5.536 9 8.580 10 0.0 00 19-19

Resul ts of P ri ncipal Com po nents Anal ysi s Table 19.3, cont.

© 2007 Prentice Hall

19-20

Resul ts of P ri ncipal Co mponen ts Analys is Table 19.3, cont.

© 2007 Prentice Hall

19-21

Resul ts of P ri ncipal Com po nents Analysi s Table 19.3, cont.

The lower-left triangle contains the reproduced correlation matrix; the diagonal, the communalities; the upper-right triangle, the residuals between the observed correlations and the reproduced correlations.

© 2007 Prentice Hall

19-22

Co nduc ti ng Fa ctor An alysi s De ter mine t he Number o f Fac to rs 

A Pr iori De terminat ion . Sometimes, because of prior knowledge, the researcher knows how many factors to expect and thus can specify the number of factors to be extracted beforehand.

De termination B as ed o n Eig env al ues. In this approach, only factors with Eigenvalues greater than 1.0 are retained. An Eigenvalue represents the amount of variance associated with the factor. Hence, only factors with a variance greater than 1.0 are included. Factors with variance less than 1.0 are no better than a single variable, since, due to standardization, each variable has a variance of 1.0. If the number of variables is less than 20, this approach will result in a conservative number of factors. 19-23 © 2007 Prentice Hall 

Co nduc ti ng Fa ctor An alysi s De termi ne th e Number o f Fac to rs 

Det er mi na tio n Based on Scr ee Pl ot. A scree plot is a plot of the Eigenvalues against the number of factors in order of extraction. Experimental evidence indicates that the point at which the scree begins denotes the true number of factors. Generally, the number of factors determined by a scree plot will be one or a few more than that determined by the Eigenvalue criterion.

Det er mi na tio n Based on Pe rce nt age of Va ri an ce. In this approach the number of factors extracted is determined so that the cumulative percentage of variance extracted by the factors reaches a satisfactory level. It is recommended that the factors extracted should account for at least 60% of the variance. © 2007 Prentice Hall 

19-24

Sc re e Plo t Fig. 19.3

3.0

Eigenvalue

2.5 2.0 1.5 1.0 0.5 0.0 1 © 2007 Prentice Hall

2

3 4 5 Component Number

6 19-25

Co nduc ti ng Fa ctor An alysi s De termi ne th e Number o f Fac to rs 

Det er mi na tio n Based on Split -Ha lf Rel ia bi lit y. The sample is split in half and factor analysis is performed on each half. Only factors with high correspondence of factor loadings across the two subsamples are retained.



Det er mi na tio n Based on Signi fican ce Test s. It is possible to determine the statistical significance of the separate Eigenvalues and retain only those factors that are statistically significant. A drawback is that with large samples (size greater than 200), many factors are likely to be statistically significant, although from a practical viewpoint many of these account for only a small proportion of the total variance.

© 2007 Prentice Hall

19-26

Co nduc ti ng F act or An alysis Ro ta te Fa ctors 

Although the initial or unrotated factor matrix indicates the relationship between the factors and individual variables, it seldom results in factors that can be interpreted, because the factors are correlated with many variables. Therefore, through rotation the factor matrix is transformed into a simpler one that is easier to interpret.



In rotating the factors, we would like each factor to have nonzero, or significant, loadings or coefficients for only some of the variables. Likewise, we would like each variable to have nonzero or significant loadings with only a few factors, if possible with only one.



The rotation is called orthogonal r ota tio n if the axes are maintained at right angles.

© 2007 Prentice Hall

19-27

Co nduc ti ng F act or An alysis Ro ta te Fa ctors 

The most commonly used method for rotation is the va rima x pr oce du re. This is an orthogonal method of rotation that minimizes the number of variables with high loadings on a factor, thereby enhancing the interpretability of the factors. Orthogonal rotation results in factors that are uncorrelated.



The rotation is called obliq ue rotat ion when the axes are not maintained at right angles, and the factors are correlated. Sometimes, allowing for correlations among factors can simplify the factor pattern matrix. Oblique rotation should be used when factors in the population are likely to be strongly correlated.

© 2007 Prentice Hall

19-28

Fac tor Mat ri x Befo re and Aft er Ro tat ion Fig. 19.4

Fac to rs Variables 1 2 3 4 5 6

1 X X X X X

(a) High Loadings Before Rotation © 2007 Prentice Hall

2 X X X X

Fac tor s Variables 1 2 3 4 5 6

1 X

2 X

X X X X

(b) High Loadings After Rotation 19-29

Co nduc ti ng F actor An alysis Inte rpre t F ac to rs 

A factor can then be interpreted in terms of the variables that load high on it.



Another useful aid in interpretation is to plot the variables, using the factor loadings as coordinates. Variables at the end of an axis are those that have high loadings on only that factor, and hence describe the factor.

© 2007 Prentice Hall

19-30

Fac to r Lo adi ng Pl ot Rota ted Componen t Matri x Compon ent 2

Fig. 19.5 Comp onen t Plot in Comp onen t 1ce Rot ated Spa

Vari abl e 2

∗∗ ∗ V6

V4

1.0

V2

V1 -2.6 6E-02

0.5 0.0

Com pon ent 1

∗ V3 ∗ V1

∗ V5

V2 V3 -0.1 46

0.9 62 -5.7 2E-02 0.9 34

-0.5

V4

-9.8 3E-02

-1.0

V5 -8.4 0E-02

-0.9 33

1.0

0.5

© 2007 Prentice Hall

0.0

-0.5

-1.0 V6

0.8 48

8.3 37 E-02

0.8 54

0.8 85

19-31

Co nduc ti ng F act or An alysi s Ca lcu late F acto r S cores The facto r sc ores for the ith factor may be estimated as follows:

Fi = Wi1 X1 + Wi2 X2 + Wi3 X3 + . . . + Wik Xk

© 2007 Prentice Hall

19-32

Co nduc ti ng F act or An alysi s Sel ect S ur rogate Va ri abl es 

By examining the factor matrix, one could select for each factor the variable with the highest loading on that factor. That variable could then be used as a surrogate variable for the associated factor.



However, the choice is not as easy if two or more variables have similarly high loadings. In such a case, the choice between these variables should be based on theoretical and measurement considerations.

© 2007 Prentice Hall

19-33

Co nduc ti ng F actor An al ysi s De ter mine the Mo del Fit 

The correlations between the variables can be deduced or reproduced from the estimated correlations between the variables and the factors.



The differences between the observed correlations (as given in the input correlation matrix) and the reproduced correlations (as estimated from the factor matrix) can be examined to determine model fit. These differences are called residuals.

© 2007 Prentice Hall

19-34

Resu lts of Com mo n Fac to r An al ysi s Table 19.4 Ba rl ett tes t of sph eric it y • Approx. Chi-Square = 111.314 • df = 15 • Significance = 0.00000 • Kaiser-Meyer-Olkin measure of sampling adequacy = 0.660

© 2007 Prentice Hall

19-35

Resu lts of Com mo n Fac to r An al ysi s Table 19.4, cont.

© 2007 Prentice Hall

19-36

Res ul ts o f Com mo n Fac to r An al ysi s Table 19.4, cont.

© 2007 Prentice Hall

19-37

Resu lts of Co mm on Fac to r Ana lys is Table 19.4, cont.

© 2007 Prentice Hall

19-38

SPSS Wi ndow s To select this procedure using SPSS for Windows click: An aly ze> Da ta Red ucti on>Fact or …

© 2007 Prentice Hall

19-39

SPSS Wi ndo ws: Princ ipal Com po ne nts Select ANALYZE from the SPSS menu bar. 2. Click DATA REDUCTION and then FACTOR. 3. Move “Prevents Cavities [v1],” “Shiny Teeth [v2],” “Strengthen Gums [v3],” “Freshens Breath [v4],” “Tooth Decay Unimportant [v5],” and “Attractive Teeth [v6].” in to the VARIABLES box.. 4. Click on DESCRIPTIVES. In the pop-up window, in the STATISTICS box check INITIAL SOLUTION. In the CORRELATION MATRIX box check KMO AND BARTLETT’S TEST OF SPHERICITY and also check REPRODUCED. Click CONTINUE. 5. Click on EXTRACTION. In the pop-up window, for METHOD select PRINCIPAL COMPONENTS (default). In the ANALYZE box, check CORRELATION MATRIX. In the EXTRACT box, check EIGEN VALUE OVER 1(default). In the DISPLAY box check UNROTATED FACTOR SOLUTION. Click CONTINUE. 6. Click on ROTATION. In the METHOD box check VARIMAX. In the DISPLAY box check ROTATED SOLUTION. Click CONTINUE. 7. Click on SCORES. In the pop-up window, check DISPLAY FACTOR SCORE COEFFICIENT MATRIX. Click CONTINUE. © 2007 Prentice Hall 8. Click OK. 1.

19-40