5_pca & Garett Rank.docx

PRINCIPAL COMPONENT ANALYSIS Its aim is to reduce a larger set of variables into a smaller set of 'artificial' variables (called principal components) that account for most of the variance in the original variables. Objective: To study the factors involved for usage of mobile Variables: All Ordinal variables Test: Factor Analysis Assumptions: Assumption #1: You have multiple variables that are measured at the continuous level (although ordinal data is very frequently used). Assumption #2: There should be a linear relationship between all variables. Assumption #3: There should be no outliers. Assumption #4: There should be large sample sizes for a principal components analysis to produce a reliable result. Many different rules-of-thumb have been proposed that differ mostly by either using absolute sample size numbers or a multiple of the number of variables in your sample. Generally speaking, a minimum of 150 cases or 5 to 10 cases per variable have been recommended as minimum sample sizes. Order of testing to run a principal components analysis There are six steps, other than the testing of assumptions, when running a principal components analysis. These are: 1. 2. 3. 4. 5. 6.

Initial extraction of the components. Determining the number of 'meaningful' components to retain. Rotation to a final solution. Interpreting the rotated solution. Computing component scores or component-based scores. Reporting the results.

A principal components analysis can be used to solve three major problems: (a) removing superfluous/unrelated variables; (b) reducing redundancy in a set of variables; and (c) removing multicollinearity. These three study designs are appropriate when you have met the assumptions of principal components analysis.

Study Design #1 Removing superfluous/unrelated variables Principal components analysis allows you to 'cluster' variables together that all load on the same component. If one component only loads on one variable, this could be an indication that this variable is not related to the other variables in your data set and might not be measuring anything of importance to your particular study (i.e., it is measuring some other construct or measure). Study Design #2 Reducing redundancy in a set of variables If you have measured many variables and you believe that some of the variables are measuring the same underlying construct, you might have variables that are highly correlated. Principal components analysis will allow the reduction of many correlated variables into a single artificial variable called a principal component. This principal component can then be used in later analyses. In addition, it can be used to create a scale of questions. Study Design #3 Removing multicollinearity If you have two or more variables that are highly correlated, principal components analysis might help. It can reduce the highlighted correlated variables into principal components that can be used to generate a component score which can be used in lieu of the original variables. Two of the assumptions that can be tested using SPSS Statistics include: (a) Linearity between all variables, which can be evaluated using a correlation matrix; and (b) Sampling adequacy, which can be detected using the Kaiser-Meyer-Olkin (KMO) measure of sampling adequacy for the overall data set, the KMO measure for each individual variable and Bartlett's test of sphericity. Procedure:

Interpretation of Outputs:

You want to examine the correlations in this table to see if there are any variables that are not strongly correlated with any other variable. The level of correlation considered worthy of a variable's inclusion is usually r ≥ 0.3. Thus, scan the correlation matrix for any variable that does not have at least one correlation with another variable where r ≥ 0.3. In this data set, all variables have at least one correlation with another variable greater than the 0.3 cut-off. Thus the assumption of Linearity is met.

Sampling adequacy

There are a few methods to detect sampling adequacy: (1) the Kaiser-Meyer-Olkin (KMO) measure of sampling adequacy for the overall data set; (2) the KMO measure for each individual variable; and (3) Bartlett's test of sphericity. (1) the Kaiser-Meyer-Olkin (KMO) measure of sampling adequacy for the overall data set The KMO measure is used as an index of whether there are linear relationships between the variables and thus whether it is appropriate to run a principal components analysis on your current data set. Its value can range from 0 to 1, with values above 0.6 suggested as a minimum requirement for sampling adequacy, but values above 0.8 considered good and indicative of principal components analysis being useful. A KMO measure can be calculated for all variables combined and for each variable individually. The overall KMO measure is provided in the KMO and Bartlett's Test table, as highlighted below:

In this example, the KMO measure is 0.730, which is good; or "Middling" on Kaiser's (1974) classification of measure values, as shown in the table below:

(2) the KMO measure for each individual variable KMO measures (Measures of Sampling Adequacy) for individual variables are found on the diagonals of the anti-image correlation matrix, as found in the Anti-image Matrices table under the "Anti-image Correlation" (2nd) section, as highlighted below (not to be confused with the "Anti-image Covariance" (1st) section):

You are again looking for the KMO measures to be as close to 1 as possible, with values above 0.5 an absolute minimum and greater than 0.8 considered good. If any particular variable has a low KMO measure (KMO < .5), you should consider removing it from the analysis. 3) Bartlett's test of sphericity: Bartlett's test of sphericity tests the null hypothesis that the correlation matrix (see screenshot above) is an identity matrix. An identity matrix is one that has 1's on the diagonal and 0's on all the off-diagonal elements. Effectively, it is saying that there are no correlations between any of the variables. This is important because if there are no correlations between variables, you will not be able to reduce the variables to a smaller number of components (i.e., there is no point in running a principal components analysis). As such, you want to reject the null hypothesis. The result of this test is contained within the second part of the KMO and Bartlett's Test table, as highlighted below:

In this example, you can see that Bartlett's test of sphericity is statistically significant (i.e. p < .05). This significance value (i.e., p-value) is located in the "Sig." row. In this example, it is stated as ".000", which actually means p < .0005. This test is particularly sensitive to deviations from multivariate normality and, as such, you might consider not using this test. The output generated by SPSS Statistics is quite extensive and can provide a lot of information about your analysis. However, you will often find that the analysis is not yet complete and more re-runs of the analysis will have to take place before you get

to your final solution. This will be discussed over the next three pages, which will focus on: (a) communalities; (b) extracting and retaining components; and (c) forced factor extraction. Communalities: The communality is the proportion of each variable's variance that is accounted for by the principal components analysis and can also be expressed as a percentage Extracting and retaining components: A principal components analysis will produce as many components as there are variables. However, the purpose of principal components analysis is to explain as much of the variance in your variables as possible using as few components as possible. After you have extracted your components, there are four major criteria that can help you decide on the number of components to retain: (a) the eigenvalue-one criterion, (b) the proportion of total variance accounted for, (c) the scree plot test, and (d) the interpretability criterion. All except for the first criterion will require some degree of subjective analysis. Forced factor extraction: When extracting components as part of your principal components analysis, SPSS Statistics does this based on the eigenvalue-one criterion. However, it is possible to instruct SPSS Statistics how many components you want to retain. Communalities The communality is the proportion of each variable's variance that is accounted for by the principal components analysis and can also be expressed as a percentage. Communalities are reported in the Communalities table, as shown below:

Extracting components: A principal components analysis will produce as many components as there are variables. There are 25 variables in this example analysis, so there are 25 components.

If you were to retain all components in your analysis you will be able to account for all the variance in your variables. However, this is not the purpose of principal components analysis; it is to explain as much of the variance as possible using as few components as possible. The first component will explain the greatest amount of total variance, with each subsequent component accounting for relatively less of the total variance. Generally speaking, only the first few components will need to be retained for interpretation and these components will account for the majority of the total variance. The amount of variance each component accounts for plus its contribution towards total variance is presented in the Total Variance Explained table under the "Initial Eigenvalues" columns, as highlighted below:

An eigenvalue is a measure of the variance that is accounted for by a component. An eigenvalue of one represents the variance of one variable, so with 10 variables there is a total of 10 eigenvalues of variance. Therefore, if you examine the first component, you will find that it explains 3.938 eigenvalues of variance (the "Total" column), which is 3.938/10 x 100 = 39.385% of the total variance, as reported in the "% of Variance" column. Each successive percentage of variance explained is calculated in the same way with the cumulative percentage recorded in the "Cumulative %" column. You can see from the above table that all 10 components explain all of the total variance. Unless you remove cases (e.g., participants) or variables from the principal components analysis, the values contained within these three columns will never change regardless of how many components you decide to retain or how you decide to rotate the solution. The next sections show you how to decide how many components to retain in your analysis. Criteria for choosing components to retain Now that you have extracted all the principal components, you need to determine how many components to retain for rotation and interpretation. Although there are some statistical considerations in deciding this number you will also have to make subjective judgements. You must remember that with principal components analysis there is no one objectively correct answer; the correct number of components to retain is something that you have to decide. There are four major criteria that can help you decide on the number of components to retain. These are the eigenvalue-one criterion, the proportion of total variance accounted for, the scree plot test, and the

interpretability criterion. All except for the first criterion will require some degree of subjective analysis. These four criteria are presented in the next four sections: The eigenvalue-one criterion The eigenvalue-one criterion (also referred to as the Kaiser criterion (Kaiser, 1960)) is one of the most popular methods for establishing how many components to retain in a principal components analysis and it is the default option in SPSS Statistics. Indeed, this default was kept for this first iteration of principal components analysis in this guide. An eigenvalue less than one indicates that the component explains less variance that a variable would and hence shouldn't be retained. The major advantage of this criterion is that it is very simple. Unfortunately, there are also some major pitfalls. One pitfall is when the eigenvalue of a component(s) hovers close to one. In this example, the fifth component has an eigenvalue of 1.049 and the sixth component an eigenvalue of 0.951. Therefore, the interpretation is fairly clear: components one to five are retained and the sixth component onwards are not. This is highlighted below:

However, if, for example, the fifth component had an eigenvalue of 1.002 and the sixth component an eigenvalue of 0.998, you would be declaring the fifth component to be meaningful (retained), but the sixth component as trivial (not retained), even though there is virtually no difference in the variance accounted for by the two components. Using the criterion in this example results in five components being retained. Percentage of variance explained There are two criteria that centre around examining the proportion/percentage of total variance explained by each component: (1) the proportion of variance explained by each component individually; and (2) the cumulative percentage of variance explained by a set number of components. You will find this information in the "% of Variance" and "Cumulative %" columns under the "Initial Eigenvalues" column in the

Total Variance Explained table, as highlighted below:

As the component number increases, each subsequent component explains less of the total variance. Thus, the first component explains 26.9% of the total variance, whereas the second component explains only 13.4%. And this trend continues with increasing component number. It has been suggested that a component should only be retained if it explains at least 5% to 10% of the total variance. In this example, this would lead to the retention of the first four components. Another criterion is to retain all components that can explain at least 60% or 70% of the total variance. Using the lower criterion of 60%, would lead to the retention of the first five components (cumulative percentage of 64.1% versus 59.9% for first four components) in this example. As you can see, these suggested criteria result in a different number of suggested components to retain. The percentage figures to determine retention of components are arbitrary and thus the weakness of this criterion is that it somewhat subjective. Scree plot The scree plot (Cattell, 1966) is presented in the output as a line graph entitled Scree Plot, as shown below:

INFLECTION POINT

A scree plot is a plot of the total variance explained by each component (its "eigenvalue") against its respective component. As there are as many components as there are variables, there are 10 components in the scree plot. The components to retain are those before the (last) inflection point of the graph (see above). The inflection point is meant to represent the point where the graph begins to level out and subsequent components add little to the total variance. In this example, visual inspection of the scree plot would lead to the retention of four components. You should already be noticing that the number of components suggested to retain using the different criteria are not always the same. This is one of the reasons why you have to make a subjective decision on how many components to retain. Interpretability criterion The interpretability criterion is arguably the most important criterion and it largely revolves around the concept of "simple structure" and whether the final solution makes sense. For the number of components to influence the interpretability of the final solution, you first need to inspect the rotated component matrix, which is presented in the Rotated Component Matrix table, as shown below:

The Rotated Component Matrix table shows how the retained, rotated components load on each variable. Be aware that each retained component will load on each variable and that the only reason there are what appears to be missing values is due to the settings you made earlier when you instructed SPSS Statistics to suppress all coefficients less than .03 (see the Factor Analysis: Options dialogue box instructions in the Procedure section, if needed). You can see from this table that there is still some "complex structure"; namely, many components loading on the same individual variables. For example, components 1 and 5 both load on variable Qu18. Ultimately, it is still difficult to interpret this rotated solution when you have "complex structure" in evidence. What you are trying to achieve is called "simple structure". Simple structure is when each variable has only one component that loads strongly on it and each component loads strongly on at least three variables. The overriding concept of the interpretability criterion is whether the final rotated solution makes sense (i.e., can you explain it). In this example, so far, the answer is No.

Findings: A principal components analysis (PCA) was run on a 10-question questionnaire that measured desired mobile characteristics on 500 candidates. The suitability of PCA was assessed prior to analysis. Inspection of the correlation matrix showed that all variables had at least one correlation coefficient greater than 0.3. The overall Kaiser-Meyer-Olkin (KMO) measure was 0.73 with individual KMO measures all greater than 0.7, classifications of 'middling' to 'meritorious' according to Kaiser (1974). Bartlett's test of sphericity was statistically significant (p < .0005), indicating that the data was likely factorizable. PCA revealed five components that had eigenvalues greater than one and which explained 39.385%,19.740% and 11.310% of the total variance, respectively. Visual inspection of the scree plot indicated that three components should be retained (Cattell, 1966). In addition, a three-component solution met the interpretability criterion. As such, three components were retained. The three-component solution explained 70.435% of the total variance. A Varimax orthogonal rotation was employed to aid interpretability. The rotated solution exhibited 'simple structure' (Thurstone, 1947). The interpretation of the data was consistent with the personality attributes the questionnaire was designed to measure with strong loadings of PERSONALITY AND SELF efficacy items on Component 1, ASSET VALUE items on Component 2 and ENTERTAINMENT items on Component 3. Component loadings and communalities of the rotated solution are presented.

Henry Garrett Ranking Method - Factor Analysis data