Testing Recursive Path Models With Correlated

Testing recursive path models with correlated errors using d-separation

Bill Shipley Département de biologie Université de Sherbrooke Sherbrooke (Qc) J1K 2R1 CANADA [email protected]

(819) 821-8000 ext. 2079 (819) 821-8049 (FAX) 6 June 2002

Structural Equation Modeling (2003) 10(2):214-221.

Running Head: d-sep test for path models with correlated errors

1

ABSTRACT This paper shows how to extend the inferential test of Shipley (2000b), that is applicable to recursive path models without correlated errors (a DAG model), to a class of recursive path models that include correlated errors (a semiMarkov model). The path model is first converted to a partial ancestral graph (PAG) and then, for PAGs that do not require latent variables, an inducing path DAG is obtained that is equivalent in its conditional independence relationships to the original path model. The null probabilities of the k tests of independence that are implied by this DAG are combined using Fisher’s test statistic C=2∑Ln(pi) which is distributed as a chi-square variate with 2k degrees of freedom.

2

INTRODUCTION

Shipley (2000a,b) introduced a new inferential test for recursive path models without correlated errors. This test has a number of advantages over the classical tests based on maximum likelihood estimation of a model covariance matrix. First, the new test is exact rather than asymptotic and therefore permits tests using small data sets. Second, being based on tests of (conditional) independence, many different types of variables, functional relationships and probability density functions can be accommodated. Third, the model test can be decomposed into individual tests of elementary predictions of the underlying causal hypothesis; this “local” property allows the researcher to identify those parts of a poorly-fitting model that are contributing to the lack of fit. Here, I describe the conditions under which the test can be extended to path models with correlated errors and how this extension is performed. The test in Shipley (2000b) is based on two notions derived from the theory of directed acyclic graphs (DAGs): d-separation and basis sets of dseparation relationships. D-separation is a manipulation of a DAG that separates the directed paths between any two variables (vi, vj) in the DAG given a set Q of other variables. In this paper I will use the notation (vi| Q| vj) to mean “variable vi is d-separated from variable vj given the set of variables Q” and say that the set Q “blocks” the directed paths linking vi and vj in the DAG. It can be proven that if any two variables in a DAG are d-separated then, in any data generated by the causal process represented by the DAG, the variables will be probabilistically

3

independent upon conditioning on the variables that are blocked (Pearl 1988; Geiger, Verma and Pearl 1990; Geiger, Paz and Pearl 1991; Geiger and Pearl 1993; Pearl 2000). This does not require assumptions of linearity of the functional relationships nor of any particular multivariate distribution of the variables. Two variables (vi, vj) in a DAG (a Markov model) or in a semi-Markov graph, for instance a path diagram with correlated errors, are d-separated given a set Q of other variables in the graph if and only if there is no undirected path U between vi and vj such that (i) every non-colliding variable along U is not a member of Q and (ii) every colliding variable along U is either a member of Q or else has a causal descendent that is a member of Q. A variable X is said to “collide” along an undirected path U if it has arrowheads pointing into it from both directions, thus: WÆXÅY. An extended discussion of d-separation is found in Shipley (2000a). The second notion is a “basis set” of d-separation statements. Let S be the set of d-separation relationships (and therefore independence claims) that are implied by a DAG. A basis B for S is a set of d-separation claims that implies, using the laws of probability and the axioms of d-separation, all other elements in S and no proper subset of B sustains such implications (Pearl 1988; Verma and Pearl 1988; Pearl 2000). This means that a test of the independencies in the basis set is necessary and sufficient to test all independencies implied in the DAG. The basis set used in Shipley (2000a,b) is the following (Pi is the set of causal parents of variable vi in the DAG): BU={vi|Pi∪Pj|vj}. This basis set has the

4

additional property that its elements imply mutually independent residuals of vi and of vj, conditional on Pi∪Pj, and therefore mutually independent tests of independence (Shipley 2000b). Other basis sets are also possible (Pearl 2000) but are not appropriate for the inferential test discussed here. An exact inferential test for such a DAG (i.e. a path model without correlated errors) is obtained by first calculating the probability pj associated with each of the k independence claims in the basis set BU and then calculating the statistic k

C = −2∑ Ln( p j ) . This statistic is distributed as a chi-squared variate with 2k j =1

degrees of freedom if all of the k independence claims are true in the statistical population. Shipley (2000a) provides more details concerning this test.

Path models with correlated errors

In graphical terms a recursive path model with correlated errors is not a DAG and therefore the test proposed by Shipley (2000b) cannot be used. This is because the basis sets for DAGs are not, in general, basis sets for path models with correlated errors. I had suggested in Shipley (2000b) a way of extending the test. This extension consisted of constructing an augmented DAG by replacing each correlated error – represented by a bi-directional arc – by a new latent variable which is the causal parent of the two variables possessing the correlated errors, constructing the basis set from this augmented DAG, and then testing only those d-separation claims in the basis set that do not involve latent variables. Unfortunately this way of testing a path model with correlated errors is 5

of limited value because there can be d-separation claims involving only observed variables that are implied by the model but which are neither in BU nor which are implied by some combination of those d-separation claims in BU that do not involve latent variables. Therefore the extension suggested in Shipley (2000b) is a necessary, but not sufficient, test of the conditional independence claims of a semi-Markov model. The extension proposed in this paper requires a few more definitions. Unshielded colliders Given a triplet of variables (X,Y,Z) in a graph, if there is an edge between X and Y, an edge between Y and Z, no edge between X and Z, then call this an “unshielded pattern”. If, in this unshielded pattern, there are arrowheads pointing into Y from both X and Z then call Y an “unshielded collider”. This is shown graphically as X•ÆYÅ•Z where the circle (•) means that there can be an arrowhead or not in that position. Note that Spirtes et al. (1993; 2000) use an open circle (o) to represent the same information. If, in this unshielded pattern, there are not arrowheads pointing into Y from both X and Z then call Y a “definite non-collider”. This is shown graphically as X•—•Y•—•Z. An augmented DAG An augmented DAG (D’) of a path model with correlated errors is obtained by replacing each correlated error – represented by a bi-directional arc – by a new latent variable (lij) which is the causal parent of the two variables (vi, vj) possessing the correlated errors. This augmented DAG will have two types of variables: observed variables denoted by the set V and latent variables denoted

6

by the set L. This augmented DAG will have the same d-separation relationships involving the original (observed) variables as the path model with correlated errors (Spirtes et al. 1995). See Figure 1a. Inducing paths An inducing path between two observed variables (vi, vj) in the augmented DAG D’ relative to the set of observed variables (V) exists if there is no subset Q⊆ V\{vi,vj}, including the null subset, such that vi and vj are d-separated given Q. In a DAG, two variables that are non-adjacent are necessarily d-separated given some subset of the remaining variables. This is not true in general for semiMarkov models. An example is seen in Figure 1b. Variables X1 and X4 in Figure 1b are not adjacent. Although they are d-separated given {X3, l24}, they are not d-separated given any subset of the remaining observed variables {X2, X3}. Using the null subset there is an open path X1ÆX2ÆX3ÆX4. Using either {X2}, {X3} or {X2, X3} the pair (X1, X4) are not d-separated because there exists the undirected path X1ÆX2Ål24ÆX4 which is activated when conditioned on either X2 and/or its descendent X3. There is therefore an inducing path between X1 and X4 in the extended DAG D’ over the observed variables. In general, two nonadjacent observed variables (vi, vj) will have an inducing path between them relative to V\vi,vj if there exists an undirected path U between them such that all observed variables on U are colliders and are ancestors of either vi or vj. Partial ancestral graphs Spirtes et al. (1993; 2000) introduced the notion of a PAG, or partial ancestral graph. Spirtes et al. (1993) previously called the same thing is a

7

“partially oriented inducing path graph” and Desjardins (1999) has called it a marginal dependency graph. Consider all those DAGs (M) containing the same set V of observed variables, different sets of latent variables, but that imply the same d-separation relationships between the set V of observed variables. A PAG is a graphical construct involving only V such that (i) two variables (vi, vj) have an edge between them if every DAG in M has an inducing path between vi and vj relative to V\{vi, vj} and (ii) each unshielded pattern that collides at Y in every DAG in M also collides in the PAG and (iii) every unshielded pattern that is a definite non-collider at Y in every DAG in M is also a definite non-collider in the PAG. There are some other orientation rules that can be applied to the PAG but these aren’t necessary for the purposes of this paper. In other words, every faithful acyclic model that has the same conditional independence relationships has the same PAG. The construction of a PAG is given in the Causal Inference Algorithm. Constructing the PAG The causal inference (CI) algorithm is given on page 183 of Spirtes et al. (1993). Theorem 6.3 of that reference states that if the input to the CI algorithm, involving the observed variables, is faithful to the generating graph, then the output of the CI algorithm is a PAG. Since we assume, under the null hypothesis that the path model with correlated errors is correct, this is always true by assumption. The following steps will produce a PAG that is sufficient for our purposes.

8

1. Given the path model with correlated edges (G), construct the extended DAG (G’) by removing each double-headed arrow (vi↔vj) and replacing it with a latent (lij) that is an exogenous common cause of only vi and vj (viÅlijÆvj). 2. Construct an undirected graph (P) from G by removing all arrowheads but retaining each edge and adding circles at the ends of each edge (vi•—•vj). 3. For each pair of non-adjacent variables in G’ (vi, vj) having an inducing path between them relative to the observed variables in G (i.e. V\{vi,vj}), add the edge: (vi•—•vj) to P. Call the resulting graph P, at this step, an undirected dependency graph. 4. For each unshielded pattern in P involving a triplet of variables (X, Y, Z) orient Y as either a definite non-collider (X•—•Y•—•Z) or an unshielded collider (X•ÆYÅ•Z) based on the d-separation relationships in G’. 5. Orient the remaining edges in P such a way that all definite noncolliders and unshielded colliders are respected, no new unshielded colliders are formed, and no cycles are formed. Verma and Pearl (1992) gave four rules for obtaining a maximally orienting P and theorems 37 and 38 of Meek (1998) prove that they are sound (i.e. any orientation other that that specified by these rules would lead to either a new unshielded collider or to a directed cycle.

9

The result of step 4 is a PAG. Step 5 simply generates one of the equivalent acyclic graphs represented by the PAG; Desjardins (1999) calls this an “inducing path graph”. Because the PAG, and therefore the resulting inducing path graph (P), are equivalent in their d-separation relationships to the original path model with correlated errors (G), every dseparation relationship in the inducing path graph exists in G and there is no d-separation relationship in G that is not also implied by the inducing path graph. If P can be oriented in such a way that it is a DAG then one can test it using the inferential test of Shipley (2000b). I will call inducing path graphs that are also DAGs “inducing path DAGs”. Figure 2 shows the steps involved. Note that since the path models in Figure 2a,b are dseparation equivalent to an inducing path DAG, one can obtain a basis set and use the inferential test of Shipley (2000b). On the other hand there is no inducing path DAG that is d-separation equivalent to the path model in Figure 2c. Although one can still obtain dseparation relationships implied by this non-DAG inducing path, and therefore of the original path model, these d-separation statements are not necessarily a basis; the only possibility is to conduct an approximate test by using a Bonferonni correction to the significance level used in each of the tests of independence.

Conclusions

10

When testing path models without correlated errors (i.e. DAG models), the inferential test of Shipley (2000b) is superior to classical SEM based on maximum likelihood for the reasons given in the Introduction. In such models all constraints on the covariance matrix are independence constraints and are therefore predicted by d-separation (Pearl 2000). This extension to the inferential test of Shipley (2000b) that is proposed here is not always superior to classical SEM when applied to path models with correlated errors. First, some such path models are not d-separation equivalent to any inducing path DAG, thus precluding the exact test. Second, path models with correlated errors can imply constraints on the covariance matrix that are equalities between functions of covariances rather than conditional independence constraints. Therefore, when the assumptions of classical SEM are met, such a test is therefore more powerful. Desjardins (1999) contains results that may point to a way of testing such constraints without resorting to maximum likelihood estimation. There are conditions in which Shipley’s (2000b) test would still be preferable to classical SEM even in the case of path models with correlated errors, assuming that an inducing path DAG exists. First, nonnormal data and non-linear relationships can be accommodated (Shipley 2000a). Second, the test is exact and can therefore be used with small samples. Third, there are models like the one in Figure 2b that are unidentified and can therefore not be tested using classical SEM but that can still be tested using the method presented here. Finally, if the model

11

is judged not to fit the data, the local property of the test can allow one to determine which parts of the model are contributing to lack of fit. This cannot be done using ML estimation since errors in one part of the model are propagated throughout the rest of the model due to the global nature of the method.

Acknowledgements This research was financially supported by the Natural Sciences and Engineering Research Council of Canada.

12

References Desjardins, B. (1999). On the theoretical limits to reliable causal inference. Faculty of Arts and Sciences. Pittsburg, University of Pittsburg, 161. Geiger, D., Paz, A & Pearl, J. (1991). Axioms and algorithms for inferences involving probabilistic independence. Information and Computation 91, 128-141. Geiger, D. and Pearl, J. (1993). Logical and algorithmic properties of conditional independence and graphical models. The annals of statistics 21, 20012021. Geiger, D., Verma, T. & Pearl, J. (1990). Identifying independence in Bayesian Networks. Networks 20, 507-534. Meek, C. (1998). Graphical models: Selecting causal and statistical models. Department of Philosophy. Pittsburg, Carnegie Mellon University. Pearl, J. (1988). Probabilistic reasoning in intelligent systems: Networks of plausible inference. Morgan Kaufmann, San Mateo, CA. Pearl, J. (2000). Causality. Cambridge University Press, Cambridge. Scheines, R., Spirtes, P., Glymour, C, & Richardson, T. (1998). The TETRAD project: Constraint based aids to causal model specification. Multivariate Behavioral Research, 33, 65-117. Shipley, B. (2000a). Cause and correlation in biology: A user's guide to path analysis, structural equations, and causal inference. Oxford University Press, Oxford.

13

Shipley, B. (2000b). A new inferential test for path models based on directed acyclic graphs. Structural Equation Modeling 7, 206-218. Spirtes, P., Glymour, C. & Scheines, R. (1993). Causation, Prediction, and Search. Springer-Verlag, New York. Spirtes, P., Glymour, C. & Scheines, R. (2000). Causation, prediction, and search. MIT Press, Cambridge, Mass. Spirtes, P., Richardson, T., Meek, A., Scheines, R. & Glymour, C. (1995). Using D-separation to calculate zero partial correlations in linear models with correlated errors. Technical Report CMU-Phil-72. Verma, T. and Pearl, J. (1988). Causal networks: Semantics and expressiveness. Pp. 69-76 in Schachter, T., Levitt, S. & Kanal, L.N. (editors). Uncertainty in Artifical Intelligence Volume 4. Amsterdam: Elsevier. Verma, T. and Pearl, J. (1992). An algorithm for deciding if a set of observed independencies has a causal explanation. Pp. 323-330 in Dubois, D., Wellman, M.P., D’Ambrosio, B. & Smets, P. (editors). Proceedings of the 8th Conference on Uncertainty in Artificial Intelligence. CA, Morgan Kaufmann.

14

Figure 1. (a) A path model with correlated errors (i.e. a semi-Markov model). (b) The augmented DAG for the path model. (c) A partial ancestral graph for both the augmented DAG and the path model. Figure 2. Three different path models with correlated errors (a, b and c) are shown along with the undirected dependency graph, the partial ancestral graph (PAG) and the inducing path acyclic graph. The d-separation relationships shown at the bottom form a basis set for models a and b, but not for c.

15

Figure 1.

(a)

X1

X2

X3

X4

(b)

X1

X2

X3

X4

L24

(c)

X1

X2

X3

X4

16

Figure 2. (a) Path model

X1

X2

(b) X3

X1

l34

l12 Extended DAG

X4

X2

(c) X3

l12

X4

X1

l34

X2 l12

X3 l23

X4 l34

X1

X2

X3

X4

X1

X2

X3

X4

X1

X2

X3

X4

Undirected X1 dependency graph

X2

X3

X4

X1

X2

X3

X4

X1

X2

X3

X4

PAG

X1

X2

X3

X4

X1

X2

X3

X4

X1

X2

X3

X4

IPG

X1

X2

X3

X4

X1

X2

X3

X4

X1

X2

X3

X4

D-separation relationships

X1_||_X3|X2 X1_||_X4|0 X2_||_X4|0

X1_||_X3|X2 X1_||_X4|{X2,X3}

X1_||_X3|0 X1_||_X4|0 X2_||_X4|0

17