Teams

Journal of Applied Psychology 2006, Vol. 91, No. 2, 467– 474

Copyright 2006 by the American Psychological Association 0021-9010/06/$12.00 DOI: 10.1037/0021-9010.91.2.467

Becoming Team Players: Team Members’ Mastery of Teamwork Knowledge as a Predictor of Team Task Proficiency and Observed Teamwork Effectiveness Robert R. Hirschfeld

Mark H. Jordan

University of Georgia

United States Air Force Academy

Hubert S. Feild, William F. Giles, and Achilles A. Armenakis Auburn University The authors explored the idea that teams consisting of members who, on average, demonstrate greater mastery of relevant teamwork knowledge will demonstrate greater task proficiency and observed teamwork effectiveness. In particular, the authors posited that team members’ mastery of designated teamwork knowledge predicts better team task proficiency and higher observer ratings of effective teamwork, even while controlling for team task proficiency. The authors investigated these hypotheses by developing a structural model and testing it with field data from 92 teams (1,158 team members) in a United States Air Force officer development program focusing on a transportable set of teamwork competencies. The authors obtained proficiency scores on 3 different types of team tasks as well as ratings of effective teamwork from observers. The empirical model supported the authors’ hypotheses. Keywords: team training, team performance, mental model, multilevel, aggregation

within larger organization systems and who require coordinated interactions to successfully accomplish relevant tasks (Cohen & Bailey, 1997; Forsyth, 1999). Because teams are often assembled to take on multifaceted and complex endeavors, team tasks present a range of knowledge-intensive challenges to team members (McIntyre & Salas, 1995; Mohrman, 2003). As such, scholars have theorized that team members’ ongoing intellectual development contributes to collective performance (e.g., DeNisi, Hitt, & Jackson, 2003; London & Mone, 1999). In other words, greater workplace learning by team members should translate into better team performance, as long as what is being learned is relevant (Austin, 2003; Kozlowski, Gully, Nason, & Smith, 1999). We investigated this general idea by specifically assessing team members’ mastery of teamwork knowledge presented in a personnel development program and testing relationships between those scores and different indicators of team performance. To test whether an individual-level intellectual resource (in whatever form) predicts team-level performance, researchers must designate the individual resource of relevance, measure it, and then aggregate scores of team members to the team level. After the aggregate variable is formed, researchers then test its relationship to variables representing team performance. It is important that this process be guided, from the outset, by multilevel theory and appropriate methodological considerations, insofar as there are different ways in which a team-level phenomenon may emerge from a corresponding individual-level phenomenon (Kozlowski & Klein, 2000). Chan (1998) provided a typology of five composition models that specify the functional relations among phenomena or factors that refer to the same content domain, yet that differ qualitatively across levels of analysis (e.g., individual vs. team). For each form of functional relationship (or lower-to-higher-level

Numerous statistics point to substantial investments by employers in employee training and development, thereby suggesting that managers widely regard employees’ workplace learning as a key factor for occupational competence (Goldstein & Ford, 2002). The knowledge that individuals attain in the workplace produces relevant intellectual resources that employees can rely on for making decisions and taking actions in their organizational roles (Noe, Colquitt, Simmering, & Alvarez, 2003). As part of a comprehensive study, Schmidt and Hunter (2004) underscored the acquisition of job knowledge (i.e., learning) as the key explanatory factor through which individuals’ general mental ability translates into individual success in the world of work. Simply put, individuals who learn better in their organizational roles most often perform better in them. Although it has been established that individuals’ mastery of job knowledge predicts their individual success in the workplace, many organizations now emphasize performance at the team level (Ilgen & Pulakos, 1999). Teams are commonly regarded as structured sets of people who pursue collective performance objectives

Robert R. Hirschfeld, Department of Management, University of Georgia; Mark H. Jordan, Department of Behavioral Sciences and Leadership, United States Air Force Academy; Hubert S. Feild, William F. Giles, and Achilles A. Armenakis, Department of Management, Auburn University. Robert R. Hirschfeld and Mark H. Jordan contributed equally to this research. The views expressed in this article are those of the authors and do not reflect the official policy or position of the United States Air Force, the Department of Defense, or the United States government. Correspondence concerning this article should be addressed to Robert R. Hirschfeld, Department of Management, Terry College of Business, University of Georgia, Athens, GA 30602-6256. E-mail: [email protected] 467

RESEARCH REPORTS

468

emergence), Chan provided an operational process by which a lower-level variable (e.g., an individual-level resource) may be combined to form a higher-level factor (e.g., a resource pool within a team). In an additive composition model, a team-level factor is a summation of individual-level units, whereas the variance among the individual-level units (within teams) is of no theoretical or operational concern (Chan, 1998). For example, a team’s resource pool may be described as high or low irrespective of the extent to which individual team members possess the same resource. Therefore, to create a construct representing a resource pool within a team, individuals’ resource levels (scores) are simply summed or averaged (without concern for within-team similarity) to represent the team-level factor (a team’s resource pool). Having used an additive composition model, a number of studies have shown that teams consisting of members with higher scores on mental ability tests learn and perform better as teams (e.g., Barrick, Stewart, Neubert, & Mount, 1998; Ellis et al., 2003; LePine, 2003; Terborg, Castore, & DeNinno, 1976; Tziner & Eden, 1985). Similarly, Austin (2003) examined the existing knowledge stock of crossfunctional business groups by additively aggregating individual team members’ scores on a situational judgment test. Austin’s results indicated that group knowledge stock was positively related to overall group performance. In the studies referenced above, the tests taken by individuals were those that measured broad intellectual resources acquired over many years and through various means (i.e., enduring mental ability). Those results, therefore, have implications for personnel selection in team settings. Although we used additive aggregation, we did so to examine team members’ mastery of teamwork concepts presented in a development program as a predictor of team performance. As such, the focus of our study has meaningful implications regarding the development of individuals’ intellectual resources specifically for optimizing team performance. This study represents a constructive replication (e.g., Eden, 2002; Lykken, 1968) of previous research that has explored the general notion that team members’ intellectual resources, in various forms, predict team performance.

Theory and Hypotheses Development For much of the 20th century, efforts to facilitate learning in the workplace focused on technical knowledge or skill so that each individual’s task proficiency would be optimized (Howard, 1995). Scholars have emphasized that to achieve overall competence in today’s team-oriented world of work, individuals must also develop a sophisticated understanding of how they are connected to others in the workplace and how they can build better working relationships (e.g., Campbell, 1999; Lankau & Scandura, 2002; Mohrman, 2003; Murphy & Jackson, 1999). This understanding is analogous to a mental model of teamwork. Smith-Jentsch, Campbell, Milanovich, and Reynolds (2001) explained that a mental model of teamwork is an individual’s understanding of the teamwork components that facilitate effective team performance, as well as the understanding of how these components are related in a scheme of teamwork. Mental models of teamwork are abstract and transcend team membership in that they are conceptual representations of effective team processes that may be learned by

individuals as preparation for performing with future teammates (Smith-Jentsch et al., 2001). Mohammed and Dumville (2001) have theorized that for teams to achieve optimal effectiveness, it is important for all team members to learn an appropriate model of teamwork. Toward that end, Smith-Jentsch et al.’s (2001) findings suggest that it may be useful to design training that uniformly guides team members toward understanding and adopting an expert model of teamwork, designated by the organization to function as a normative frame of reference. Through such developmental guidance, team members’ mental models of teamwork become more accurate and more compatible, insofar as they develop a correct understanding of what effective teamwork entails (Smith-Jentsch et al., 2001). We propose that for team members in training to ultimately function well together in performing team tasks, they must individually master the expert model of teamwork presented. In this study we explored the idea that teams consisting of members who, on average, demonstrate greater mastery of designated teamwork knowledge (in a personnel development program), demonstrate better team task proficiency (assessed objectively), and receive higher observer ratings of effective teamwork. The focus of our study is on pooled teamwork knowledge within teams which translates from the individual to the team level by way of an additive composition model and which represents the overall level (and thus, accuracy) of teamwork knowledge within teams. To the extent that members of a team, on average (additively), have learned the expert model of teamwork, the team may be characterized as possessing an accurate understanding of appropriate teamwork (which should predict successful team performance). Yet, this overall level of mastery within a team (a team’s resource pool) does not capture the extent to which the same mastery has been achieved among team members in their individual understanding of each element composing the expert model of teamwork. In contrast, a shared mental model of teamwork would be centered on team members possessing convergent structures of teamwork knowledge. That is, relative to the simplicity of pooled teamwork knowledge, a shared mental model of teamwork is a complex phenomenon that emerges from within-team consensus (or isomorphism) on various aspects of an overall scheme of teamwork (see Kozlowski & Bell, 2003; Kozlowski & Klein, 2000; Marks, Zaccaro, & Mathieu, 2000; Mathieu, Heffner, Goodwin, Salas, & Cannon-Bowers, 2000; Smith-Jentsch et al., 2001). The team literature suggests that a reasonably complete model of team-level intellectual resources and effectiveness would encompass a number of factors, to include pooled mental ability (an exogenous factor), pooled knowledge (a precursor to shared knowledge structures), shared mental models, various team processes (functioning as inputs to and outcomes of performance episodes), and team performance outcomes (Kozlowski & Bell, 2003). In our research we have included several factors that would be only part of a more comprehensive model. Accordingly, our focus is relatively limited and differs somewhat from a conventional model of input 3 process 3 outcome, insofar as we do not explicitly explore team processes as mediating factors (see Marks, Mathieu, & Zaccaro, 2001). Nevertheless, our underlying logic is that pooled teamwork knowledge engenders constructive team processes that are manifested in task proficiency and observed by external raters. We believe it is useful, therefore, to investigate the

RESEARCH REPORTS

role of pooled teamwork knowledge in predicting teams’ task proficiency and observed teamwork effectiveness. For general guidance regarding our specific hypotheses, we also referred to the individual-level and empirically based framework presented by Schmidt and Hunter (2004) and Hunter and Schmidt (1996). Part of that framework posits that individuals’ acquisition of job knowledge predicts their task proficiency (measured objectively using hands-on work-sample tests) and supervisor-rated performance. Moreover, these authors theorized job knowledge to predict supervisor ratings even while controlling for task proficiency. We submit that because the team-level factors we explored were somewhat similar (though not equivalent) to three of the individual-level factors in Schmidt and Hunter’s framework (i.e., knowledge, task proficiency, and rated performance), our research may be informed by their framework (even though we do not include mental ability). Nevertheless, our research should not be viewed as a direct team-level extension. Figure 1 presents our conceptual framework for exploring team members’ teamwork knowledge, additively aggregated to the team level, as a predictor of team task proficiency and observer ratings of effective teamwork. Teams consisting of members who demonstrate mastery of the organization’s conceptual model of effective teamwork should develop superior approaches for accomplishing team tasks and function well as teams. Therefore, as indicated in Figure 1, Hypothesis 1 states that team members’ mastery of designated teamwork knowledge predicts team task proficiency. In addition, it is likely that greater mastery of teamwork knowledge by team members is expressed in constructive team processes that are observable and valued by observers as indicative of effective teamwork, yet not entirely manifested in task proficiency (insofar as proficiency may also be shaped by team members’ psychomotor and physical skills). Therefore, as indicated in Figure 1, Hypothesis 2 states that team members’ mastery of designated teamwork knowledge predicts higher observer ratings of effective teamwork, even while controlling for task proficiency.

Research Setting and Development of the Hypothesized Structural Model The research setting, which determined the specific taskproficiency factors that were available for inclusion in the structural model, was a 5-week United States Air Force (USAF) officer development program (ODP). The ODP is designed to be a pro-

469

gram in which 5th- to 7th-year USAF officers step out of their specialties and acquire a transportable set of teamwork competencies (see Cannon-Bowers, Tannenbaum, Salas, & Volpe, 1995) as preparation for future positions involving greater responsibilities in team-centered military units that will likely face a wide variety of complex and dynamic challenges. Classroom activities and assigned readings in the ODP presented participants with information representing an expert model of USAF teamwork in general. In this type of setting, learning is tantamount to absorbing the knowledge presented (Hunter & Schmidt, 1996). In addition, teams of participants engaged in three types of tasks that underscored the development and use of teamwork in pursuing explicit standards of accomplishment.

Specific Team Tasks Field operations, which occurred on three consecutive Fridays (Weeks 1–3), involved three outdoor field campaigns involving competition between two teams. For each team, the field campaigns involved the same novel team sport, but the competitor team was different for each campaign. Teams competing against each other had the same number of members on the field at the same time, except when penalty time was levied. The field campaigns were dynamic in that teams had to respond promptly to the actions of their competitors. Problem solving, which occurred on Thursday of Week 2 (one exercise) and Wednesday of Week 4 (a different exercise), encompassed two exercises to be completed by teams within explicit time limits. Teams attempted to solve difficult problems representing realistic military scenarios (e.g., solving an enemy code by piecing together information given to each team member, while complying with specific parameters). The parameters were such that the code could not be successfully solved by only a few team members. Each of the two exercises consisted of two periods: a 45-min planning period and a 15-min execution period. Physical task accomplishment, which occurred on Tuesday of Week 3 (7 tasks) and Thursday of Week 4 (7 tasks), encompassed a total of 14 tasks to be completed by teams within explicit time limits. The time countdown began when each task was presented. Teams were expected to quickly develop an action plan and accomplish the required task in the allotted time. An example exercise was a team given 15 min to plan and execute crossing a river, with all of its equipment, without touching the water. The only physical resources available were a piece of rope and a board.

Figure 1. Guiding framework for the research hypotheses. Plus signs represent relationships hypothesized to be positive.

RESEARCH REPORTS

470 Hypothesized Structural Model

We investigated our research hypotheses by developing and testing the hypothesized structural model presented in Figure 2. We included the number of team members as a control variable, given that it may have had an influence on the predicted (endogenous) factors. Teams with more members would have greater resources for solving problems quickly but would likely require more time to complete the required physical tasks. Except for the number of team members, the position of the factors in Figure 2 (from left to right) roughly reflects the temporal order in which they occurred and were assessed. The posited directional relationships are not meant to imply cause and effect. As noted in Figure 2, three of the designated paths (i.e., Paths B, C, and F) represent the two research hypotheses. First, we specified (Path A) team members’ mastery of teamwork knowledge as being positively correlated with field-operations proficiency (most of the fieldoperations activity occurred in Week 1 and Week 2). As described earlier, Hypothesis 1 states that team members’ mastery of designated teamwork knowledge predicts team task proficiency. Accordingly, we specified teamwork knowledge as a predictor of problem-solving proficiency (Path B) and physical-task proficiency (Path C), even while controlling for field-operations proficiency as a predictor of problem-solving proficiency (Path D) and physical-task proficiency (Path E). To explore the robustness of teamwork knowledge (relative to that of field-operations proficiency) as a predictor of subsequent task proficiency, we also tested an alternative, more parsimonious model in which Paths D and E were eliminated. To assess Hypothesis 2, we specified teamwork knowledge as a predictor of higher observer ratings of effective teamwork (Path F), while controlling for fieldoperations proficiency (Path G), problem-solving proficiency (Path H), and physical-task proficiency (Path I) as positive contributors to observer ratings of effective teamwork.

Method The ODP was conducted at a large USAF base, and all 1,158 participants were full-time USAF officers with 5–7 years of commissioned service. Participants averaged 31 years of age; and 83% of them were men, and 17% were women. ODP participants were assigned to teams on the first day of the program; team members were not familiar with each other prior to teams being formed. Team assignments were determined by a computer model that considered variables such as participants’ demographic characteristics, job classifications, military status, and rank. Each team remained intact for the duration of the program. Data were collected on 92 teams. Teams ranged in size from 11 to 13 members, with 65 teams (71%) having 13 members. As such, the number of participants in each team was consistent with Levi’s (2001) characterization of teams as being typically composed of 4 –20 members who interact directly in striving toward collective performance outcomes. The focus of the ODP was on developing individuals’ transportable teamwork knowledge and skills as preparation for leadership functions entailing sociotechnical workplace issues, such as evaluating and shaping team processes (see Mohrman, 2003). Much of the information presented via classroom activities was new for the participants, whereas other information addressed a deeper understanding of teamwork concepts with which participants likely had some superficial familiarity. Through what they learn in the ODP, officers from various specialties and backgrounds also develop an understanding of appropriate teamwork in the USAF (as advocated by Cannon-Bowers et al., 1995, as well as Mohammed & Dumville, 2001). Of the 14 teamwork competencies enumerated by Stevens and Campion (1994), 12 were addressed in the ODP. In addition, all of the transportable teamwork competencies proposed by CannonBowers et al. (1995) as important for effective team performance were addressed. Given that the information presented was not specific to the idiosyncratic demands of any team task in the ODP, the content differed from that of the team-interaction training used by Marks et al. (2000). Classroom activities totaled approximately 22 hr in Week 1 and 19 hr in Week 2, and assigned readings amounted to hundreds of pages. Classroom time was devoted mostly to lecture but included several sessions in which

Figure 2. Hypothesized structural model. Number of team members is a control variable in the model. Letters A through I identify designated paths. Hypothesis 1 is represented by Paths B and C; Hypothesis 2 is represented by Path F. Plus signs represent relationships hypothesized to be positive.

RESEARCH REPORTS team members met with their entire team to review and discuss teamwork concepts and principles that were previously presented in readings, lecture, or both. On Friday of Week 2, participants individually took a test assessing their mastery of designated teamwork knowledge. For each team task, team members together had to learn the objectives and rules, formulate a strategy, organize their resources, develop plans for execution, assess their progress during execution, and implement corrective actions. Therefore, the nature of team interaction was consistent with Mohrman’s (2003) description of self-regulating teams. For two of the tasks, teams were action teams, insofar as they responded to and influenced their operating environment (see Marks et al., 2000). Yet, all three tasks met Wageman’s (2001) definition of task interdependence in that inputs into the work itself required multiple individuals to complete the tasks. Each team was assigned an observer (i.e., a USAF officer) who continually observed his or her team during all of the ODP activities. The primary functions of the observers were to watch the behavior of the team members, ensure that the team complied with the parameters of team tasks, and evaluate the leadership behavior and potential of each team member (after completion of the ODP). The observers neither functioned as active leaders nor intervened in team activities, and they had undergone extensive training on standardizing their methods of observation. Before team results on the factors were computed and compiled by personnel at a central office at the end of the ODP, observers were summoned to a meeting room and then asked to evaluate, for the purpose of our research only, the extent to which their team demonstrated effective teamwork. When providing their ratings, observers did not have the knowledge-test scores of their team members or the results of their team’s proficiency on the team tasks (or how their team compared with others).

Team Members’ Teamwork Knowledge Each team member independently completed a multiple-choice test on Friday of Week 2. The test measured mastery of the teamwork concepts or principles that were presented or reviewed in hundreds of pages of readings and the roughly 40 hr of classroom activities during the first 2 weeks of the ODP. The test did not assess knowledge pertaining specifically to the team tasks themselves, and a number of items were similar to those presented by Stevens and Campion (1994). Individual team members could score up to 100 points on the test (75 was the minimum for a passing score). We used the mean of team members’ scores to aggregate teamwork knowledge to the team level. Team scores ranged from 80.87 to 94.73 points (M ⫽ 85.08, SD ⫽ 2.36).

Team Task Proficiency Field-operations proficiency. Teams’ proficiency on field operations was tallied as a function of (a) whether a team won each of its three field campaigns (4 points for a victory, 2 points for a tie) and (b) the margin of each victory (e.g., 6 bonus points for winning by a score margin of 10 or more). Team scores ranged from 2 to 28 (M ⫽ 14.64, SD ⫽ 5.98).

471

Problem-solving proficiency. Teams’ proficiency on problem solving was tallied as a function of (a) whether each of the two problems was successfully solved (5 points each) and (b) the amount of time taken to solve a problem (e.g., 2 bonus points for being in the top 10% on a given task). Team scores ranged from 0 to 14 points (M ⫽ 6.17, SD ⫽ 3.61). Physical-task proficiency. Each successfully accomplished task (of 14 total tasks) was worth 1 point. In addition, teams received 1 bonus point for accomplishing all seven tasks on Tuesday of Week 3 and 1 bonus point for accomplishing all seven tasks on Thursday of Week 4. Team scores ranged from 3 to 16 points (M ⫽ 9.91, SD ⫽ 2.82).

Observer Ratings of Effective Teamwork To obtain observer ratings of teamwork, for research purposes only, we asked each team’s observer to assess his or her team’s behavior over the course of the ODP. To develop the measure, we asked four observers, serving as subject matter experts, to identify the main factors they would consider if asked to assess overall teamwork. On the basis of 100% agreement among the subject matter experts, the following components of effective teamwork emerged: (a) a team’s characteristic level of effort toward task accomplishment; (b) team members’ level of commitment to helping each other so that team performance would be maximized; (c) the extent to which a team had active participation, communication, and interaction among all of its members; and (d) the extent to which a team demonstrated an appropriate use of time, material resources, and member expertise in striving to accomplish different team tasks. The four components are congruent with aspects of teamwork identified by McIntyre and Salas (1995) and the team processes rated in Mathieu et al.’s (2000) study. Behavioral examples of extremely low and high levels were provided for each of the four components. In providing their ratings, observers used a rating scale that ranged from extremely low (1) to extremely high (6). We calculated scores by averaging the four item ratings. Coefficient alpha was .90 and team scores ranged from 2.00 to 6.00 (M ⫽ 4.35, SD ⫽ 0.99).

Results Table 1 presents the means, standard deviations, and intercorrelations for the study variables. To test our theoretical model, we used structural equation modeling. We used four indicators of effective teamwork and one indicator for each of the five predictive factors: number of team members, teamwork knowledge, field-operations proficiency, problem-solving proficiency, and physical-task proficiency. For number of team members, we fixed lambda (factor 3 indicator parameter) to 1.0 and theta (random error variance) to 0. For each of the other factors represented by one indicator, we fixed lambda to 1.0 and theta to 1.0 minus the estimated reliability of the manifest variable multiplied by the

Table 1 Variable Means, Standard Deviations, and Intercorrelations Manifest variable 1. 2. 3. 4. 5. 6.

Number of team members Team members’ teamwork knowledge Field-operations proficiency Problem-solving proficiency Physical-task proficiency Observer ratings of effective teamwork

M

SD

1

2

3

4

5

6

12.58 85.08 14.64 6.17 9.91 4.35

0.71 2.36 5.98 3.61 2.82 0.99

— .09 ⫺.03 .22* ⫺.26** .06

— .28** .29** .24** .47***

— .20* .19* .53***

— .05 .37***

— .32***

(.90)

Note. N ⫽ 92 teams. All variables were operationalized at the team level. Coefficient alpha is listed in parentheses on the diagonal for observer ratings of effective teamwork. Dashes indicate variables for which the computation of alpha was inappropriate. * p ⬍ .05, one-tailed. ** p ⬍ .01, one-tailed. *** p ⬍ .001, one-tailed.

472

RESEARCH REPORTS

variance of the observed score (e.g., Frone, Russell, & Cooper, 1992; Gellatly, 1996; Netemeyer, Johnston, & Burton, 1990). We used the conventional value of .90 (e.g., Anderson & Gerbing, 1988; Chen, Gully, Whiteman, & Kilcullen, 2000) as the estimated reliability for teamwork knowledge, field-operations proficiency, problem-solving proficiency, and physical-task proficiency. To assess model fit, we used four fit indices: the comparative fit index (CFI), goodness-of-fit index (GFI), adjusted goodness-of-fit index (AGFI), and root mean square error of approximation (RMSEA). Researchers generally agree that a model fits the data well when CFI, GFI, and AGFI values are .90 or higher (Mulaik et al., 1989) and when the RMSEA value is .08 or lower (Browne & Cudeck, 1993). The hypothesized structural model provided a very good fit to the data, ␹2(20, N ⫽ 92) ⫽ 18.88, p ⫽ .53, CFI ⫽ 1.00, GFI ⫽ .96, AGFI ⫽ .90, RMSEA ⫽ .00. However, the number of team members did not predict observer ratings of effective teamwork. Therefore, we dropped this path in testing an alternative (more parsimonious) model. In addition, to test the robustness of teamwork knowledge (relative to that of field-operations proficiency) as a predictor of teams’ subsequent task proficiency, we dropped two additional paths: (a) field-operations proficiency 3 problemsolving proficiency (i.e., Path D) and (b) field-operations proficiency 3 physical-task proficiency (i.e., Path E). The increase of three degrees of freedom was not associated with a significant increase in chi-square, ⌬␹2(3, N ⫽ 92) ⫽ 2.19, p ⬎ .50. Moreover, the values of the fit indices continued to be favorable, ␹2(23, N ⫽ 92) ⫽ 21.07, p ⫽ .58, CFI ⫽ 1.00, GFI ⫽ .95, AGFI ⫽ .90, RMSEA ⫽ .00. Owing to the relative parsimony of the alternative model, it is preferred. Figure 3 shows path coefficients from the alternative model; p values represent directional (one-tailed) significance levels. Teamwork knowledge was associated with field-operations proficiency (␤ ⫽ .32, p ⬍ .005) and directly predicted problem-solving pro-

ficiency (␤ ⫽ .31, p ⬍ .005), physical-task proficiency (␤ ⫽ .30, p ⬍ .005), and effective teamwork (␤ ⫽ .25, p ⬍ .01). In addition to being directly predicted by teamwork knowledge, effective teamwork (R2 ⫽ .51) was directly predicted by field-operations proficiency (␤ ⫽ .44, p ⬍ .0005), problem-solving proficiency (␤ ⫽ .22, p ⬍ .025), and physical-task proficiency (␤ ⫽ .17, p ⬍ .05). Additional results indicated that teamwork knowledge had a total relationship of .37 ( p ⬍ .0005) with effective teamwork, which consisted of a direct relationship of .25 ( p ⬍ .01) and an indirect relationship of .12 ( p ⬍ .025) by way of paths through problem-solving proficiency and physical-task proficiency.

Discussion All of the teams in this study were newly formed (i.e., observed relationships were not influenced by team phenomena occurring prior to the ODP). Although observers were present for team tasks, they did not compute or compile the knowledge or proficiency scores reported in this study and, therefore, did not have them when rating teamwork effectiveness. Nevertheless, fieldoperations proficiency was the best predictor of observed teamwork effectiveness, suggesting that field operations were relatively salient to observers. In comparison with team problem solving and physical-task accomplishment, considerably more time was allocated to field operations. In addition, field operations entailed direct competition against other teams. It is possible that intensive interdependence and mutual adjustments required among team members during competition made the extent of their demonstrated teamwork more evident to observers. In addition, however, subject matter experts suggested that observers would rather easily be able to recall the win–loss record of their team, perhaps contributing to the salience of field operations. Postrel (2002) suggested that knowledge acquisition and its application have a dis-

Figure 3. Final empirical model. Path values represent standardized estimates; one-tailed significance levels are in parentheses.

RESEARCH REPORTS

tinguishable theoretical existence. To separate the relationships of teamwork knowledge and its initial application with the endogenous factors in our structural model, we controlled for fieldoperations proficiency in testing teamwork knowledge as a predictor of problem-solving proficiency, physical-task proficiency, and observed teamwork effectiveness. We believe that this partialing of variance contributes to the methodological rigor of this study and, by extension, the level of support that the findings provide for the importance of team members’ additive pool of teamwork knowledge. Although the teamwork knowledge test was taken the day after the initial problem-solving exercise (in Week 2), roughly 41 hr of classroom activities (the basis for test content) preceded the initial problem-solving exercise, and the second problem-solving exercise did not occur until the middle of Week 4. In addition, the physical tasks did not begin until Week 3. Despite content and temporal differences among the three types of team tasks, the relationships between teamwork knowledge and teams’ proficiency on the three types of tasks were virtually identical in our multivariate structural model (ranging from .30 to .32, as shown in Figure 3). As such, the relevance of teamwork knowledge for task proficiency was uniform. In addition, given that we controlled for teams’ proficiency on three types of tasks, the direct relationship between teamwork knowledge and teamwork effectiveness (␤ ⫽ .25, p ⬍ .01) supports the idea that teams consisting of members who have a mastery of appropriate teamwork knowledge display constructive team processes that are observable and not entirely manifested in team task proficiency. The predictive efficacy of teamwork knowledge has meaningful implications for the training and development of individuals for the role of team member. Smith-Jentsch et al.’s (2001) results suggest that team members move toward the same correct understanding of teamwork when they receive equivalent training aimed at guiding them toward an expert model of teamwork. Our results build on those of SmithJentsch et al. (2001) in suggesting that team members achieving greater individual mastery of designated teamwork knowledge facilitate better team task proficiency and greater teamwork effectiveness. Although team members’ individual knowledge provides a team with a mental resource pool, it would be meaningful for future research to directly capture the shared and unshared knowledge that composes the pool, rather than only the overall (average) level of the resource pool. With the additive method of aggregation (which we used), a team score of 80 would emerge not only from all team members scoring an 80 on exactly the same items (i.e., if 20% of the items were not mastered by any team member) but also from all team members scoring an 80 while getting different items correct and incorrect, such that 100% of the test items were mastered by at least one team member. Accordingly, future research might benefit from using more complex aggregations of teamwork knowledge (see Kozlowski & Klein, 2000). Other limitations of the present study are that we investigated teamwork effectiveness as only a criterion and that we did not explore additional team processes. Future research should, therefore, investigate differentiated team processes as inputs to and outcomes of various team performance episodes over time (Marks et al., 2001). Given that our results deal with only teamwork knowledge, they do not pertain to other types of knowledge possessed by team

473

members. Although we focused on the extent to which designated teamwork knowledge was mastered by team members, it is likely that the variety of task work or technical expertise distributed among team members (individual specialists) is generally important for team performance. Mohammed and Dumville (2001) noted that the diversity of task work expertise among team members is a main reason for forming teams in the first place, insofar as team members possessing different technical knowledge offer a larger pool of intellectual resources for dealing with multifaceted team endeavors. Therefore, it is important for researchers to use complex methods of aggregation in exploring how teams combine various information or ideas held by different members and how teams use the distinct contributions of different individuals to solve multifaceted problems and accomplish multifaceted team tasks (Kozlowski et al., 1999; Kozlowski & Klein, 2000). Such research may be pursued by investigating a transactive memory system that combines the unique knowledge possessed by specific team members with a shared and accurate awareness among team members of who knows what (Austin, 2003; Mohammed & Dumville, 2001). Whereas a mental model of teamwork transcends team membership (Smith-Jentsch et al., 2001), transactive memory develops over time from repeated interactions among team members and is, therefore, embedded in team membership (Austin, 2003). In closing, McIntyre and Salas (1995) suggested that to optimize team performance in organizations, leaders must identify what constitutes effective teamwork in their particular organization and incorporate this conceptualization of teamwork into the organization’s operating philosophy as well as in resulting efforts to develop individuals and teams. Our results are consistent with this notion in suggesting that teams are likely to perform better when they consist of members who, through training provided by the organization, master designated knowledge of what effective teamwork entails.

References Anderson, J. C., & Gerbing, D. W. (1988). Structural equation modeling in practice: A review and recommended two-step approach. Psychological Bulletin, 103, 411– 423. Austin, J. R. (2003). Transactive memory in organizational groups: The effects of content, consensus, specialization, and accuracy on group performance. Journal of Applied Psychology, 88, 866 – 878. Barrick, M. R., Stewart, G. L., Neubert, M. J., & Mount, M. K. (1998). Relating member ability and personality to work-team processes and team effectiveness. Journal of Applied Psychology, 83, 377–391. Browne, M. W., & Cudeck, R. (1993). Alternative ways of assessing model fit. In K. A. Bollen & J. S. Long (Eds.), Testing structural equation models (pp. 136 –162). Newbury Park, CA: Sage. Campbell, J. P. (1999). The definition and measurement of performance in the new age. In D. R. Ilgen & E. D. Pulakos (Eds.), The changing nature of performance: Implications for staffing, motivation, and development (pp. 399 – 429). San Francisco: Jossey-Bass. Cannon-Bowers, J. A., Tannenbaum, S. I., Salas, E., & Volpe, C. E. (1995). Defining competencies and establishing team training requirements. In R. A. Guzzo, E. Salas, & Associates (Eds.), Team effectiveness and decision making in organizations (pp. 333–380). San Francisco: JosseyBass. Chan, D. (1998). Functional relations among constructs in the same content domain at different levels of analysis: A typology of composition models. Journal of Applied Psychology, 83, 234 –246. Chen, G., Gully, S. M., Whiteman, J., & Kilcullen, R. N. (2000). Exami-

474

RESEARCH REPORTS

nation of relationships among trait-like individual differences, state-like individual differences, and learning performance. Journal of Applied Psychology, 85, 835– 847. Cohen, S. G., & Bailey, D. E. (1997). What makes teams work: Group effectiveness research from the shop floor to the executive suite. Journal of Management, 23, 239 –290. DeNisi, A. S., Hitt, M. A., & Jackson, S. E. (2003). The knowledge-based approach to sustainable competitive advantage. In. S. E. Jackson, M. A. Hitt, & A. S. DeNisi (Eds.), Managing knowledge for sustained competitive advantage: Designing strategies for effective human resource management (pp. 3–33). San Francisco: Jossey-Bass. Eden, D. (2002). Replication, meta-analysis, scientific progress, and AMJ’s publication policy. Academy of Management Journal, 45, 841– 846. Ellis, A. P. J., Hollenbeck, J. R., Ilgen, D. R., Porter, C. O. L. H., West, B. J., & Moon, H. (2003). Team learning: Collectively connecting the dots. Journal of Applied Psychology, 88, 821– 835. Forsyth, D. (1999). Group dynamics (3rd ed.). Belmont, CA: Wadsworth. Frone, M. R., Russell, M., & Cooper, M. L. (1992). Antecedents and outcomes of work–family conflict: Testing a model of the work–family interface. Journal of Applied Psychology, 77, 65–78. Gellatly, I. R. (1996). Conscientiousness and task performance: Test of a cognitive process model. Journal of Applied Psychology, 81, 474 – 482. Goldstein, I. L., & Ford, J. K. (2002). Training in organizations: Needs assessment, development, and evaluation. Belmont, CA: Wadsworth. Howard, A. (1995). A framework for work change. In A. Howard (Ed.), The changing nature of work (pp. 3– 44). San Francisco: Jossey-Bass. Hunter, J. E., & Schmidt, F. L. (1996). Intelligence and job performance: Economic and social implications. Psychology, Public Policy, and Law, 2, 447– 472. Ilgen, D. R., & Pulakos, E. D. (1999). Introduction: Employee performance in today’s organizations. In D. R. Ilgen & E. D. Pulakos (Eds.), The changing nature of performance: Implications for staffing, motivation, and development (pp. 1–18). San Francisco: Jossey-Bass. Kozlowski, S. W. J., & Bell, B. S. (2003). Work groups and teams in organizations. In W. C. Borman, D. R. Ilgen, & R. J. Klimoski (Eds.), Handbook of psychology: Vol. 12. Industrial and organizational psychology (pp. 333–375). New York: Wiley. Kozlowski, S. W. J., Gully, S. M., Nason, E. R., & Smith, E. M. (1999). Developing adaptive teams: A theory of compilation and performance across levels and time. In D. R. Ilgen & E. D. Pulakos (Eds.), The changing nature of performance: Implications for staffing, motivation, and development (pp. 240 –292). San Francisco: Jossey-Bass. Kozlowski, S. W. J., & Klein, K. J. (2000). A multilevel approach to theory and research in organizations: Contextual, temporal, and emergent processes. In K. J. Klein & S. W. J. Kozlowski (Eds.), Multilevel theory, research, and methods in organizations: Foundations, extensions, and new directions (pp. 3–90). San Francisco: Jossey-Bass. Lankau, M. J., & Scandura, T. A. (2002). An investigation of personal learning in mentoring relationships: Content, antecedents, and consequences. Academy of Management Journal, 45, 779 –790. LePine, J. A. (2003). Team adaptation and postchange performance: Effects of team composition in terms of members’ cognitive ability and personality. Journal of Applied Psychology, 88, 27–39. Levi, D. (2001). Group dynamics for teams. Thousand Oaks, CA: Sage. London, M., & Mone, E. M. (1999). Continuous learning. In D. R. Ilgen & E. D. Pulakos (Eds.), The changing nature of performance: Implications for staffing, motivation, and development (pp. 119 –153). San Francisco: Jossey-Bass. Lykken, D. T. (1968). Statistical significance in psychological research. Psychological Bulletin, 70, 151–159. Marks, M. A., Mathieu, J. E., & Zaccaro, S. J. (2001). A temporally based framework and taxonomy of team processes. Academy of Management Review, 26, 356 –376.

Marks, M. A., Zaccaro, S. J., & Mathieu, J. E. (2000). Performance implications of leader briefings and team-interaction training for team adaptation to novel environments. Journal of Applied Psychology, 85, 971–986. Mathieu, J. E., Heffner, T. S., Goodwin, G. F., Salas, E., & CannonBowers, J. A. (2000). The influence of shared mental models on team process and performance. Journal of Applied Psychology, 85, 273–283. McIntyre, R. M., & Salas, R. (1995). Measuring and managing for team performance: Emerging principles from complex environments. In R. A. Guzzo, E. Salas, & Associates (Eds.), Team effectiveness and decision making in organizations (pp. 9 – 45). San Francisco: Jossey-Bass. Mohammed, S., & Dumville, B. C. (2001). Team mental models in a team knowledge framework: Expanding theory and measurement across disciplinary boundaries. Journal of Organizational Behavior, 22, 89 –106. Mohrman, S. A. (2003). Designing work for knowledge-based competition. In. S. E. Jackson, M. A. Hitt, & A. S. DeNisi (Eds.), Managing knowledge for sustained competitive advantage: Designing strategies for effective human resource management (pp. 94 –123). San Francisco: Jossey-Bass. Mulaik, S. A., James, L. R., Van Alstine, J., Bennett, N., Lind, S., & Stilwell, C. D. (1989). Evaluation of goodness-of-fit indices for structural equation models. Psychological Bulletin, 105, 430 – 445. Murphy, P. R., & Jackson, S. E. (1999). Managing work role performance: Challenges for twenty-first century organizations and their employees. In D. R. Ilgen & E. D. Pulakos (Eds.), The changing nature of performance: Implications for staffing, motivation, and development (pp. 325– 365). San Francisco: Jossey-Bass. Netemeyer, R. G., Johnston, M. W., & Burton, S. (1990). Analysis of role conflict and role ambiguity in a structural equations framework. Journal of Applied Psychology, 75, 148 –157. Noe, R. A., Colquitt, J. A., Simmering, M. J., & Alvarez, S. A. (2003). Knowledge management: Developing intellectual and social capital. In. S. E. Jackson, M. A. Hitt, & A. S. DeNisi (Eds.), Managing knowledge for sustained competitive advantage: Designing strategies for effective human resource management (pp. 209 –242). San Francisco: JosseyBass. Postrel, S. (2002). Islands of shared knowledge: Specialization and mutual understanding in problem-solving teams. Organization Science, 13, 303–320. Schmidt, F. L., & Hunter, J. (2004). General mental ability in the world of work: Occupational attainment and job performance. Journal of Personality and Social Psychology, 86, 162–173. Smith-Jentsch, K. A., Campbell, G. E., Milanovich, D. M., & Reynolds, A. M. (2001). Measuring teamwork mental models to support training needs assessment, development, and evaluation: Two empirical studies. Journal of Organizational Behavior, 22, 179 –194. Stevens, M. J., & Campion, M. A. (1994). The knowledge, skill, and ability requirements for teamwork: Implications for human resource management. Journal of Management, 20, 503–530. Terborg, J. R., Castore, C., & DeNinno, J. A. (1976). A longitudinal field investigation of the impact of group composition on group performance and cohesion. Journal of Personality and Social Psychology, 34, 782– 790. Tziner, A., & Eden, D. (1985). Effects of crew composition on crew performance: Does the whole equal the sum of its parts? Journal of Applied Psychology, 70, 85–93. Wageman, R. (2001). The meaning of interdependence. In M. E. Turner (Ed.), Groups at work: Theory and research (pp. 197–217). Hillsdale, NJ: Erlbaum.

Received March 24, 2004 Revision received November 16, 2004 Accepted December 22, 2004 䡲