Appraisal1

Int. J. of Human Resource Management 16:11 November 2005 2029– 2044

The role of interpersonal affect in performance appraisal: evidence from two samples – the US and India

Arup Varma, Shaun Pichler and Ekkirala S. Srinivas Abstract The interpersonal affect, a li ke – dislike relationship between a supervisor and his/her subordinate, has traditionally been conceptualized as a source of bias in performance appraisals. However, some researchers have argued that the interpersonal affect may not be a bias, especially where it develops as a result of past performance. In this field study, using data from 190 supervisors in the US, and 113 supervisors in India, we delineate the relationship between interpersonal affect and performance ratings. In both samples, interpersonal affect and performance level were found to have significant effects on performance ratings. Results from the US sample indicated that raters are able to separate their liking for a subordinate from actual performance when a ssigning performance ratings, suggesting that the interpersonal affect does not operate as a bias in the appraisal process. Results from the Indian sample, however, suggest that supervisors inflate ratings of low performers, suggesting that local cultural norms may be operating as a moderator. Keywords Interpersonal affect; performance appraisal; performance ratings; crosscultural studies; bias in appraisals; US versus India.

Introduction Over the years, several researchers (see, e.g., Zajonc, 1980 and Dipboye, 1985) have suggested that the performance appraisal literature has ignored affective and behavioural variables. Since the publication of these reviews, several researchers have attempted to integrate the interpersonal affect into studies of performance appraisal. A majority of these studies treat interpersonal affect as a potential source of bias in performance ratings (see, e.g. Landy and Farr, 1980; Latham and Wexley, 1981). In this study, we investigate the relationship between interpersonal affect and performance ratings, and further examine whether interpersonal affect operates as a bias under all performance conditions. Additionally, we expand the growing literature on cross-national comparative analyses, by conducting our investigations in two separate samples – US and India. Arup Varma, Institute of Human Resources & Industrial Relations, Loyola University Chicago, 820 North Michigan Avenue, Chicago, IL 60611, USA (tel: þ 1-312-915-6664; fax: þ1-312-915-6231; e-mail: [email protected]). Shaun Pichler, School of Labor and Industrial Relations, 4th Floor, South Kedzie Hall, Michigan State University, East Lansing, MI 48824, USA (tel: þ1-517-4106046; fax: þ1-619-465-4019; e-mail: [email protected]). Ekkirala S. Srinivas, Xavier Labor Relations Institute (XLRI), PO Box 222, Circuit House Area (East), Jamshedpur 831001, India (tel: þ91-657-222-5506; fax: þ 91-657-222-7814; e-mail: [email protected]). The International Journal of Human Resource Management ISSN 0958-5192 print/ISSN 1466-4399 online q 2005 Taylor & Francis http://www.tandf.co.uk/journals DOI: 10.1080/09585190500314904

2030 The International Journal of Human Resource Management The interpersonal affect is defined as a ‘like–dislike relationship’ between a supervisor and his/her subordinate, and has been shown to occur very early in stimulus observation (Zajonc, 1980) and performance evaluation (Cardy and Dobbins, 1986). In other words, if a supervisor likes his/her subordinate, s/he is deemed to have a high interpersonal affect toward that subordinate. In this connection, research has consistently indicated that a rater’s interpersonal affect towards a ratee is difficult to separate from performance information when assigning ratings (Cardy and Dobbins, 1986; Robbins and DeNisi, 1994). As such, if interpersonal affect for a ratee develops before the rater processes performance-related information, and is difficult to disconnect from actual performance, it is logical to argue that interpersonal affect is a source of bias in performance appraisal, diminishing rater accuracy. Recent research and theory suggest, however, that this conclusion may be premature (e.g. Lefkowitz, 2000; Robbins and DeNisi, 1998; Varma et al., 1996). Most of the studies examining interpersonal affect have been conducted in the laboratory, where interpersonal affect and performance levels can be easily manipulated. The laboratory, however, is not the best setting for studying interpersonal affect, and the external validity of laboratory results is often suspect (Lefkowitz, 2000). Indeed, interpersonal affect develops over time between a supervisor and subordinate, and systematically influences the performance rating process (Robbins and DeNisi, 1998). Further, the relationship between a supervisor and subordinate is also a developmental process (Bauer and Green, 1996) that is a function of the length of the relationship. As such, the effects observed in the laboratory may differ in important ways to the effects of interpersonal affect in the field. Indeed, it was Dipboye’s (1985) criticism that passiveobserver research had led to such an over-emphasis on cognitive determinants of performance ratings that directed researchers to studying affective variables in the appraisal process in the first place. Clearly, there is a critical disconnect between performance appraisal research and practice (Arvey and Murphy, 1998). Accordingly, this paper adds to the growing body of literature that examines the relationship between interpersonal affect and performance ratings in the field. This is an important way to facilitate the unification of research findings and practice in organizational settings. India versus the US Furthermore, we combine the interest in performance appraisal with the growing interest in cross-national research, specifically comparing the US and India. The recent media coverage of US jobs being outsourced to India, and the attractiveness of India as a potential investment location due to its large domestic market (Datt and Sundharam, 1999) make it a prime candidate for a cross-national study. Indeed, several authors have called for more cross-national studies (e.g. Brewster et al., 1996). However, most of the cross-cultural research in the existing management literature is limited to studying and/or comparing management practices in European and/or North American countries (Budhwar and Debrah, 2001; Napier and Vu, 1998). This seems rather ironic, given that multinational corporations (MNCs) are increasingly establishing operations in developing countries such as India and China, and foreign direct investment to these nations has also substantially increased (Budhwar and Debrah, 2004). Clearly, if organizations are to succeed in their international operations, they need a clear understanding of local workplace norms and management practices (Hofstede, 1993). In this connection, several authors (e.g. Kapur and Ramamurti, 2001) have noted that India is becoming an increasingly important

Varma et al.: Interpersonal affect in performance appraisal

2031

player on the world economic scene, and is projected to have the fourth largest economy by 2020, according to World Bank estimates. This is not surprising, given that India offers several unique features that are very attractive to MNCs – a highly-skilled and educated workforce, a rapidly growing middle class, and an English-speaking populace. Historically, India has shared a special relationship with the US. For decades, Indian students have continued to flock to the US for higher studies (Harbison and Myers, 1964; Kapur and Ramamurti, 2001), while the US has been the largest foreign investor in India (Budhwar, 2001). Clearly, this exchange of human resources between these two nations is likely to continue and increase in the foreseeable future. As such, an understanding of the unique cultures and workplace norms would help organizations in both nations develop suitable policies and procedures (Gopalan and Stahl, 1998) to help them succeed. Given the context-specific nature of HR practices (Budhwar and Khatri, 2001), it is important that organizations understand the unique factors that might impact the design, implementation, and success or failure of management systems in different countries (e.g. Schuler et al., 2002). Performance appraisal – India versus the US In the Indian scenario, it is important that US organizations (and managers) become aware that ‘hierarchy and inequality are deeply rooted in India’s traditions’ (Jain and Venkata Ratnam, 1994). As Sinha (1990) points out, this often leads to subordinates being dependent on their supervisors to help resolve their problems. In the context of performance appraisal, this would mean that subordinate performance (and, hence, performance rating) is much more dependent on the supervisor’s relationship with the subordinate, than it would be in the US. Furthermore, India is a ‘collectivist’ nation (Hofstede, 1980), and managers are likely to motivate their subordinates through social and interpersonal interventions (Budhwar and Khatri, 2001). The US, on the other hand, is deemed to be much more ‘individualistic’, and managers are less likely to rely on social and interpersonal interactions in their dealings with subordinates. In this connection, Kanungo and Mendonca (1994) suggest that employees in India are often more concerned with their personal relationships with the supervisor, instead of the actual job performance. Again, this situation is very different from what one might expect to find in the US, where work-related outcomes take precedence over personal relationships in the workplace. It is in this context that we were interested in studying the impact of interpersonal affect on performance appraisal. Clearly, if being ‘liked’ by the supervisor is more important to the subordinate than the actual job performance, as in India, interpersonal affect is more likely to have influence the subordinate’s performance evaluation. Interpersonal affect as bias in performance appraisal A review of the relevant literature reveals that evidence on interpersonal affect as a source of bias in performance appraisals, is mixed, at best. Specifically, some scholars (e.g. Dipboye, 1985) have suggested that affect influences performance ratings independent of actual performance, and some evidence confirms this (e.g. Dobbins, 1982; Ferris et al., 1994; Lefkowitz and Battista, 1995). For example, Tsui and Barry (1986) suggest that affect is a source of bias in appraisal, as it reduces rater accuracy in performance ratings. Indeed, several laboratory studies (e.g. Cardy and Dobbins, 1986; Wayne and Ferris, 1990) have reported that interpersonal affect has a significant positive, main effect on performance ratings.

2032 The International Journal of Human Resource Management However, in recent times, some authors have even suggested that interpersonal affect might develop as a result of performance (Robbins and DeNisi, 1994, 1998; Varma et al., 1996). On the other hand, other authors (e.g. Ferris et al., 1994) have argued that affect may not be a result of past performance, and that raters simply give higher ratings to those subordinates that they like. In this connection, past research has shown that supervisors prefer to work with high performers (e.g. Farris and Lim, 1969). High performers may be easier to get along with, and may be more well liked, simply because they do not need to be reprimanded for poor work behaviour. Supervisors may also be naturally inclined to working with those subordinates that can most easily be identified as prote´ge´s. In this connection, Olian et al. (1993) found that mentors were more willing to engage in career enhancing as well as friendship behaviours with high performing subordinates, and expected greater rewards from a mentoring relationship with high performers. As such, it is logical to argue that interpersonal affect is, at least partially, explained by past performance and, therefore, is not necessarily a source of bias in performance appraisal. However, robust field evidence has yet to resolve this issue. As such, the present study was designed to test this issue in the field setting, in both nations – US and India. Hypotheses Interpersonal affect and performance level In laboratory studies, actual job performance has consistently been found to positively influence performance ratings (e.g. Carey, 1997; DeNisi and Stevens, 1981; Robbins and DeNisi, 1994; Wayne and Ferris, 1990). In other words, better performance garners better ratings. Indeed, ‘studies of the effect of the performance level of the ratee on performance ratings generally support the validity of the ratings’ (Landy and Farr, 1980). Clearly, objective performance is a strong predictor of performance ratings, and any model of the appraisal process should highlight this. As such, we propose that, in both samples: Hypothesis 1: Performance level will have a significant, positive effect on performance ratings. Some studies have shown that performance and interpersonal affect interact to influence performance ratings. For instance, Cardy (1982) found that performance and interpersonal affect interact to influence the differential accuracy of performance ratings. Similarly, while interpersonal affect was not found to influence ratings of consideration in another laboratory study (Dobbins, 1982), it did interact with performance, such that when the liking manipulation occurred prior to observing behaviour, liked low performers were rated more favourably than a disliked low performer. Exploring such interaction effects in the field should shed light on whether or not interpersonal affect operates as a bias, and the primary intention of this study is to closely examine the interaction between interpersonal affect and performance. We propose that for both samples: Hypothesis 2: Performance level and interpersonal affect will interact to influence performance ratings. Inflation and deflation – underevaluation and overevaluation In a series of experiments, Katz and Glass (1979) demonstrated empirically what they termed the ambivalence-amplification response. The researchers argued that majority

Varma et al.: Interpersonal affect in performance appraisal

2033

raters have ambivalent feelings towards minority ratees. A rater’s response towards a minority ratee is amplified either positively or negatively depending on the context of the situation or interaction. If a rater has a negative experience with a particular minority ratee, the performance rating will be significantly more negative than for a non-minority ratee. Likewise, if a rater has a positive experience with a ratee, the rating will be significantly higher than a rating given to a non-minority ratee. Therefore, performance ratings towards minority ratees are biased by the type of interpersonal affect a rater has towards a ratee. While the conclusions of the Katz and Glass (1979) experiments do not necessarily indicate that ratings will generally be inflated or deflated because of an interpersonal affect bias, they certainly provide a framework to test the biasing effects of interpersonal affect. Specifically, if ratings are inflated for low performers when interpersonal affect is high, or if ratings are deflated for high performers when interpersonal affect is low, then interpersonal affect can be said to bias the evaluation. Testing this type of relationship, of course, is dependent on an interaction effect between performance level and interpersonal affect. Interestingly, Robbins and DeNisi (1994) found that ratings were deflated when raters had negative interpersonal affect, but ratings were not inflated as a result of positive interpersonal affect. That is, ratings were lower when a rater had negative interpersonal affect than when a rater had neutral or positive interpersonal affect. In this connection, Feldman (1981) and others (Ilgen and Feldman, 1983; Srull and Wyer, 1980) have argued that information about ratees is stored in memory on the basis of categories, or prototypes, and that systematic underevaluation and overevaluation of a ratee is based on category membership (Feldman, 1981). If, for instance, a ratee is categorized as a likeable subordinate, recall of ratee performance information would be biased by the general evaluative implication of this category –which is clearly positive – and the ratee would be given an inaccurately favourable performance rating based on his or her category membership. Based on the above discussion, we predict systematic underevaluation and overevaluation of a ratee based on interpersonal affect. Specifically, if interpersonal affect operates as a source of bias in performance appraisal, there should be a significant mean difference in ratings between raters with neutral interpersonal affect, and raters with positive interpersonal affect among the low performing group. In this case, interpersonal affect could be interpreted to inflate ratings. Conversely, if there is a significant mean difference in ratings between raters with neutral interpersonal affect, and raters with negative interpersonal affect among the high performing group, then interpersonal affect could be interpreted to deflate ratings. As such, we predict that, in both samples: Hypothesis 3: Performance ratings for high performers will be deflated by raters with negative interpersonal affect towards them, and Hypothesis 4: Performance ratings for low performers will be inflated by raters with positive interpersonal affect toward them. Method Data for our study were collected through contacts in the HR departments of participating organizations. The US sample consisted of 190 supervisors drawn from a large manufacturing corporation, while the Indian sample consisted of 113 supervisors, also drawn from a large manufacturing organization. In the US sample, surveys were distributed to roughly 500 supervisors, giving us a return rate of 38 per cent, while in the Indian sample, the return rate was 30 per cent (113 out of 380).

2034 The International Journal of Human Resource Management Fifty-seven per cent of the participants in the US sample were male, and their average age was 42. The average tenure with the company was 12 years, while the average amount of work experience of participants was 20 years. Since each supervisor rated two ratees, our effective sample size is 380. In the Indian sample, 89 per cent of the participants were male, and their average age was also 42. The average company tenure was 16.5 years, and the average work experience was 18 years. Since each supervisor rated two ratees, our effective sample size for this group is 226. Procedure The supervisors participating in our study, in both samples, were asked first to think of a high performer among their direct reports, and provide performance and interpersonal affect ratings for this subordinate. Next, supervisors were asked to think of a lowperformer, and provide the same ratings for this subordinate. Supervisors from both settings provided ratings of subordinates using the same survey instrument. Measures Independent variables Interpersonal affect (alpha ¼ 0.85 US, 0.73 India) was measured using five items (e.g. ‘I think this person could become a good friend of mine’) drawn from Tsui and Barry (1986). Subordinate performance level was manipulated by instructing the participating supervisors to think of a high performer and a low performer. Independent samples t-tests indicated that there were significant differences between high performers and low performers on the overall performance rating for the US sample [t (377) ¼ 1.057, p , 0.01] as well as the Indian sample [t (249) ¼ 11.208, p , 0.01]. Dependent variables Respondents rated subordinates on both trait and task measures of performance. We decided to use performance ratings as dependent variables because the performance rating is the most common method of judging performance (Landy and Farr, 1980). Drawing from Varma et al. (1996), six task-related measures (e.g. quantity of work) of objective job performance were scaled to form a task rating score (alpha ¼ 0.88 US, 0.78 India), and ten trait items (e.g. sensitivity) were used to create a trait scale score (alpha ¼ 0.93 US, 0.88 India).

Analyses Following the procedure used by Tsui and Barry (1986), interpersonal affect was trichotomized into three levels: positive, neutral, and negative. A 2 (Performance level) £ 3 (Affect) between-subjects ANOVA was conducted three times on both samples, with trait ratings, task ratings, and overall performance as dependent variables. We calculated effect sizes using eta squared. Multivariate analyses were not used due to the strong, positive relationships between the dependent variables. Results Means, standard deviations, and correlations among the dependent and independent variables for the American sample are reported in Table 1. As can be seen, trait ratings and task ratings were strongly and significantly correlated (r ¼ 0.89; p , 0.01). The overall performance measure was also strongly correlated with task ratings (r ¼ 0.87; p , 0.01) and trait ratings (r ¼ 0.83, p , 0.01). Further, interpersonal affect was

Varma et al.: Interpersonal affect in performance appraisal

2035

Table 1 Descriptive statistics, correlations and scale reliabilities Variable

Mean

SD

(1)

(2)

(3)

(4)

3.28 3.63 3.52 3.58

0.77 0.83 0.80 1.06

(0.85) 0.69* 0.73* 0.66*

(0.88) 0.89* 0.87*

(0.93) 0.83*

–

3.51 3.56 3.62 3.60

0.69 0.69 0.67 0.63

(0.73) 0.69* 0.67* 0.63*

(0.78) 0.91* 0.90*

(0.88) 0.83*

–

United States 1 2 3 4

Affect Task ratings Trait ratings Overall performance

India 1 2 3 4

Affect Task ratings Trait ratings Overall performance

Notes *p , 0.01; Scale reliabilities on diagonal.

significantly correlated with task ratings (r ¼ 0.69; p , 0.01), trait ratings (r ¼ 0.73; p , 0.01), and overall performance (r ¼ 0.66; p , 0.01). Means, standard deviations, and correlations for the Indian sample are also reported in Table 1. Again, trait ratings and task ratings were strongly and significantly correlated (r ¼ 0.91; p , 0.01). The overall performance measure was strongly correlated with task ratings (r ¼ 0.90; p , 0.01) and trait ratings (r ¼ 0.83; p , 0.01). Much like the American sample, interpersonal affect was significantly correlated with task ratings (r ¼ 0.69; p , 0.01), trait ratings (r ¼ 0.67; p , 0.01), and overall performance (r ¼ 0.63; p , 0.01). Results from the univariate ANOVA tests are reported in Tables 2 and 3. Before testing our hypotheses, we examined the effect of interpersonal affect on performance ratings to determine if the liking manipulation was successful. For the US sample, interpersonal affect had a significant effect on task ratings (F (2, 77) ¼ 7.97, p , 0.01), trait ratings (F (2, 76) ¼ 21.07, p , 0.01), and overall performance (F (2, 161) ¼ 10.38, p , 0.01). Results from the Indian sample are similar. Interpersonal Table 2 Results from between-subjects ANOVAs – US Variable

Task rating M

Affect Negative Neutral Positive [Note: df ¼ 2]

*p , 0.01.

eta2

F 7.97*

M

0.042

3.53 3.65 3.88

Performance level Low 3.12 High 4.25 [Note: df ¼ 1] A £ PL [Note: df ¼ 2]

Trait rating

Overall perf. eta2

F 21.07*

.103

3.36 3.56 3.91 297.00*

0.448

0.026

eta2

F 10.38*

0.053

3.50 3.59 4.04 209.53*

0.363

3.15 4.08 4.87*

M

191.32

0.341

3.07 4.35 6.61*

0.035

7.06*

0.037

2036 The International Journal of Human Resource Management Table 3 Results from between-subjects ANOVAs – India Variable

Task rating M

Affect Negative Neutral Positive (Note: df ¼ 2)

2.95 3.65 4.25

Performance level Low High (Note: df ¼ 1)

3.01 4.12

A £ PL (Note: df ¼ 2)

F 51.31*

Trait rating eta2

M

0.299

Overall perf.

F

eta2

46.08*

0.277

3.04 3.69 4.26 117.90*

0.329

0.152

F

eta2

33.08*

0.213

77.91*

0.242

17.66*

0.126

2.89 3.75 4.32 98.95*

0.291

3.11 4.12 21.54*

M

2.99 4.21 25.41*

0.174

*p , 0.01.

affect had a significant effect on task ratings (F (2, 241) ¼ 51.31, p , 0.01), trait ratings (F (2, 241) ¼ 46.08, p , 0.01), and overall performance (F (2, 244) ¼ 33.08, p , .01). The effect sizes on each of the dependent variables varied from moderate to large for the American sample, and were generally large for the Indian sample. Interpersonal affect had the largest effect on trait ratings and the least powerful effect on the task rating variable for the American sample. Further, interpersonal affect had the largest effect on the task rating variable and the least powerful effect on the overall performance measure for the Indian sample. For each of the analyses, among both samples, mean ratings increased as the level of interpersonal affect increased from negative to positive. Therefore, the interpersonal affect manipulation was successful. Hypothesis 1 predicted that subordinate performance level would have a significant effect on supervisory performance ratings of American and Indian subordinates. For the US sample, performance level had a significant main effect on task ratings (F (1, 77) ¼ 297.00, p , 0.01), trait ratings (F (1, 76) ¼ 209.53, p , 0.01), and overall performance (F (1, 161) ¼ 191.32, p , 0.01). Performance level also had a significant main effect on task ratings (F (1, 241) ¼ 117.90, p , 0.01), trait ratings (F (1, 241) ¼ 98.95, p , 0.01), and overall performance (F (1, 244) ¼ 77.91, p , 0.01) for the Indian sample. The effect sizes for each of the dependent variables are also quite large among both samples. In all of the analyses, mean ratings were higher in the high performance category than in the low performance category. Hypothesis 1, accordingly, received strong support. Hypothesis 2 predicted that performance level and interpersonal affect would interact to impact performance ratings. There were significant interaction effects between performance level and interpersonal affect for each of the three dependent variables among both samples. Significant interaction effects were found for task ratings (F (2, 77) ¼ 4.87, p , 0.01), trait ratings (F (2, 76) ¼ 6.61, p , 0.01), and overall performance (F (2, 161) ¼ 7.06, p , 0.01) for the American sample. Analogously, significant interaction effects were found for task ratings (F (2, 241) ¼ 21.54, p , 0.01), trait ratings (F (2, 241) ¼ 25.41, p , 0.01), and overall performance (F (2, 244) ¼ 17.66, p , 0.01) for the Indian sample. As such, hypothesis 2 also received strong support.

Varma et al.: Interpersonal affect in performance appraisal

2037

Results from post-hoc (Tukey) tests from the US and Indian samples reveal that there were significant differences in performance ratings between each of the three interpersonal affect levels for all of the dependent variables. For each of the dependent variables, ratings were higher in the positive affect condition than in the neutral and negative affect conditions; ratings were higher in the neutral condition than in the negative affect condition; and ratings were lowest in the negative affect condition. The significant interaction effect between performance level and interpersonal affect, and the results from the Tukey tests demanded that we further examine the interaction between performance and interpersonal affect. Not only were we interested in examining how interpersonal affect interacts with performance to influence performance ratings among both samples, we were also interested in ascertaining what specific differences there may be between the US sample and the Indian sample in terms of this interaction. To examine this issue, we performed simple effects analyses (see Tables 4 and 5) by splitting each dataset between high and low performers, and then running one-way ANOVAs on both samples using interpersonal affect as an independent variable for each of the three dependent variables. Each overall ANOVA was significant among both high performers and low performers at the 0.01 level for both samples, except the overall performance measure for the Indian sample, which was significant at the 0.05 level. Hypothesis 3 predicted that performance ratings for high performers would be deflated when raters had negative interpersonal affect. We tested for mean differences in performance ratings among the three different affect levels by using Tukey’s Honestly Significantly Different (HSD) Test. If there is a significant difference in ratings between raters with neutral interpersonal affect, who are essentially indifferent about the ratee, and raters with negative interpersonal affect, then the negative interpersonal affect would adversely influence the performance rating. In this case, interpersonal affect could be said to operate as a bias in the appraisal process. However, as we can see in Tables 6 and 7, there were no significant differences in the performance ratings of high performers between raters with neutral and negative interpersonal affect for any of the dependent variables among either sample. We should note, however, that there were significant differences in trait ratings and overall performance ratings between raters with positive and negative interpersonal affect for the American sample. There were also significant differences in task ratings between raters

Table 4 Results of simple-effects analysis – US Task rating M Low performers Negative Neutral Positive (Note: df ¼ 2)

2.83 3.16 3.36

High performers Negative Neutral Positive (Note: df ¼ 2)

4.22 4.14 4.40

*p , 0.01.

Trait rating

F

eta2

10.86*

0.107

M

Overall perf.

F

eta2

20.12*

0.181

2.73 3.13 3.57 8.63*

0.085 3.98 3.99 4.26

M

F

eta2

13.37*

0.127

5.45*

0.047

2.61 3.00 3.58 13.92*

0.130 4.38 4.17 4.50

2038 The International Journal of Human Resource Management Table 5 Results of simple-effects analysis – India Task rating M

F

Trait rating eta2

M

F

Overall perf. eta2

M

Low performers 49.87** 0.454 52.82** 0.466 Negative 2.56 2.64 2.44 Neutral 3.22 3.34 3.31 Positive 3.11 4.18 4.20 (Note: df ¼ 2) High performers Negative 3.97 Neutral 4.00 Positive 4.30 (Note: df ¼ 2)

8.64** 0.125

7.37** 0.109 4.06 3.97 4.30

F

eta2

37.86** 0.383

3.52*

0.055

4.08 4.10 4.37

*p , 0.05. **p , 0.01.

with positive and negative interpersonal affect for the Indian sample. One might interpret mean differences between raters with positive and negative interpersonal affect in any performance condition to signal bias, but the referent must be the neutral condition, and the comparison must be the negative condition. For ratings to be deflated there must be a difference between raters with negative and neutral interpersonal affect as the neutral condition is the unbiased, objective condition. As such, hypothesis 3 was not supported, indicating that interpersonal affect is not operating as a bias in the high performing condition. That is, ratings of high performers are not deflated by raters that have negative interpersonal affect for them. This finding does contradict results of Robbins and DeNisi (1994) somewhat. Robbins and DeNisi found a significant difference in performance ratings between raters with negative and neutral interpersonal affect, but their finding was not constricted to high performers. Hypothesis 4 predicted significant differences in ratings of low performers between raters with neutral interpersonal affect and raters with positive interpersonal affect. If ratings of low performers differ between raters with neutral and positive interpersonal affect, then interpersonal affect would be biasing the performance rating such that the rating would be inflated. Using Tukey’s HSD, we found no significant differences between neutral and positive rating conditions for task ratings or overall performance in the American sample (see Table 6). There was, however, a significant difference for trait ratings. A potential explanation of this result is that trait ratings are more subject to bias. As suggested by Varma et al. (1996), trait ratings involve more ambiguous, subjective evaluation factors than do outcome-related ratings, and may, therefore, be more easily influenced by interpersonal affect. As above, there were significant differences between raters with negative and positive interpersonal affect for each of the dependent variables. Unlike the American sample, there were significant differences between neutral and positive interpersonal affect conditions for each of the dependent variables in the Indian sample. This is a clear and important difference between the two samples. Again, there were significant differences between positive and negative interpersonal affect conditions for each of the dependent variables.

Negative , Neutral** NEUTRAL < POSITIVE* Positive . Negative** NEGATIVE < NEUTRAL Neutral , Positive** Positive . Negative

Negative , Neutral** NEUTRAL < POSITIVE Positive . Negative** NEGATIVE > NEUTRAL Neutral , Positive** Positive . Negative**

Low performers

High performers

*p , 0.05. **p , 0.01. Note CAPS indicate comparison of interest. The only significant relationship involves trait ratings of low performers.

Trait ratings

Task ratings

Table 6 Tukey’s HSD between affect levels (high and low performers) – US

NEGATIVE > NEUTRAL Neutral , Positive** Positive . Negative**

Negative , Neutral** NEUTRAL < POSITIVE Positive . Negative**

Overall performance

Varma et al.: Interpersonal affect in performance appraisal 2039

Negative , Neutral** NEUTRAL < POSITIVE** Positive . Negative** NEGATIVE < NEUTRAL Neutral , Positive** Positive . Negative

Negative , Neutral** NEUTRAL < POSITIVE** Positive . Negative** NEGATIVE < NEUTRAL Neutral , Positive** Positive . Negative**

Low performers

High performers

NEGATIVE < NEUTRAL Neutral , Positive Positive . Negative

Negative , Neutral** NEUTRAL < POSITIVE** Positive . Negative**

Overall performance

*p , 0.05. **p , 0.01. Note CAPS indicate comparison of interest. Unlike the US sample, there are significant differences in the performance ratings of low performers between raters with neutral and positive affect.

Trait ratings

Task ratings

Table 7 Tukey’s HSD between affect levels (high and low performers) – India

2040 The International Journal of Human Resource Management

Varma et al.: Interpersonal affect in performance appraisal

2041

As such, hypothesis 4 received mixed support. Specifically, it appears that interpersonal affect is not operating to inflate ratings of low performers in the American sample. However, it is clear that interpersonal affect is operating as a bias in the Indian sample; ratings of low performers are consistently inflated by raters with positive interpersonal affect towards them across various rating formats.

Discussion The present study was designed with two specific purposes in mind: (1) to examine the role of interpersonal affect in a field setting, and (2) to expand comparative cross-cultural studies of HR issues to include developing countries, by comparing US and India. While several scholars have argued that interpersonal affect operates as a bias in the appraisal, others (e.g. Lefkowitz, 2000; Varma et al., 1996) have suggested that it may not be a bias after all, especially if it is a result of previous performance. However, Lefkowitz (2000) also strongly urges readers away from the assumption that raters simply like good performers. He argues that the relationship between interpersonal affect and performance is rather complicated. Indeed, the results of our study suggest that raters are able to separate affective and performance-related information in the field when assigning performance ratings, but that this ability to ‘separate’ is moderated by cultural norms. As we note above, ratings of low performers were consistently inflated by raters with positive interpersonal affect towards them, in the Indian sample, but not in the US sample. In other words, interpersonal affect did operate as a ‘bias’ in the Indian context, perhaps due to the culturally influenced dependence of subordinates on the supervisor, and the precedence given to personalized relationships over actual job performance. Table 4 summarizes the interactive relationship between interpersonal affect and performance. A couple of findings deserve special mention. Rather curiously, performance ratings were actually slightly higher in the negative interpersonal affect condition than the neutral condition for two of the dependent variables: task ratings and overall performance. There are a variety of reasons why this relationship may exist. For instance, raters with negative interpersonal affect towards a high performer may actually give a slightly over-generous rating because the supervisor may not want to seem biased in his/her assignment of performance ratings. That is, if the relationship between the rater and ratee is unfriendly, but the ratee is conspicuously a high performer, the rater may want to protect himself or herself from accusations of giving unfair performance evaluations, perhaps because of organizational or legal repercussions. In addition, in the low performing condition, there were significant mean differences in ratings between raters with negative and neutral interpersonal affect for each of the dependent variables; there were also significant differences between raters with negative and high interpersonal affect for each of the dependent variables. Again, we argue that while there might be significant differences between raters with positive and negative interpersonal affect, for interpersonal affect to cause ratings of low performers to be inflated, the key comparison involves the neutral and positive conditions. Furthermore, while significant, these differences are actually quite small. While research has illustrated interpersonal affect’s primacy (Zajonc, 1980), more research should be done to investigate exactly how liking develops between a rater and a ratee. Some researchers have found that interpersonal affect may actually facilitate more accurate performance ratings. For instance, Tsui and Barry (1986) found that raters with neutral interpersonal affect exhibited much less range restriction than raters with either

2042 The International Journal of Human Resource Management high or low interpersonal affect. This disinclination to distinguish among performers is certainly undesirable for performance evaluation purposes. Implications for practice and research The findings of our study should prove of interest to both HR researchers and practitioners, in India and the US. While, on the one hand, we have added to the literature on ‘interpersonal affect as a bias’, we have also contributed to the growing literature on cross-national and comparative HR studies. As can be seen, the same performance appraisal system applied to these two nations would produce different results, and be unable to capture ‘true performance.’ Clearly, this could have a rather interesting and unique impact on individuals from one of these countries when working in the other country. An expatriate from India, for example, might make efforts to develop a relationship with the supervisor with the expectation that the relationship would influence his/her performance rating. On the other hand, a US expatriate working in India may be surprised to note the importance of developing a ‘like–dislike’ relationship with his/her supervisor. In both cases, the individuals are likely to be faced with situations rather different from their previous experience(s). Clearly, this could lead to performance anxiety and context-ambiguity for both individuals, potentially impacting their performance. It is clear that organizations that operate in both these nations need to understand better local practices and procedures, and revisit their HR systems to incorporate local cultural norms. Furthermore, supervisor training on performance appraisal techniques and procedures should be modified to include discussions of the differences in the workplaces of the two countries, and the potential role that these differences could play in performance evaluations. From an academic perspective, our research provides new information on a critical question in the field of performance appraisal research – does interpersonal affect bias performance appraisals? While previous research has addressed this question, and reported mixed results, the potential effect of cultural norms has not been investigated prior to our study. We hope future investigations of the role of interpersonal affect in performance appraisal will study the moderator effect of cultural norms. Limitation and suggestions for future research While we believe that the present study makes a significant contribution to the literature on interpersonal affect, as well as cross-national comparative studies, it does have certain limitations. First, all our data were collected from supervisors. As such, our sample is potentially subject to mono-method bias. Therefore, the relationships between the variables of interest may be inflated as potentially indicated by the strong intercorrelations among the dependent variables. We hope that future investigators will use objective performance measures when studying interpersonal affect in field situations whenever possible. Second, our data in both samples were collected from manufacturing organizations. As such, we hope that future investigations of interpersonal affect will include other types of organizations. References Arvey, R.D. and Murphy, K.R. (1998) ‘Performance Evaluations in Work Settings’, Annual Review of Psychology, 49: 141 – 68. Bauer, T.N. and Green, S.G. (1996) ‘Development of Leader-Member Exchange: A Longitudinal Test’, Academy of Management Journal, 39(6): 1538 –67.

Varma et al.: Interpersonal affect in performance appraisal

2043

Brewster, C., Tregaskis, O., Hegewisch, A. and Mayne, L. (1996) ‘Comparative Research in Human Resource Management: A Review and an Example’, The International Journal of Human Resource Management, 7(3): 585 – 604. Budhwar, P. (2001) ‘Doing Business In India’, Thunderbird International Business Review, 43(4): 549– 68. Budhwar, P. and Debrah, Y. (2001) ‘Rethinking Comparative and Cross National Human Resource Management Research’, The International Journal of Human Resource Management, 12(3): 497– 515. Budhwar, P. and Debrah, Y. (2004) ‘Introduction: HRM in Developing Countries’. In Budhwar, P. and Debrah, Y. (eds) HRM in Developing Countries. London: Routledge, pp. 1 –15. Budhwar, P.S. and Khatri, N. (2001) ‘A Comparative Study of HR Practices in Britain and India’, International Journal of Human Resource Management, 12(5): 800–26. Cardy, R.L. (1982) The Effect of Affect on Performance Appraisal. Unpublished Doctoral Dissertation. Virginia Polytechnic University. Cardy, R.L. and Dobbins, G.H. (1986) ‘Affect and Appraisal Accuracy: Liking as an Integral Dimension in Evaluating Performance’, Journal of Applied Psychology, 71(4): 672– 8. Carey, J.A. (1997) Factors Affecting the Influence of Liking on Observation and Evaluation Accuracy in Performance Appraisal. Unpublished Doctoral Dissertation, Arizona State University. Datt, R. and Sundharam, K.P.H. (1999) Indian Economy. New Delhi: S. Chand. DeNisi, A.S. and Stevens, G.E. (1981) ‘Profiles of Performance, Performance Evaluations, and Personnel Decisions’, Academy of Management Journal, (24): 596–602. Dipboye, R.L. (1985) ‘Some Neglected Variables in Research on Discrimination in Appraisals’, Academy of Management Executive, 10(1): 116 –27. Dobbins, G.H. (1982) ‘The effect of leader performance and leader likeableness upon ratings of leader behaviors’. Unpublished Master’s Thesis, Virginia Polytechnic University. Farris, G.C. and Lim, F.G. Jr (1969) ‘Effects of Performance on Leadership, Influence, Satisfaction, and Subsequent Performance’, Journal of Applied Psychology, 53: 490– 7. Feldman, J.M. (1981) ‘Beyond Attribution Theory: Cognitive Processes in Performance Appraisal’, Journal of Applied Psychology, 66(2): 127 – 48. Ferris, G.R., Judge, T.A., Rowland, K.M. and Fitzgibbons, K.M. (1994) ‘Subordinate Influence and the Performance Evaluation Process: Test of a Model’, Organizational Behavior and Human Decision Processes, 58: 101 – 35. Gopalan, S. and Stahl, A. (1998) ‘Application of American Management Theories and Practices to the Indian Business Environment: Understanding the Impact of National Culture’, American Business Review, 16(2): 30– 41. Harbison, F. and Myers, C.A. (1964) Education, Manpower and Economic Growth. New York: McGraw-Hill. Hofstede, G. (1980) Culture’s Consequences. Beverly Hills, CA: Sage. Hofstede, G. (1993) ‘Cultural Constraints in Management Theories’, Academy of Management Executive, 7(1): 81 –94. Ilgen, D.R. and Feldman, J.M. (1983) ‘Performance Appraisal: A Process Focus’, Research in Organization Behavior, 5: 141 – 97. Jain, H.C. and Venkata Ratnam, C.S. (1994) ‘Affirmative Action in Employment for the Scheduled Castes and the Scheduled Tribes in India’, International Journal of Manpower, 15(7): 6 – 25. Kanungo, R. and Mendonca, M. (1994) ‘Culture and Performance Improvement’, Productivity, 35(4): 447– 53. Kapur, D. and Ramamurti, R. (2001) ‘India’s Emerging Competitive Advantage in Services’, Academy of Management Executive, 15(2): 20– 33. Katz, I. and Glass, D.C. (1979) ‘An Ambivalence-Amplification Theory of Behavior Toward the Stigmatized’. In Austin, G. and Worschel, S. (eds) Social Psychology of Intergroup Relations. Monterey, CA: Brooks-Cole Publishing, pp. 55–70. Landy, F.J. and Farr, J.L. (1980) ‘Performance Rating’, Psychological Bulletin, 87(1): 72 –107.

2044 The International Journal of Human Resource Management Latham, G.P. and Wexley, K.N. (1981) Improving Performance Through Effective Performance Appraisal. Reading, MA: Addison-Wesley. Lefkowitz, J. (2000) ‘The Role of Interpersonal Affective Regard in Supervisory Performance Ratings: A Literature Review and Proposed Causal Model’, Journal of Occupational and Organizational Psychology, 73(1): 67– 86. Lefkowitz, J. and Battista, M. (1995) ‘Potential Sources of Bias in Supervisor Ratings Used for Test Validation’, Journal of Business and Psychology, 11(3): 389– 414. Napier, N.K. and Vu, V.T. (1998) ‘International Human Resource Management in Developing and Transitional Economy Countries: A Breed Apart?’, Human Resource Management Review, 8(1): 39– 77. Olian, J.D., Carroll, S.J. and Giannantonio, C.M. (1993) ‘Mentor Reactions to Prote´ge´s: An Experiment with Managers’, Journal of Vocational Behavior, 43: 266– 78. Robbins, T.L. and DeNisi, A.S. (1994) ‘A Closer Look at Interpersonal affect as a Distinct Influence on Cognitive Processing in Performance Evaluations’, Journal of Applied Psychology, 79(3): 341–53. Robbins, T.L. and DeNisi, A.S. (1998) ‘Interpersonal Affect: Identifying Process and Rating Distortions in Performance Appraisal’, Journal of Business and Psychology, 12(3): 313– 25. Schuler, R.S., Budhwar, P. and Florkowski, G.W. (2002) ‘International Human Resource Management: Review and Critique’, International Journal of Management Review, 4(1): 41– 70. Sinha, J.B.P. (1990) Work Culture in Indian Context. New Delhi: Sage. Srull, T.K. and Wyer, R.S. (1980) ‘Category Accessibility and Social Perception: Some Implications for the Study of Person Memory and Interpersonal Judgments’, Journal of Personality and Social Psychology, 38(6): 841 – 56. Tsui, A.S. and Barry, B. (1986) ‘Interpersonal Affect and Rating Errors’, Academy of Management Journal, 29(3): 586 – 99. Varma, A., DeNisi, A.S. and Peters, L.H. (1996) ‘Interpersonal Affect and Performance Appraisal: A Field Study’, Personnel Psychology, 49: 341 – 59. Wayne, S.J. and Ferris, G.R. (1990) ‘Influence Tactics, Affect, and Exchange Quality in Supervisor – Subordinate Interactions: A Laboratory Experiment and Field Study’, Journal of Applied Psychology, 75(5): 487 – 99. Zajonc, R.J. (1980) ‘Feeling and Thinking: Preferences Need No Inferences’, American Psychologist, 35(2): 151 – 75.