Best Journals Versus Best Fit Journals

Best Journals versus Best Fit Journals: A Strategic Orientation to Research Quadrants Dr Stephen Dann, ANU Abstract This paper is proposing a heresy of sorts – perhaps marketing should practice what it teaches, and develop journal ranks that meet the needs of the organisation, its stakeholders, and which are based on segmenting the journal output market. This process may even create, communicate and deliver value for the institution and our government and industry stakeholders. What follows is a simple model of heresy that permits the research goals of the university, the strengths of the researchers, and the need to enhance marketing thought within the marketing academy to determine the relative match between research output and ideal publication medium. The author full expects this notion of an alternative to the JCR/JM/JMR Billboard Top 40 will go down like the Titanic, but sometimes, it's worth shouting "Iceberg ahead" in hopes of changing the course (and destiny) of the ship. Introduction O'Connor and Moodie (2006) remind us that once again, the higher education sector in Australia is under criticism from its political masters - this time over the lack of diversification between the universities. Under the current minister's vision for higher education, universities are meant to diversify their portfolios, concentrate on their strengths, and cede ground to rival institutions in the name of sector wide reform. In the pursuit of this goal, most university systems, schools and departments are rushing towards the paradoxical goal of diversifying to the same tune. Across the Australian marketing academy, schools, departments and professors all acknowledge the need for segmentation, positioning, diversification and the pursuit of publication in the same top ten journals. If irony still had meaning in the post-modern world of academia, Alanis Morrissette would sing the praises of this strategy. The problem, in part, is the pursuit of league table style metrics that provide evidence of "best" rather than "best fit".

Diversification requires universities to focus on their strengths, but if each university uses the same measures to determine those "strengths", then the rigid frameworks of "best journals" and "best conferences" will see best-fit modified to match the "best journals". In other words, diversification will result in the uniform pursuit of the same goals, with everyone citing the same strengths to see themselves competitive on the same scorecard. Rewarding quality by assessing it against a rigid criteria of "best" will simply result in more of the same outcomes - rejection slips from the same "top tier" journals. The Standardised Journal Ranking Schema Journal ranking schema are old news in the business academic sector, with the first studies of journal quality appearing in 1974, and continuing unabated through to the current paper (Koojaroenprasit et al, 1998, Polonsky and Whitelaw, 2006). To quote Polonsky and Whitelaw (2005): …with Hawes and Keillor (2002) identifying that between 1980 and 2001 there were at least 16 journal ranking studies in marketing published in academic journals and conferences. Since 2002 there have been additional ranking studies, including; Baumgartner and Pieters (2003), Theoharakis and Hirst (2002), Mort et al. (2004) and Polonsky and Whitelaw (2004). It appears that ranking journals may in fact be a predisposition within business faculties in general (Armstrong and Sperry 1994, Van Fleet et al. 2000). Twenty two years, twenty journal ranking systems exist in the published field, with a few more proprietary research ranks existing within different universities around the globe. As Polonsky and Whitelaw (2005) note, once the reader looks outside of the top three or five journals, the inconsistency of these lists stand as a tribute to "one size fits nobody" production orientations. Even within the quality index industry there is a mute acceptance of one-dimensional measures of “best”, based on the respondent's perception of "quality" (Baumgarter and Pieters, 2003). Would the academic who endorses this list for funding back the same single item measure from a student project? For most aspect of the marketing disciplines, the use of a onedimensional scale of "best" is inconsistent with the fundamental principles of the discipline and the practice of the industry (Corfman, 1991). Does the systematic avoidance of accepted methodology maintain the relevance of the discipline? One would think not, unless the methodological two-step assists in the creation of "quality index" for

rating publication outputs. For marketing, an important question needs to be considered – does a single item list of “best” journals match the philosophy of the discipline? Should the notion of market orientation, market segmentation, positioning and an emphasis on not being “everything to everyone” be something we teach but not practice? The author contends that this question of disciplinary relevance is often overlooked in order to mask the fundamental clash between what marketing preaches and what marketing does with league tables of journal rankings. Model for Assessing Research Output / Research Agenda fit Research output is assumed to have two core components - the business strategy component which represents the organisational research goals of the University, and the goal of developing knowledge within a specific discipline. Hawes and Keillor (2002 in Polonsky and Whitelaw (2005) recommend that the using the institutional mission statements in the ranking of marketing journals. This view is also supported by the AACSB (2006) who specify that intellectual contributions, measured in part by peer reviewed journal articles, should be consistent with the school’s mission and strategic management processes. Therefore, pursuit of disciplinary research goals or organisational research goals gives rise to the 2x2 matrix of discipline/university agenda alignment of the research Ffigure 1). These four areas are defined in the practical context for the development of a research output ranking structure as : Figure 1. Discipline and University Research Goals • Discipline Research Goals

Academy Driven Research (Tier 1)

Marketing Unit Research Agenda (Tier 3)

Non aligned Research (Tier 4

University Research Objectives (Tier 2) University Research Goals

Quadrant 1: Academy driven research which encourages the development of the marketing discipline through the engagement in relevant marketing academies

•

Quadrant 2: the strategic research goals of the host institution

•

Quadrant 3: the pursuit of a coherent research

agenda for the marketing department school or group •

Quadrant 4: Independent research output which is not aligned to the above objectives

Assumptions Two key assumptions underpin the entire process of journal research priority setting. First, research will be treated as a strategic undertaking, and organisational research outputs can be treated as products to be offered to a market of publication outlets. Second, the paper assumes that journals can be treated as consumers of research output, with each journal having research preferences, varying levels of demand for research products. For example, a university with a research centre in retailing is likely to generate research products of interest for retailing journals such as Journal of Retailing and Journal of Retailing and Consumer Services. Similarly, the Journal of Consumer Behaviour is not likely to be looking for a research output product on business to business research. Treating journals as consumers of research also allows for researchers to target their research output products where they have a strong match between market need (e.g. journal coverage) and research strength (topic of the paper). Sample The preliminary sample of marketing journals was selected from the University of Queensland Business School (UQBS) List 2003, and confirmed by the University of Auckland List (2003). Although up to 120 journals have been identified as appropriate outputs for marketing research, the preliminary Quadrant Ranking schema was restricted to the 57 journals previously identified in these two papersTwo additional lists of research quality ranks were used to assist the quadrant development, Polonsky and Whitelaw (2006) paper (P&W List) which ranked 65 different journals according to five different criteria, and the Mort et al (2003) paper which ranked 72 different journals. These two lists were selected to reflect US (Polonsky and Whitelaw, 2006) and Australian (Mort et al 2003) perceptions of journal quality. Quadrant 1: Developing the discipline of marketing To ask a marketing academic what they do for a living is to invite the response “I’m a marketer”. The membership of the academy of marketing is a universal and transferable trait, unbound to specific nations or employers. Consequently, and in recognition of the global nature of the discipline, the first quadrant priority for marketing journals is the

pursuit of the enhancement and development of the academy of marketing. Quadrant 1 consists of a list of the journals published by the respective academies of marketing across the major geographic areas, e.g. Australasia, Europe and America. Members of an academy should be rewarded for supporting the development and enhancement of their academy specific journals as a means of recognising the contribution to and support of the development of the discipline. Publication in these journals is recognised as contributing to the broader development of the discipline by disseminating research to the academy body, and into the community through the outreach of the academy. Researchers should be encouraged to contribute to the development of their local academy as a priority, and should also be encouraged to contribute to other academy journals where appropriate. (Appendix 1) Quadrant 2: Addressing Institutional Research Goals The second quadrant of journals is based on providing a framework to encourage academics to pursue the research interest of the university in conjunction with the development of their relevant academies. For example, an institution may elect to focus on developing Australian research, and advancing knowledge of Australian research. Consequently, the Quadrant 2 journals would be selected on the basis of their track record in the publication of Australian research and Australian researchers. In contrast, a university with a priority in econometric theory and econometric modelling would select journals which represented the best track record of publishing econometric papers. Quadrant 2 level journals are selected by matching the research strength of the organisation against the market demand for the research output product. Quadrant 3: School, Department and College level research objectives Quadrant 3 journals are the disciplinary and sub-disciplinary specific journals which are representative of the peak quality outlets for a specific area or specific research concentration in marketing. Inclusion in this list is based on the match between the journal’s publication objectives and the marketing group’s research agenda. Determining a Research Agenda

Development of a research agenda of is based on the staff member’s self reported identification of their areas of research specialisations conducted using a list of track streams from the Australia and New Zealand Marketing Academy Conference. Each the respondent identifies the track streams closest to their PhD topic, primary research area and second research areas. These research clusters are matched against the “areas of research interest” identified in Polonsky and Whitelaw’s (2006) journal ranking paper which incorporated an examination of the research interests of the US marketing community. Table 2 features a sample research agenda list Table 1. Developing a Research Agenda Sum

Polonsky & Whitelaw (2006)

ANZMAC Track Areas Corporate Responsibility, Social Issues and Social Responsibility Marketing and

12

Marketing & Society

Society Non profit marketing Political Marketing Social Marketing Entrepreneurship, Firms in Networks, Marketing Strategy, New Product

11

Marketing Strategy

Development, Supply Chain Management, Financial & Value-Based Marketing

10

Marketing Communications

Advertising / Marketing Communication / IMC, Branding

Journals are selected for the research agenda quadrant based on the number of articles containing the relevant research keyword, and the stated research goal of the journal. For example, the Journal of Public Policy and Management has a strong fit with the goals, whilst the Journal of Retailing has a weak fit with the research agenda. Consequently, although the Journal of Retailing has a high individual rank score in the Polonsky and Whitelaw (2006) and Mort et al (2003) league tables, it would not be an appropriate priority target for this research agenda. Quadrant 4: Independent research output Good scholars can be expected to publish in these journals, and journals of this level are equal in quality to those in Quadrant 3. These are quality journals, and cover a broad range of fields, but they are not aligned to the research agendas demonstrated above. Inclusion in quadrant 4 is not a criticism of the quality of the journal, and as such, should not be seen as a threat to the legitimacy of external rankings of the journal. As such, it is a recognition that it is possible for a high quality research outlet to not be relevant to the research agenda of an institution. In a commercial marketing context, objectively

lucrative markets are often ignored by organisations who lack organisational strengths to meet the needs of these markets. Similarly, if the research institution does not have a strength in the area of specialisation of a high quality journal, strategic marketing management theory would suggest not targeting this research output market. The existence of the fourth quadrant as a catch-all for peer reviewed journals outside of the research agendas of the organisation still allows for academic freedom to pursue research in fields of personal interest. Limits of the Method All existing measures of journal quality used in this paper, either as a multiple quadrant structure or a single league table were found to contain reference to the Journal of Market-Focused Management which ceased publication in January 2003. In several instances, this defunct journal was deemed to be a higher quality research publication outlet than active journals. Given the Journal was discontinued in Wednesday, January 01, 2003, it is understandable that the Mort et al (2003) paper, based on research presented at the December 2002 ANZMAC conference would not have had the opportunity to correct the ranking for their journals. However, subsequent quality lists, such as University of Auckland (September, 2003) and Polonsky and Whitelaw (2006) have included the defunct journal in the list. It would be an article of a certain quality that could be published in a journal that ceased trading nearly four years prior to this paper’s publication. A second limitation of the process arises in the process used for determining the research agenda journal list. There is a prospect of false positive/false negative hits from the keyword assessment of each journal. In testing the method, the keyword “Australia” was used as a proxy measure for the alignment of the journal with a pro-Australian research agenda. Steps were taken to ensure that where possible, obvious false positives were removed, for example, where the full bibliographic details of a reference included the keyword but the article contained no reference to Australian content or was not by an Australian author. Although this process was followed with reasonable care, time constraints on the initial study did not permit the grading of each positive keyword match article on a scale of relevance to the research agenda. Further research and development of a agenda relevance measure could be used to enhance this

framework, and sharing of research agenda grading of journals would be able to reduce bias inherent in the reliance on single source interpretation. Conclusions In what will be a disappointment for many readers, this paper does not feature a full ranking index. Simply put, for the paper that argues for multidimensional priority indexes based on organisational objectives and research priorities, presenting a onedimensional index would be to miss the point of the paper entirely. Only the primary quadrant of academy journals should be consistent between organisations that ostensibly pursue differentiated research agendas. It's time for Australian universities to prioritise research outputs based on their own goals and agendas, and for business schools to walk the talk of segmentation, positioning and targeting. Practicing what we preach is a quality outcome for research, and one that is far more likely to make an impact than blind adherence to antiquated league table score cards.

References References AACSB. (2006). Eligibility procedures and accreditation standards for business accreditation, AACSB International. http://www.aacsb.edu/accreditation/business/STANDARDS.pdf Armstrong, J.S. (1994) “Business School Prestige – Research versus Teaching,” Interfaces 24(2): 13-42. Baumgartner, H, and Pieters, R.. (2003). The structural influence of marketing journals: A citation analysis of the discipline and it subareas over time. Journal of Marketing 67 (1): 123-139. Corfman, K (1991) “Perceptions of Relative Influence: Formation and Measurement,” Journal of Marketing Research, 28(2): 125-136. Easton, G. and Easton, D.M.. (2003). Marketing journals and the research assessment exercise, Journal of Marketing Management 19 (1-2): 5-24. Hawes, J. M., and Keillor, B. (2002). Assessing marketing journals: a mission-based approach. Journal of the Academy of Business Education 3 (2): 70-86. Journal of Market-Focused Management, http://www.springerlink.com/link.asp?id=100292 Koojaroenprasit, N, Weinstein, A, Johnson, W. C. and Remington, D. O. (1998) “Marketing Journal Rankings Revisited: Research Findings and Academic Implications”, Marketing Education Review, 8(1), 95-102.

Mort, G, McColl-Kennedy, J, Kiel, G. 2002. Senior Faculty Perceptions of the Quality of Marketing Journals: An Australian and New Zealand Perspective. ANZMAC 2002 Conference Proceedings, 2431-2438. Mort, G. S., McColl-Kennedy, J. R., Kiel, G., and Soutar, G. N. (2004). Australian and New Zealand senior academics’ perceptions of marketing journals Australasian Marketing Journal 12 (2): 51-61. O'Connor, I and Moodie, G. (2006) "Unis can't diversify to a formula" The Australian Higher Education Supplement, June 21, 2006 [Accessed online http://www.theaustralian.news.com.au/story/0,20867,19534243-12332,00.html June 30, 2006] Polonsky, M J and Whitelaw, P. (2004). Exploring multi-dimensional perceptual ranking of marketing journals by North American academics. 2004 Summer American Marketing Association Educators Conference Summer: 13-20. Polonsky, M, and Whitelaw, P. What Are We Measuring When We Evaluate Journals?, Journal of Marketing Education.2005; 27: 189-201 Polonsky, M, and Whitelaw, P. (2006) “A Multi-Dimensional Examination of Marketing Journal Rankings by North American Academics” Marketing Education Review, forthcoming Theoharakis, V, and Hirst A. (2002). Perceptual differences of marketing journals: a worldwide perspective. Marketing Letters 13 (4): 398-402. University Of Auckland (2003) Department of Marketing, Guideline To Assessing Journals And Books, University Of Auckland – internal document University of Queensland Business School (UQBS) List 2003– internal document

Van Fleet, David D., McWilliams, A., and Siegel, D. S.. (2000). A theoretical and empirical analysis of journal rankings: the case of formal lists. Journal of Management 26 (5): 839-861.

Appendix 1 Table 1. Quadrant 1 Journals UQ

UA

P&W

Mort

Quadrant

Quadrant

List

List

Academy of Marketing Science Review

3

n/a

20

n/a

Australasian Marketing Journal

3

2

57

34

International Journal of Research in Marketing

2

1

12

8

Journal title

Journal of International Marketing

2

2

22

23

Journal of Marketing

1

1

1

2

Journal of Marketing Management

3

2

35

16

Journal of Marketing Research

1

1

2

3

Journal of Public Policy & Marketing

2

2

7

13

Journal of the Academy of Marketing Science

1

1

6

4