The Internal Audit As Tool To Enhance The Quality Of Qualitative Research

THE INTERNAL AUDIT AS TOOL TO ENHANCE THE QUALITY OF QUALITATIVE RESEARCH W. Schurink Sanlam Centre for Public Management and Governance University of Johannesburg

INTRODUCTION n general, researchers will agree that social science research is a systematic and organised process whereby enough knowledge (evidence) is gained to provide an accurate or truthful representation of a phenomenon under study. However, agreeing on this does not mean that they will agree on theoretical or methodological issues. In fact, there is a vast difference of opinion between qualitative and quantitative researchers as to how the social world should be studied. (http://bgpinqmr.group.shef.ac.uk/workshop/ Facilitators_Guide_2.pdf). The root of the difference lies in the worldview or ontology of the researcher. The one will believe that human behaviour can be explained from the outside (epic) by means of objectivistic observation and the use of general scientific laws (erklaren). The other will assume that human behaviour can only be understood from an insider’s point of view (emic) by gaining insight into the meaning (verstehen) that the participant gives to his/her life world. Qualitative and quantitative researchers therefore use different epistemological assumptions stemming from different theoretical contexts to understand human behaviour. In the context of this article it is important to comprehend that qualitative research aims at understanding actors’ subjective meanings and interpretations to explain their behaviour (Schwandt 2007), and that by accepting subjectivity as a legitimate domain for social scientific research, a break is established between the natural and social sciences. (http:// bgpinqmr.group.shef.ac.uk/workshop/ Facilitators_Guide_2.pdf). This is also true for the criteria by which qualitative research is measured. In other words, a qualitative study resting on the premise of verstehen cannot be judged as truthful or not by using the criteria meant to measure a study based on the premise of erklaren. Therefore, when the quality of a piece of research is validated, the ontological and epistemological position of the researcher should be the frame of the “magnifying glass” used to establish the truthfulness of a study.

788

Journal of Public Administration • Vol 44 no 4.2 • November 2009

DESIGNING TRUSTWORTHY RESEARCH he most important criterion that qualitative research should meet is that of credibility. An important decision one needs to make when designing a qualitative research study is to start thinking on how, and according to what principles and logic your claims and analysis will be formulated and substantiated. Differently put, you should consider how you are going to ensure that you do quality research. It is crucial that measures of quality are considered during the research design phase, since they need to guide you in selecting appropriate methods, sources and combinations of these as well as in other decisions that need to be taken. Since these measures form part of the decision making process they cannot be reserved for a later phase of the research process. Thus, it is of crucial importance that these issues are addressed during the research design phase. “If they are not, the researcher risks assembling an untidy bag of methods with little logic, and with little hope of sensibly integrating the products into a coherent analysis” (Mason, 1996:26). When conducting qualitative research, researchers face many ethical dilemmas and must constantly decide on the proper conduct. Of course, ethical issues are the concerns and dilemmas that arise over the proper way to execute research, more specifically not to create harmful conditions for the participants in the research process. According to Neuman (1997) these issues involve trade-offs between competing values and are typically situational, that is they depend on the researcher, topic and research participants. The following account by Jones (1996:33) illuminates these conflicts quite well: “Although researchers may agree that knowledge and understanding are ideals worthy of pursuit, there are other ideals that they may, at times, value even more highly. When such a conflict of ideals occurs, the issue of whether or not one should conduct the research in question may be said to pose an ethical dilemma. Quite literally, the researcher must ask him or her, ‘Can this research be ethically justified’? Ethically responsible research depends on the integrity of the individual researcher and his or her values. Neumann (1997:443) puts this as follows: “Ethics begins and ends with you, the researcher. A researcher’s personal moral code is the strongest defense against unethical behavior. Before, during, and after conducting a study, a researcher has opportunities to, and should, reflect on research actions and consult his or her conscience… Ethical behavior arises from sensitivity to ethical concerns that researchers internalise during their professional training, from a professional role, and from personal contact with other researchers. Moreover, the norms of the scientific community reinforce ethical behavior with an emphasis on honesty and openness. Researchers who are orientated toward their professional role, who are committed to the scientific ethos, and who interact regularly with other researchers are likely to act ethically”. In the light of the preceding remarks, Mason (1996) is correct when she points out that qualitative researchers should be as concerned to construct an ethical research design as they are to construct an intellectually coherent and compelling one. “This means attempting not only to carry out our data generation and analysis in an ethical manner, but also to begin by framing ethical research questions. Of course this is easier said than done, because however the research questions are framed, any project is likely to involve a range of interests, some of W. Schurink

789

Article

which may be competing. Therefore the idea that there is one ethical or moral route which is equally fair to all concerned may sound good in theory, but be elusive in practice”. Since research ethics is a complex issue, and it is very unlikely that there will ever be one clear solution, it is particularly appropriate to follow a practical approach in which you ask questions and push yourself hard to answer them. You need to be honest about the purpose of your research. Your study is likely to include not only the advancement of knowledge or understanding of some aspect of the social world, but also factors involving personal gain, such as the achievement of a personal qualification, a promotion, some standing in a discipline (amongst colleagues, friends, rivals, relatives, etc.), and/or some research funding. “It is part of the politics of research that you should engage with this wider context in which your research is being done. Your research may have explicitly moral or political purposes. You may wish to advance the interest of a particular group through it, or to gain acceptance for some particular form of social organization, or to expose some form of immoral organization or activity, and so on. This does not necessarily make the ethics of your research more straightforward however, not least because ‘the interests of a particular group’ may be diverse or contested. The notion of one moral route may therefore still be elusive” (Mason, 1996:29-30). Mason (1996) suggests that researchers ask a number of questions to themselves like: (i) which parties, bodies, practices, or whatever, are potentially interested or involved in or affected by a particular study? (ii) What are the implications of framing particular research questions in a study for these parties, bodies, practices, and so on? Naturally researchers’ answers to questions like these will not inform them whether their research questions are ethical or not, but will guide them towards identifying the potentially complex range of interests touched upon by their studies. If researchers are explicit about these interests, they can begin to work out which courses of action seem the most reasonable and ethical, and which do not and are better left alone.

ASSESSING THE TRUSTWORTHINESS OF QUALITATVE RESEARCH he conventional criteria for good research are: (i) internal validity, (ii) external validity or representativeness, (iii) reliability and (iv) objectivity. Qualitative researchers generally regard these constructs as inappropriate in establishing the “truth value” of a qualitative research project. Two prominent qualitative researchers, Lincoln and Guba (1985), propose the following four alternative constructs they believe reflect the assumptions of the qualitative paradigm more accurately: • Credibility/authenticity Is there a match between research participants’ views and researchers’ reconstruction and representation of it? • Transferability Can the findings of the research be transferred from a specific situation or case to another? • Dependability Is the research process logical, well documented and audited?

790

Journal of Public Administration • Vol 44 no 4.2 • November 2009

• Confirmability Does the researcher provide evidence that corroborates the findings and interpretations by means of auditing? The first of these criteria namely credibility is considered as the most important one. Lincoln and Guba (1985) outline various strategies to increase the credibility of qualitative research: • Prolonged engagement and persistent observation in the field; • Triangulation of different methods; • Peer debriefing; • Member checks; and • Formalised qualitative methods such as grounded theory and analytic induction. However, researchers (Schwandt, 1996) became increasingly critical of using research criteria based on the quantitative paradigm. They argued that these criteria are based on positivism and is therefore incompatible with qualitative research. How indeed could criteria based on erklaren be used to evaluate research based on the premise of verstehen? Researchers, especially post-modernists, challenged with renewed intensity the use of any evaluation criteria to measure the quality of qualitative research. They argued that these criteria are incompatible with the features of qualitative research and that it is especially unfair to use criteria based on positivism to evaluate qualitative (interpretive and constructivist) research. Therefore they argued that a separate set of evaluation criteria is needed (Bryman, 2007). Many other attempts were made to develop criteria for “good” qualitative research based on the qualitative paradigm (Giorgi, 1988; Kavale, 1996 and Stiles, 1993). From the many formulations we will discuss only a few prominent ones that could help to further the discussion on the use of the internal audit as a possible means to enhance the quality of qualitative research. Morse (1994) focuses her criteria for qualitative research on the analysis of qualitative data by stressing the following: • Comprehension (learning about a setting); • Synthesising (identifying patterns in the data); • Theorising (explanations that fit the data) and; • Recontextualising (abstracting emergent theory to new setting and relating it to • established knowledge). Hammersley (1989; 1990 and 1992) adds to these criteria by developing the concept internal reflexivity that leads to the development of the following criteria: (Kincheloe and McLaren (1998) elaborate on this concept). • Reflexive interrogation by the researchers of the epistemological baggage they bring with them. • Through a critical ethnography, researchers attempt to sensitise themselves and participants to how hegemonic regimes of truth impact upon the subjectivities of the disadvantaged. W. Schurink

791

• Positivist conception of validity rejected in favour of the credibility of socially constructed realities to those who have democratically participated in their development. • Ability to generalise rejected in favour of accommodation – where researchers use their knowledge of a range of comparable contexts to assess similarities and differences. • Catalytic validity - extent to which research changes those they study so that they understand the world in new ways and use this knowledge to change it - link to pragmatist criterion of practical adequacy (http://bgpinqmr.group.shef.ac.uk / workshop/Facilitators_Guide_7.pdf). Elliot, Fisher and Rennie (1999) developed what they term evolving guidelines for critiquing qualitative research, namely: • Owning one’s perspective, meaning that researchers describe their theoretical, methodological and personal assumptions. • Situating the sample, where the researcher describes the research participants and their life circumstances. • Grounding in examples. Examples of the data are presented to illustrate analytic procedures used in the study to show how the researcher’s understanding developed from it. • Providing credibility checks. For example, checking understandings with participants, using triangulation and comparing varied qualitative perspectives (analytic(al) induction). • Accomplishing general vs. specific research tasks. General understanding of the phenomenon should be based on an appropriate range of participants and situations. The findings are not intended to be extended to other contexts or participants. In the case of specific understanding, participants and situations are systematically and comprehensively described to provide the reader with a basis of understanding and it also addresses limitations of transferability. • Resonating with readers, meaning that the material is presented in such a way that the reader judges it as credible. Stating that the traditional criteria for qualitative research are inappropriate, Yardley (2000) proposes the following criteria that could, according to her, lead to flexible interpretation: • Sensitivity to context. Here, the following dimensions are important: theory building, use of relevant literature, awareness of the socio-cultural setting of the study, the relationship between researcher and participant, and discussion of ethical issues. • Commitment and rigour. This refers to an in-depth discussion of the topic/argument and methodological competence through data collection and in-depth analysis of the data. • Transparency and coherence. The arguments must be clearly described, there must be a fit between theory and research strategy and the methods used, and the researcher should disclose and critically reflect on all his/her decisions taken during the entire research process.

792

Journal of Public Administration • Vol 44 no 4.2 • November 2009

• Impact and importance. Theoretical, methodological and practical contributions should be made. Charmaz (2007) presents the following premises, principles and practices for quality qualitative research:

Premises • A deep understanding of a studied life means entering it and thus taking an insider perspective, and presenting an incisive analytical interpretation. • To understand what is happening in a setting we need to know what things mean to participants. Grasping the meaning participants have of their innermost experiences and feelings, unlock our understanding of their actions and intentions. • To learn participants’ meaning we need to be reflexive upon our own. • Relationships between meanings and actions are dynamic and reciprocal. Actions impact on meaning and meaning shapes actions. • Flowing from this, the questions we ask and how we ask it, shape the answers we obtain. For example, our way of gathering data impacts on what participants tell us. • Truths are relative, multiple and subject to redefining. For example, feminist-orientated inquiry questioned traditional viewpoints on minority groups and women.

Principles • A deep understanding of a studied life is based on the principle of intimate familiarity with the phenomenon. • Show a deep respect for our participants as human beings and treat them with dignity. • Using established theoretical perspectives as starting points, not ending points, can help us to generate new theoretical insights. • Accuracy means collecting sufficient data in order to have a full range of observations of the phenomenon, not just theoretical saturation. • Studying meanings and processes at both the subjective and social levels illuminates the context.

Practices Avoid using qualitative research methods in a mechanical way by: • opening yourself up to the experience by allowing space for the unexpected to happen; • gathering sufficient data to make your study credible; • paying attention to language; and • looking beneath the surface. W. Schurink

793

From the above, it is clear that many differently orientated qualitative research scholars attempted to adapt or re-write the criteria for qualitative research as originally proposed by Lincoln and Guba (1985). Apart from the attempts discussed above, there are many other appraisal criteria lists for qualitative research, each mirroring the different ontological and epistemological viewpoints of those who constructed it (Attee and Milton 2007:262). The positivist will emphasise internal validity, construct validity, external validity, reliability and objectivity. The realist stresses internal reflexive audit trails to demonstrate credibility, dependability, confirmability, transferability and ecological validity or naturalism, while subjectivists or constructionists would mostly agree with Schwandt that all measures to assess qualitative research should be abandoned (http://bgpinqmr.group.shef.ac.uk/ workshop/Facilitators_Guide_7.pdf). The important point is that despite the many attempts, there is still a lack of consensus among researchers as to what criteria would be acceptable to measure the quality of qualitative research. “A lack of consensus among researchers means that standards for good qualitative enquiry remain elusive” (Attee and Milton, 2007:262). Finding the golden standard for the evaluation of qualitative research is therefore difficult if not impossible since qualitative researchers differ vastly in their approach, each having their own theoretical background, methodological principles and aims (Flick, 2007:6). Just as there is no consensus between the different qualitative approaches there can never be a list of criteria for the appraisal of qualitative research that would be accepted by all qualitative researchers. This means that when judging the quality of qualitative research, the truthfulness of the study can only be justly validated if the reader/ reviewer is informed and understands the epistemological and ontological position the researcher adopted. Differently stated, when evaluating qualitative research, the researcher’s theoretical approach, research strategy and methodology followed in the study should be seen as the launching pad from where the trustworthiness of the study could be critiqued. From a scan of recent literature, it appears that presently at least three distinct perspectives on assessing the quality of qualitative research can be found amongst scholars. These are: (i) qualitative and quantitative research should be evaluated by the same measures, (ii) qualitative research should be evaluated by standards that have been particularly developed for it, and (iii) what Holloway and Wheeler (2002) call criteriology, that is, abandoning measures to asses qualitative work. “The belief that qualitative research would achieve paradigm takeoff by imitating the methods of natural sciences is thus to a greater extent discarded by qualitative researchers and in particular by postmodernists (Schwandt, 2007:40). In the process qualitative researchers came up with more and more checklists and frameworks for the development of alternative criteria based on qualitative research principles” (Flick, 2007). Existing criteria for trustworthy or credible research can never meet the approval of all qualitative researchers. It is therefore not surprising that the notion of developing criteria of soundness to meet the approval of all qualitative researchers has to date been discarded to a large extent, or as Schwandt (2007) states that qualitative researchers have gone beyond it. This however, does not mean that qualitative research has become unscientific

794

Journal of Public Administration • Vol 44 no 4.2 • November 2009

and that anything now goes. In fact, qualitative researchers have now turned to the use of specific strategies to ensure rigour and quality in the research process.

STRATEGIES TO ENHANCE THE QUALITY OF QUALITATIVE RESEARCH ince a clear and universal solution to the quality question, or criteria for good qualitative research has to date not been attainable, researchers started to apply strategies to promote quality in the research process. Thus, the notion of management of the research process and transparency with specific reference to the auditing trail1 and reflexivity became important strategies to ensure quality research. The auditing trail displays the interaction between the researcher and his/her research participant(s) in such a way that the research can be understood not only in terms of what was discovered but also how it was discovered. The advantage of this is that our interpretations can be better understood and validated by readers who are informed about the position we adopt in relation to the study, and by our explicit questioning of our own involvement. By revealing aspects of themselves and the research process as a traceable audit trail, qualitative researchers establish the credibility, dependability and confirmability of findings, and thereby show that the research was rigorously done Seale (1999). For qualitative researchers engaging themselves with difficult questions and issues in the reasoning process through which they arrive at their answers, it is today not only important to properly plan their studies, but also to provide a reflexive account of how it was done, e.g. constructing natural histories, research stories, or internal audits, and/or defending the logic of their decisions. Therefore, in order to be able to refer to particular issues you were confronted with during various phases of the development of your research proposal, as well as the decisions you took and your reasons therefore, you will be well-advised to keep a research diary or journal from the onset of the research. The problem of reflexivity has been discussed extensively by many researchers (Denzin, 1997, 1995; Hammersley and Atkinson, 1995 and Etherington, 2006) to name but a few. It is important to note that the post-modern trend in qualitative research encourages researchers to undertake and record a continuous critical analysis of their thinking and feelings concerning their conceptual framework, research questions, methods, values and biases. They must also have an awareness of their presence in the very situations they want to study, as well as of the inter-subjective dynamics between themselves as researchers and the research they are conducting. Such a practice forms an important tool that could be used to enhance the quality of qualitative research (see Figure 1). This argument could be taken a bit further by reasoning that researchers, relying on interpretive methods, could only make sense of their data if they continuously reflect on their decisions taken during the research process and having a heightened awareness of their epistemological, theoretical, and ontological conceptions and how this impact on their research practices (Mauthner and Doucet, 2003). The suggestion is made by (Mauthner and Doucet, 2003) that the more researchers can be self-conscious about, W. Schurink

795

Figure 1: Elements of the auditing trail

PROCESS OF OBSERVATION AND REFLEXIVITY Reconstruction of data and results of synthesis: categories, findings and integration of it with the literature

DISTORTED BY: Personal notes and expectations of the participants

Data reduction and results of syntheses: summarising and theoretical notes.

SOCIO-CULTURAL

EMOTIONAL Process notes: methodological notes and decisions regarding trustworthiness and credibility of findings

Observational notes

THEORY-LADEN PHYSIOLOGICAL BACKGROUND

The raw data, the collection and recording FACTS OF REALITY

and articulate their role in research processes and products, the more readers can engage in symbolic dialogues with the author(s), and the more their confidence in the work will increase. More specifically, it could help the researcher to be reflexively interconnected in the research process, particularly when analysing his/her data. The analysis of qualitative data is mostly regarded as a process of sorting, organising, and indexing the information obtained. This is often wrongly seen as a mechanical process that takes place in a social vacuum. In reality (as is clear from Figure 1), researchers are all but invisible since they have their own ontological, epistemological and theoretical assumptions as well as their own backgrounds, personal, emotional and interpersonal influences that they bring with them to the research process (Mauthner and Doucet, 2003). It has therefore become more important for qualitative researchers to reflect on the research process in terms of: What happened? Who was involved? Where did the activities occur? What circumstances or issues impacted on the data? What are the major issues emerging? What issues need to be followed up on? It also involves exploring the meaning of the data further by asking “so what” and “what if” questions. The reflexive process is very helpful to relating occurrences, for example words, expressions, interactions and social processes to people, events, other occurrences, values and norms of particular

796

Journal of Public Administration • Vol 44 no 4.2 • November 2009

groups of people. Discovering such linkages are important in selecting further theoretical incidents, persons, behaviours, etc. and establishing and verifying evolving ideas, themes and typologies. The practice of reflexivity questions the assumption that research findings naturally emerge from the process of data collection and analysis. Researchers supporting the practice of reflexivity feel that findings are shaped by the choices made by researchers and their critical reflection of these choices. (http://bgpinqmr.group.shef.ac.uk/workshop/ Facilitators_Guide_3.pdf). In this regard, reflexivity can be described as an effort to understand how the process of doing research shapes its outcomes. Many attempts are made to define reflexivity (Hardy, et al., 2001; Alvesson and Skoldberg, 2000; and Woolgar, 1988). Perhaps the best definition is that of Etherington (2006): “This means interpreting our own interpretations, looking at one’s own perspectives, and turning a self-critical eye into one’s own authority as interpreter and author … this enhances the trustworthiness of the findings and outcomes of research”. This definition reflects the essence of reflexivity namely that it entails a critical appraisal of the knowledge produced and how that knowledge was generated (Guillemin and Gillam, 2004). “In essence, reflexivity entails engaging in critical appraisal of own practice... Reflecting on why we frame issues in particular ways, investigate them in particular ways, and how such approaches lead us to particular kinds of solutions and theories and not others” (http://bgpinqmr.group.shef.ac.uk/workshop/ Facilitators_Guide_3.pdf). Although self-critical, a clear distinction should be drawn between being critical and being judgmental. In fact, Davies, Browne, Gannon, Honan, Laws, Mueller-Rockstroh and Bendix Petersen (2007) warn us that “the moral gaze restricts, binds one’s thoughts, defers and refers constantly to the discourses of respectability, of morality, of discipline of method… The reflexive eye/I in contrast, ideally gazes without judgement, like a good novelist, finds the unexpected, the surprising – the contradictions, the “good” and the “bad” in all its detail – not with a mind to censor, but to say with fascination “oh so that’s how it is!” As Charmaz (2007:13) states: “… enter the phenomenon and open yourself to the research experience. Face the inevitable ambiguities. Flow with the existential dislocation of bewilderment. Bring passion, curiosity and care to your work. In the end, you will transform our images of studied life, and your research journey will transform you”. On a deeper level, reflexivity also means continuously questioning your understanding of the phenomenon, synthesising the data by identifying patterns, theorising by developing explanations that fit the data and recontextualising by relating emergent theory to established knowledge. Therefore the process of continuous reflexivity should help us and others to understand why we frame questions in particular ways, investigate them in a certain manner, how we gain an understanding of the meanings research participants give to their life-world, gather sufficient data to make our research credible and how such approaches lead us to interpret the data the way we do (http://bgpinqmr.group.shef.ac.uk/ workshop /Facilitators_Guide_3.pdf). Just as in the case of criteria lists, various frameworks for reflexivity have been offered by differently orientated researchers. For example Johnson and Duberley (2003) equate different kinds of reflexivity with different kinds of theoretical commitments (e.g. W. Schurink

797

positivism, postmodernism and critical theory), Holland (1999) talks about four different ‘levels’ and Finlay (2002) suggests five different ‘maps’ (http://bgpinqmr.group.shef. ac.uk/ workshop /Facilitators_Guide_3.pdf). In order to gain a deeper understanding of the concept reflexivity it is necessary to place it within the framework of methodology, epistemological commitments and disciplinary maps (http://bgpinqmr.group.shef.ac.uk/ workshop /Facilitators_Guide_3.pdf). Methodologically, we are centrally concerned with practices and procedures for research and how these impacted on in the conclusions reached, i.e. how and why the research was designed, conducted and analysed, and how this led to particular conclusions. Questions asked in this regard include: • How should the research be designed or conducted in order to provide a convincing account? • What alternative interpretations are there? • What role do researchers play in producing results? • What choices were made and what were the reasons for them? Reflexivity on the methodology of the research is important because it renders the research process transparent and accountable (http://bgpinqmr.group.shef.ac.uk/ workshop /Facilitators_Guide_3.pdf). Ontological and epistemological reflexivity concerns our assumptions about what we can know and how we can claim to know it. ‘An important function of reflexive analysis is to expose the underlying assumptions on which arguments and stances are built (Holland, 1999). We become more consciously reflexive by thinking about our own thinking, by noticing and criticising our own epistemological pre-understandings and their effect on research, and by exploring possible alternative commitments (Johnson and Cassell, 2001). Epistemological assumptions reflect researcher commitments to particular philosophical beliefs. But all ontological and epistemological stances have weaknesses. Consequently awareness, debate and reflection are crucial if we want to enhance the quality of qualitative research. In this regard the following questions should be asked. • What can our measures actually tell us about the nature of the world and human action? • What are our aims in conducting the research? • What assumptions are implicated in the theories that drive our research and are produced as a result of our research? It is necessary for us to reflect on how our disciplinary assumptions and background may structure a research project and how such assumptions may favour the production of certain types of knowledge, and how participants may be affected by this. For example, in psychology, the ‘psychologising’ of the individual (examination of ‘internal’ cognitions etc.) can have the effect of individualising or putting responsibility on the individual and ignoring the structural/policy aspects of a situation (http://bgpinqmr.group.shef.ac.uk/ workshop/Facilitators_Guide_3.pdf).

798

Journal of Public Administration • Vol 44 no 4.2 • November 2009

However, despite the growing interest in reflexivity as a method to enhance quality research, the theoretical and philosophical discussions about reflexivity and its relationship with the construction of knowledge remain weak (Davies, Browne, Gannon, Honan, Laws, Mueller-Rockstroh and Bendix Petersen, 2007). Researchers are offered little guidance as to understand how to turn a self-critical eye on themselves and decisions that they took during the research process and how these decisions impacted on their study (Mauthner and Doucet, 2003) 2. Researchers are also scared to reflect by putting words on paper for everyone to see. As Davies, Browne, Gannon, Honan, Laws, Mueller-Rockstroh and Bendix Petersen (2007) warn: “Reflexive writing can be passionate and emotional, it can be writing in which the mind, heart and body are all engaged. Yet once those words are out there in the world, objects themselves of reflection by others as well as ourselves, they can become weapons to turn against us….The self is both subject and object of the reflexive eye…” Yet no researcher can deny that he/she is present in the research setting, in field note writing, in analysing and interpreting the data, and in the writing up of the findings (Davies, Browne, Gannon, Honan, Laws, Mueller-Rockstroh and Bendix Petersen, 2007). This is true even when he/she takes an objectivist stance. Then the question is how can the researcher not reflect on the research process? Denzin (1997) refers to the reflexive individual in the world. To take this argument further, everyone reflects upon life in order to make meaning of it. Reflexivity could thus be seen as the researcher’s attempts to make meaning of his/her research journey. The construction of knowledge in research is therefore a reflexive process that could bring rigour to the study (Guillemin and Gillam, 2004). This ensures as Flick (2007:64) states that “qualitative research, including research with a post-modernistic paradigm should be developed and produced in the tensional field of (theoretical, conceptual, practical and methodological) creativity and (methodological) rigour in studying phenomena. Therefore quality should be located in the tensional field – between being rigorous and being flexible”. The goal of reflexivity in this sense has to do with improving the quality of the research (Guillemin and Gillam, 2004). Seeing that reflexivity plays such an important role in qualitative research, the question is how can the practice of reflexivity be enhanced and clarified? Schatzman and Strauss (1973) developed a model or scheme, which considers the prerequisites of reflexive field-note taking. In many ways this scheme provides the specific guidelines lacking in the frameworks proposed by Johnson and Duberley (2003), Holland (1999), and Finlay (2002). This scheme has been utilised by various qualitative researchers. This model originally consisted of three elements, namely observational notes, theoretical notes and methodological notes. Richardson (2004) later added a fourth category, namely personal notes. The elements of this suggested model as depicted in Figure 1 may be described as follows: • Observational notes (ON). Observational notes are detailed notes on what happened, what was heard, seen and experienced. “Each ON represents an event deemed important enough to include in the fund of recorded experience, as a piece of evidence for some proposition yet unborn or as a property of context or situation. An ON is the W. Schurink

799

Who, What, When, Where and How of human activity. It tells who said or did what, under stated circumstances” (Schatzman and Strauss, 1973:100). • Methodological notes (MN). Methodological notes are mainly reminders, instructions and critical comments to the recorder or researcher on how to collect the data, how to improve the quality of the interviews, whom to talk to next, etc. “Methodological notes might be thought of as observational notes on the researcher himself and upon the methodological process itself; as complete a chronicle as the recorder finds necessary or fruitful “(Schatzman and Strauss, 1973:101). • Theoretical notes (TN). Theoretical notes are self-conscious, systematic attempts by the researcher to critically reflect on what took place (ON’s), what he/she thought and experienced. Schatzman and Strauss (1973: 101) write: “The observer as recorder thinks about what he has experienced, and makes whatever private declaration of meaning he feels will bear conceptual fruit. He interprets, infers, hypothesises, conjectures; he develops new concepts, links these to older ones, or relates any observation to any other…” • Personal notes (PN). These are the researcher’s critical reflection about his/her feelings about the research.

CONCLUSION he above guidelines could help us to reflect critically on how we construct knowledge from the research process and how the quality of that knowledge could be enhanced. More specifically it could help us to answer the following essential questions about the research process: • What kind of factors influenced the researcher’s construction of knowledge? • How are these influences accounted for in the research process? These guidelines could also help us to focus on improving the quality of qualitative research by not only critically evaluating the research methods and the data but also the researcher, the participants and the research context. Since these proposed guidelines also focus on the interpersonal aspects of the research and the interaction between researcher and participant, it also reflects on the very important ethical dimensions of research. Therefore, the use of the above guidelines could help researchers to do both rigorous and ethical research. By using it to critically reflect on ourselves and our participant(s) within the framework of our research design, we could enhance the quality of our research. As Etherington (2006:32) states: “If we can be aware of how our own thoughts, feelings, culture, environment and social and personal background inform us as we dialogue with participants, transcribe their conversations with us and write our representations of the work, then perhaps we can come close to the rigour that is required of good qualitative research”.

NOTES 1 That is, a systematically maintained documentation process of the researcher’s continuous critical analysis of all deci-

800

Journal of Public Administration • Vol 44 no 4.2 • November 2009

sions and actions taken during the entire research process, often also referred to as research story (Schwand 2007). 2 The only exception is interviewing and the researcher-participant relationship (Denzin 2001).2.

Bibliography Alvesson, M. and Skoldberg, K. 2000. Reflexive Methodology: New Vistas for Qualitative Research. London: Sage Publications. Attee, P. and Milton, B. 2007. Critically appraising qualitative research for systematic reviews: defusing the methodological cluster bombs. In Bryman, A. (Ed). Qualitative research 2 volume II – Quality issues in qualitative research. 261–280. Thousand Oaks: Sage. Bryman, A. 2007. Editor’s introduction. In: Bryman, Alan (Ed.). Qualitative research 2. Volume 1. Collecting data for qualitative research (pp. xxiii-xlv). London: Sage. Charmaz, K. 2007. Premises, Principles, and practices in qualitative research: Revisiting the foundations. In Bryman, A. (Ed.) Qualitative research 2 volume II – Quality issues in qualitative research, pp.112-134. Thousand Oaks: Sage. Davies, B., Browne, J., Gannon, S., Honan, E., Laws, C., Mueller-Rockstroh, B., and Bendix Petersen, E. 2007. The ambivalent practices of reflexivity. In Bryman, A. (Ed.) Qualitative research 2 volume III –Issues of representation and reflexivity, pp. 136–167. Thousand Oaks: Sage. Denzin, N.K. 1997. Interpretive ethnography: Ethnographic practices for the 21st century. Thousand Oaks: Sage. Denzin, N.K. 2001. The reflexive interview and performative social science. Qualitative research, 1(1), pp. 23–46. Elliot, R., Fisher C.T. and Rennie, D.L. 1999. Evolving guidelines for publication of qualitative research studies in psychology and related fields. British Journal of Clinical Psychology. Flick, U. 2007. (Eds.) Designing qualitative research. The SAGE qualitative Research Kit. Thousand Oaks, CA: Sage. Guba, E.G. and Lincoln, Y.S. 1989. Fourth generation evaluation. Newbury Park, CA: Sage. Giogi, A. 1988. Validity and reliability from a phenomenological perspective. In Baer, Moss, L., Rappard, H. and Stam, H. (Eds.), Recent trends in theoretical psychology, New York: SpringerVerlag. Guillemin, M. and Gillam, L. 2004. “Ethics and reflexivity, and Ethically important moments” in research. Qualitative Inquiry, 10(2), pp. 261–280. Hammersley, M. 1990. Reading Ethnographic Research: A Critical Guide, Longman: London. Hammersley, M. 1992. What’s Wrong with Ethnography? Routledge: London. Hammersley, M. 1995. The Politics of Social Research. Sage: London. Hammersley, M. and Atkinson, P. 1995. Ethnography: Principles in practice. London: Routledge. Hardy, C., Phillips, N. and Clegg, S. 2001. Reflexivity in organization and management theory: A study of the production of the research ‘subject’. Human Relations, 54, pp. 531–560. Holland, R. 1999. Reflexivity. Human Relations, 52, pp. 463–483. W. Schurink

801

Holloway, I. and Wheeler, S. 2002. Qualitative research in nursing. (2nd ed) Oxford, UK: Blackwell. http://bgpinqmr.group.shef.ac.uk/workshop/Facilitators_Guide_2.pdf. Johnson, P. and Cassell, C. 2001. Epistemology and work psychology: New agendas. Journal of Occupational and Organizational Psychology, 74, pp. 125–141. Johnson, P. and Duberley, J. 2003. Reflexivity in management research. Journal of Management Studies, 40, pp. 1279–1303. Jones, M.O. 1996. Studying Organizational Symbolism: What, How, Why?, Qualitative research methods series, Vol. 39(1). Kavale, S, 1996. Interviews: An introduction to qualitative research interviewing. Thousand Oaks: Sage. Kinchloe, J.L. and McLaren, P.L. 1998. Rethinking critical theory and qualitative research. In Denzin, N. and Lincoln, Y. (eds.) Handbook of Qualitative Research. London: Sage. Lincoln, Y.S. and Guba, E. 1985. Naturalistic Enquiry. Beverly Hill CA: Sage. Mason, J. 1996. Qualitative researching. Thousand Oaks: Sage. Mauthner, N.S. and Doucet, A. 2003. Reflexive accounts and accounts of reflexivity in qualitative data analysis. Sociology, 37(3), pp. 413–431. Morse, J.M. 1994. Emerging from the data: the cognitive process of analysis in qualitative enquiry. In Morse, J.M. Critical Issues in Qualitative Research Methods. London: Sage. Neuman, W.L. 1997. Social research methods: Qualitative and quantitative approaches. Boston: Allyn and Bacon. Richardson, L. 2004. Writing a method of inquiry. In Hesse-Biber, S. and Leavy, P. (Eds.), Approaches to qualitative research. A reader on theory and practice. New York: Oxford University Press, pp. 473–495. Schatzman, L. and Strauss, A.L. 1973. Field research; strategies for a natural sociology. New York: Prentice Hall. Schwandt, T.A. 1996. Farewell to criteriology. Qualitative inquiry, 2(1), pp. 58–72. Schwandt, T.A. 2007. The Sage dictionary of qualitative inquiry. Thousand Oaks: Sage. Seale, C. 1999. Quality in Qualitative Research. Qualitative Inquiry, 5(4), pp. 465–478. Stiles, W.B. 1993. Quality control in qualitative research. Clinical Psychology Review, 13, pp. 593–618. Woolgar, S. (ed.) 1988. Knowledge and Reflexivity. London: Sage. Yardley, L. 2000. Dilemmas in qualitative health research. Psychology and health, 15(2), pp. 215–228.

802

Journal of Public Administration • Vol 44 no 4.2 • November 2009