Fitzgerald Skills

Skills for Evaluating Web-Based Information Mary Ann Fitzgerald, Ph.D. Department of Educational Psychology and Instructional Technology University of Georgia Children can do anything with technology; they are born with a chip in their brains that makes them naturally attuned to all things wired. Along with this new physical endowment, they are naturally able to navigate the Internet, seeking, finding, and applying information for all their needs. They are invulnerable to the scams and low-quality information lurking there. Absurd? Yes. However, this false belief – that children are somehow naturally more able to use technology and are also somehow more Internet savvy – is a common one. Problems with Internet information and in more traditional forms of information are well documented. Professors of entering college undergraduates observe that students “are ill prepared to function in a technological and information-rich environment” (Quarton, 2003, p. 120). The Stanford Web Credibility Project (Stanford Persuasive Technology Lab, 2004) documents gullibility in the entire Internet-using population. While it is true that many of today’s youngsters live in a wired world from birth and are hence more comfortable and adept with technology than their elders, problems remain. These problems are old problems: how to detect misinformation; how to tell when someone is lying; how to counter the subliminal commercial messages that saturate much broadcast media today; and how to evaluate the strength of an argument, among many others. While much could and should be done to apply technology to improve web information quality, a major portion of the task will continue to fall on users. Children and youth are traditionally considered among the most vulnerable members of our society. This paper focuses on the skills that people, especially young people, need to evaluate information. With this focus, I can only provide a brief overview of a vast territory, synthesized through the lens of the user’s cognitive perspective. The important problems of information quality and the philosophical controversies concerning censorship and filtering are beyond the scope of this paper. The first section describes and defines the basic process of evaluation, synthesizing ideas from cognitive psychology and critical thinking theory. The second section discusses the importance of goals and information context, and the third discusses the critical Skills: Fitzgerald

1

beginning of the whole process. Finally, I list the special problems children have in evaluating information, skills we all need to evaluate information, and remaining research questions.

Evaluating web-based information: Breaking down a complex process Discerning the credibility of web-based information is essentially an evaluative task. As such, it is usually situated within the context of a larger task, such as increasing background knowledge, making a decision, or constructing an argument. Research shows that evaluating information is itself a complex task made up of many smaller elements (Fitzgerald, 2000). What, then, are the specific skills needed to evaluate information on the web? Conversely, what factors may inhibit the process of evaluating information well? In simplest terms, to evaluate is to judge the quality of an idea, an object, or a person. Components of evaluation include critical thinking, metacognition, epistemology, prior knowledge, and strategies, leading to deliberation and eventual decision. These components are grounded in some kind of information context and influenced by a set of possible factors. Evaluation as a process has been mapped by at least three teams (Fitzgerald, 2000; Fitzgerald & Galloway, 2001 [see Appendix A]; Fogg & Tseng, 1999; Wathan & Burkell, 2002), all of which owe much to Petty and Cacioppo’s Elaboration Likelihood Model (1986). All three models are theoretical, supported by some empirical data, but not yet thoroughly tested. Rather than spend a great deal of time analyzing, comparing, and explaining them here, I will simply say that they are complex. Interestingly, the three have many characteristics in common. Two of them (Fitzgerald/Galloway and Wathen/Burkell) point to the iterative nature of evaluation, meaning that users may cycle back and forth in some evaluative situations. All three contain strategies and algorithmic questions. While these need more study, here I will discuss several primary components, leading ultimately to the list of skills that is the goal of this paper. These components include a cluster of processes known as critical thinking, metacognition, epistemology, prior knowledge and bias, and strategy use. I will also provide related findings, which often point to problems in the process.

Skills: Fitzgerald

2

Critical thinking Evaluation is closely associated with critical thinking. Some writers such as Beyer (1985), D’Angelo (1971), and Yinger (1980) seem to equate “critical thinking” with “evaluation.” Most theorists, however, describe critical thinking as including evaluation among several other higher order thinking processes such as problem solving, decision making, and analysis (Cromwell, 1992; Ennis, 1989; Paul & Elder, 2001). Another key relevant theory is Bloom’s Taxonomy of cognitive skills, which places evaluation at the top (or most complex) of a range of thinking activity (Bloom, Engelhart, Furst, Hill, & Krathworhl, 1956). Because of these ties between evaluation and critical thinking, much theory and research about critical thinking informs an understanding of evaluation. Within the critical thinking paradigm, evaluation is defined as the making of judgments about the value, for some purpose, of ideas, works, solutions, methods, etc. The target of evaluation can be an object, as in a piece of art, an idea, or a person. Most writers list component processes such as finding inconsistencies, comparing and contrasting, and judging by criteria (Ennis, 1987). When information is the object of evaluation, a person typically studies it for reliability, quality, credibility, and personal usefulness. These qualities overlap in meaning, but together they describe what a person considers when judging information, leading to the idea of criteria discussed later. Bloom et al. (1956) also acknowledge a “link with the affective behaviors” (p. 185), due to the inclusion of values. This affective link is richly born out in empirical literature, leading often to biases, to be discussed later. Within the world of education, it is vital to note that standardized tests provide only rudimentary measures of critical thinking and information literacy (Dunn, 2002; Partnership for 21st Century Skills). This shortcoming is understandable when given the parameters of rapid, mass assessment. Unfortunately, standardized testing tends to drive curriculum. Therefore, difficult-to-assess skills like critical thinking and information literacy are often neglected systematically in schools. Metacognition Although the relationship between metacognition and evaluation may not be readily apparent, effective evaluation may not be possible without at least some thinking about one’s

Skills: Fitzgerald

3

own thinking. Flavell defines metacognition as “knowledge or cognition that takes as its object or regulates any aspect of any cognitive endeavor. Its name derives from this ‘cognition about cognition’ quality” (1981, p. 37). Brown, Bransford, Ferrara, and Campione (1983) identify two major strands of research usually labeled “metacognition.” One concerns knowledge about thinking, whereas the other concerns regulation of thinking and learning. Both strands are of interest here. Procedural knowledge of the evaluation process and memories of successful past thinking episodes form part of prior knowledge and will be discussed later. Metacognitive regulation is vital because the thinker must often consciously choose strategies or meta-strategies to apply in a given evaluative situation. Flavell calls the regulatory type of metacognition “cognitive monitoring.” Simply recognizing the need to evaluate information is probably a metacognitive event.

Epistemology Another factor that probably affects the evaluation process is epistemology. Epistemology is one branch of philosophy dealing with the nature of knowledge and sources of knowledge. Belenky, Clinchy, Goldberger, and Tarule (1986) coined the phrase “ways of knowing,” and it aptly defines epistemology as the beliefs people hold about how we come to know what we know. A thorough discussion of the impact of epistemology on information evaluation can be found in Fitzgerald (1999). The ideas a person holds about the origin of knowledge provides foundation for crucial elements of evaluation, such as the identification of authority and the choice of criteria. A person who believes some knowledge to be unquestionable may neither criticize new information about that knowledge nor consider new information that contradicts it. On the other hand, the person who believes in the fluidity of all or most knowledge may be more likely to consider new information in an evaluative light and how it may change knowledge already in memory. Much of the theory and research surrounding information evaluation is implicitly or explicitly based upon some assumptions about users’ epistemology. One notable problem is in identifying which web sites contain “true” information. “True” and “false” may vary according to the user. Arguably, information specialists have little right to make this distinction for any given user, leading to important philosophical dilemmas. To avoid this

Skills: Fitzgerald

4

debate, I will state that these assumptions are important, that epistemologies may drift as people mature, and that puzzling differences of opinion based upon similar evidence may well be traced to epistemological differences. King and Kitchener (1994) are the most important and relevant writers in this area.

Prior knowledge and bias Absorbing new information is difficult when no related constructs exist in memory (Gilovich, 1991). Therefore, people depend upon their prior knowledge when new information appears. Domain knowledge provides structure and corroboration and may point out inconsistencies. If new information agrees with prior knowledge, confidence in its reliability can justifiably increase (Stripling & Pitts, 1988). Paradoxically, prior knowledge may hamper the evaluation process, as when people ignore new information conflicting with belief (Abelson, 1986; Ross, Lepper, & Hubbard, 1975; Svenson, 1981). This paradox introduces a continuum that ranges between the enhancement domain knowledge lends to judgment to its suppression exhibited by people with strong biases in a domain area. It is also important to note that everyone has biases. In this respect, people vary mainly according to what these biases are about. Prior knowledge provides at least five specific advantages that assist reasoning: 1. Identification of what new information needs to be found, based upon gaps in prior knowledge (McGregor, 1994) 2. Procedural knowledge, or recipes for how to tackle problems (Ashcraft, 1994; Brown et al., 1983; Flavell, 1981) 3. Comparison between old information in memory to new information for consistency and hence trustworthiness (Baker, 1979; Flavell, 1981; Moskowitz & Stroh, 1996; Osman & Hannafin, 1992; Pitts, 1994) 4. Access to “relevant associations, images and experiences” (Petty & Cacioppo, 1986, p. 128), which provide structure and background for new information 5. Cues (or markers) of problems in the information environment, learned from experience

Skills: Fitzgerald

5

Due to the advantages listed above, prior knowledge is probably a necessary but insufficient condition for effective critical thinking (Ennis, 1989). However, Ennis asserts that prior knowledge both helps and hampers critical thinking. Any of the five types of stored information above may contain errors. Another highly significant problem is determining the difference between knowledge and belief, and belief mistaken for knowledge easily assumes the negative aspect of bias. People may become so convinced of their own expertise in a particular area that contradictory incoming information is dismissed without consideration. This problem links critical thinking to epistemology, and illustrates the uncertain demarcation between prior knowledge, beliefs, and bias. A lengthy discussion of the much-researched prior knowledge territory can be found in Fitzgerald (1999). Strategies One way to break down the complex process of evaluation is into a collection of strategies. Most of these strategies, if not all, may be conceptualized as questions users pose to themselves, or miniature operations (Fitzgerald, 2000). For example, one user might ask himself while browsing a web site, “How is this site organized?” The answer he discovers might be “It isn’t.” He might then ask “Is the information communicated in a professional, educated manner?” The answer to this might be “yes.” The answers to these strategic questions would figure in a summative judgment or decision later on, sometimes to perform other strategies, or leave the site. I have observed users employing meta-strategies as well, in which they conduct an operation consisting of several related strategies. For example, the meta-strategy of verifying web information through an outside source could be broken down into a series of questions, such as “What authority might I consult that would have the expertise to agree or disagree with this source? What does the outside source say? Does this outside information agree, in whole or in part, with the questioned resource?” and so on. Appendix B displays a number of disparate strategies gleaned from qualitative studies. Using one or more strategies, the evaluator deliberates for a brief or sustained time, ending in a decision of some kind. These deliberation and decision phases deserve a great deal of work and attention, but are beyond the scope of this paper. (For more information, see Fitzgerald, 2000; Fitzgerald and Galloway, 2001.)

Skills: Fitzgerald

6

An important meta-strategy is the deliberate choice and application of criteria. Criteria can be thought of as yardsticks or standards, and many theorists have furnished lists of them for many different situations, sometimes in the form of a rubric. Rubrics are popular because they scaffold the selection and application of criteria. Criteria in an information search context might include aspects of information quality such as objective content, sufficient depth, and clear articulation (Eisenberg & Small, 1993; Taylor, 1986). Paul and Elder list “clarity, accuracy, relevance, logicalness, breadth, precision, significance, completeness, fairness, depth” as “standards” (2001, p. 50). All three evaluation models listed earlier use strategies as components. The origin of strategies is most likely prior knowledge, possibly triggered in memory through learned associations. A major component of skill building for evaluative thinking would be for users to learn a large number of strategies to be employed in appropriate situations. A side benefit of thinking of evaluation this way is that strategies are proactive. Their mastery and deployment are in the control of the user, once the user knows about them. See Appendix B for a list of strategies.

Section summary In summary, evaluation of web-based information relates closely to educational and cognitive theories dealing with critical thinking. Major components of evaluation include metacognition, epistemology, and prior knowledge. All of these underlie strategies that users apply in information settings. As such, evaluation involves many sub-processes or strategies, including the application of criteria.

Context and purpose Evaluation does not occur in a vacuum. In this section, I discuss the impact of user goals and list other influential characteristics of information-seeking situations. Goals Fritch (2003) contends: “…some information is not important enough to require careful evaluation, but each individual must determine when this is true” (p. 327). Purposes (or motivation) for using information include entertainment, fact collection, simple curiosity, Skills: Fitzgerald

7

collecting information to inform consumer decisions, and an infinite number of others. The cognitive strategies chosen and level of engagement depend largely upon this goal. In school library media literature, Weisburg and Toor (1994) point out that a reader’s purpose helps to establish source evaluation and selection criteria. In persuasion literature, Petty and Cacioppo (1986) identify motivation as one of several key factors determining whether or not a person will deliberate upon a problem and to what depth. To illustrate, a person browsing an online celebrity magazine for entertainment will probably be less likely to evaluate displayed information than a prospective car buyer searching for safety information. This illustration assumes that car buyers are generally more motivated to establish facts than entertainment-seekers. Partial support for this idea in another context is provided by accountability research. In accountability studies, subjects are made responsible for the outcomes of their judgments in some way. Judgment accuracy seems to increase as motivation to judge accurately increases, demonstrating that people can critically evaluate information if they choose to do so (Simonson & Nye, 1992). In several studies, subjects motivated to be accurate through expectation of accountability exhibited decreased susceptibility to certain flawed judgment patterns (Bodenhausen, Kramer, & Susser, 1994; Freund, Kruglanski, & Shpitzajzen, 1985; Kruglanski & Freund, 1983; Tetlock, 1983). Returning to the realm of web information evaluation, Flanagin and Metzger (2000) found that greater motivation (and hence, engagement) affected attitudes toward the message and tended to lead to verification behaviors. If motivation increases evaluation accuracy through the deliberate avoidance of flawed thinking patterns, it is fair to assume that lack of motivation may decrease evaluation accuracy.

Other contextual factors It is clear that context greatly impacts thinking in general, and evaluation in particular. Many variables make up context, including physical environment, social situation, and the contexts surrounding the information itself. The web provides a new context with the inherent power and challenge of hyperlinking. Bilal (2000) found that the nonlinear, branching nature of web-based information leads to confusion in some young users, who become overwhelmed by the need to make navigation decisions. There are many relevant contextual factors, most

Skills: Fitzgerald

8

documented empirically as influential. Time is one such contextual factor. It seems logical to predict that less time spent deliberating will lead to lower-quality decisions, and research results support this proposition (e.g., Garbarino & Edell, 1997; Kruglanski & Freund, 1983). Information problem type, whether ill-structured or well-structured, is an often overlooked but crucially important aspect of all evaluation situations. A more detailed discussion of contextual factors can be found in Fitzgerald (1999).

Summary Any analysis of an information evaluation situation must begin with the user’s goal. Choice of criteria, strategies, and depth of engagement stem directly from this goal. Often overlooked, but still important, contextual factors include available time, environmental confusion, and the nature of the question itself.

The beginning: Signals and switches Now that I have briefly described elements of evaluation, it is important to return to a critical but neglected element: whether or not it begins at all. As Fritch (2003), quoted earlier, says, not all information is important enough to evaluate. How do users decide to start evaluation? If not by choice, what compels them to do so? This “switching on” part of the process is extremely difficult to study, because of the challenge of constructing research settings that naturally contain the right conditions to observe this phenomenon. However, the literature has much to say about dispositions toward critical thinking, cognitive signals to begin the process, and the initialization phase of evaluation. Disposition A precursor or necessary condition for evaluation may be what Ennis (1987) and Glaser (1985) label “disposition.” Glaser defines disposition as the “attitude of being disposed to consider in a thoughtful, perceptive manner the problems and subjects that come within the range of one’s experiences” (p. 25). However, like Fritch, Siegel and Carey (1989) point out that people cannot evaluate every message they encounter. Nor do they always evaluate information when they probably should. In fact, much empirical research vividly describes incidents in which people failed to evaluate fraudulent information (e.g., Aycock & Buchignani, 1995; Belli, Skills: Fitzgerald

9

1989; Bird, 1996). In an updated information context, Fogg et al. (2003) found that the most significant factor by far that participants considered in evaluating web sites were related to appearance. Flanagin and Metzger (2000) found that people seldom took steps to verify webbased information. It seems most likely that the strength of critical disposition varies among individuals, but also that it varies within the same individual from situation to situation. Several factors may work to increase a person’s disposition to think evaluatively in a given situation. Goals and prior knowledge are probably vital to successful cognitive performance. Memories of past experiences and content knowledge about a topic will equip an individual with tools to notice inconsistencies. Another factor may be epistemological orientation (King & Kitchener, 1994; Siegel & Carey, 1989). Certainly, numerous studies show that people will be critical when implicitly asked to do so. Signals themselves A critical early component of evaluation is the “signal,” named by both Flavell (1981) and Markman (1981) as the cognitive event initiating a metacognitive episode. Signals are the specific thoughts that launch the evaluation process, or recognitions that something may be wrong with the information. Unfortunately, even the best descriptions of these signals are vague and mostly theoretical. Flavell likens them to urges: “implicit signals to pay close attention, listen intently, read carefully, store or retrieve information intentionally, try to solve a problem” (p. 43). He also refers to them as “feeling[s]” of “vague puzzlement” (p. 45). Dewey calls them “a state of doubt, hesitation, perplexity, mental difficulty, in which thinking originates” (1933, p. 12). Siegel and Carey (1989) advocate sensitivity to “anomalies” (p. 2). Often, signals indicate miscomprehension. However, the problem could be that the information itself is flawed, and signals notify the reader of this possible flaw. The reader may then take steps to determine the stimulus of the signal, or ignore it. Questions remain about the specific source of signals and the mechanisms that generate them. Unfortunately, signals could easily be confounded by the confusion and frustration that users often feel when confronted with innocuous but disorganized or complex web sites, as Bilal (2000) found with 43% of youngsters in her study of children using a well-known children’s search engine.

Skills: Fitzgerald

10

Initialization Given the paucity of research that describes the “switching on” of evaluation, we have focused on this element in naturalistic studies (Fitzgerald, 2000; Fitzgerald & Galloway, 2001). Without cuing participants that I was looking for critical behavior, I observed them using information in projects of their own construction. In many incidences, they would be immersed in some information task, and suddenly they would say something critical about the information. Probing of these incidents revealed that there were different cues that made people jump from a browsing mode to a critical one. These evaluation initializations seemed to occur in one of two ways. In the first form, participants noticed a specific quality or clearly identifiable anomaly in the information and began evaluating because of the presence of this “problem marker.” For example, one participant noticed several typographical errors on a web page and began to evaluate the entire document more closely due to her discovery of these typos. Problem markers varied markedly among participants. Typographical mistakes were important to some, and ignored by others. For the most part, it seemed that participants had individualized sets of problem markers that were meaningful only to them. However, the problem markers themselves make sense as cues (but not proof) of the presence of information problems. Appendix C presents a subset of the problem markers recognized by participants. Fogg et al. (2001) found “commercial implication” and “amateurism” to be elements that hurt credibility, and these correspond to the idea of problem markers. In the second form, evaluation initialization occurred in response to a vague thought that something was wrong with the information, with no clear idea of what that something was. For example, one participant said she was “disappointed” after reading an article. It took several questions to uncover the specific reasons that led to this feeling of disappointment, and it was due to the presence of overgeneralizations. These thoughts correspond to the “cognitive signals” described by Flavell (1981, p. 43). Besides disappointment, other similar “signals” included distaste, disagreement, curiosity, surprise, and confusion.

Summary Because people cannot evaluate all incoming messages, there must theoretically be an “evaluation-on” switch. Perhaps some people are disposed to be “evaluation-on” proportionately more of the time than others. Specific thoughts and urges have been recognized as characteristic

Skills: Fitzgerald

11

of the initial recognition that something is wrong in information. Research has also identified specific cues or markers within information that people recognize as cues to start evaluating; some are listed in Appendix C. Still, this area is mysterious and poorly mapped.

Children Most of the literature above deals with adults. How are children different, if at all? According to a recent Corporation for Public Broadcasting study (2002), 65 percent of American children (aged 2-17) use the Internet. Tenopir, in her 2003 summary of many virtual library usage studies, summarized that high school and college students now use the Internet more than the library for research. Against the backdrop of these usage figures are several more troubling findings. A Pew study (Fallows, 2005) found that young users (defined as under 30) are more “trusting” than older users, although by a small margin. Tenopir also found that while young adult students seemed to be evaluating Internet materials, faculty did not always approve of their choice of criteria. In view of increasing patterns of Internet usage in young people, it is important to summarize what more traditional literature says about the evaluative abilities of children and youth: •

Ability to evaluate increases with age and education, with a notable developmental plateau when formal education comes to an end (Dunn, 2002; Garett & Wulf, 1978; King & Kitchener, 1994; Mackinnon, 1987).

•

Critical thinking, as measured by standardized tests, correlates positively with academic achievement (Garett & Wulf, 1978; Mackinnon, 1987).

•

Elementary-aged children can detect problems in information if forewarned (Baker,1984; Markman & Gorin, 1981), but are unlikely to if not (Markman, 1979). Older children have been observed to fail to evaluate in information-seeking contexts (McGregor, 1994; Pitts, 1994).

•

Children are “universal novices” (Brown & DeLoache, 1978, p. 14) and thus may generally lack relevant domain knowledge (prior knowledge), along with all associated benefits summarized earlier (Carey, 1985).

•

Children differ from adults in terms of goals, being less able or likely to articulate them, set reasonable ones, or to pursue imposed goals (Flavell, 1981; Markman, 1981).

•

Children are less able to pursue multiple goals at once (Markman, 1981).

Skills: Fitzgerald

12

•

Children may focus on one particular puzzling or interesting aspect of information, ignoring all others (Flavell, 1981).

•

Children are socialized to trust and obey authority (King & Kitchener, 1994) and later are disproportionately influenced by peers (Paul & Elder, 2001).

•

Children have difficulty distinguishing between knowledge and belief (King & Kitchener, 1994).

•

Pre-adolescents believe in the absolute reality of what they observe (Miller & Lipps, 1973; Osherson & Markman, 1974-1975).

•

Poor readers add challenges of decoding and comprehension to all of the above (Garner, 1980; Paris & Myers, 1981). •

While experience and maturity are not the same thing, inexperienced Internet users are particularly unlikely to verify the information they find (Metzger, Flanagin, Eyal, Lemus, & McCann, 2003). In summary, children are at a developmental disadvantage when it comes to evaluating

web-based information. Adolescence marks the beginning of overcoming these disadvantages. Education helps. Experience and background reading help, too. Youths who have created their own websites may be particularly cognizant of the ease of misinforming on the Internet (Jesdanun, 2004). It is likewise important to remember that many older adults are new Internet users and thus lack all of the advantages of experience in that context. Further, many have lower levels of education and poor levels of literacy, a factor often forgotten in the elitist expectation that everyone must go to college. Thus, many of the problems considered normal in developing children may persist in disadvantages adults.

Skills As an end to this paper, I will summarize and translate much of the theory and research into a proactive set of skills. •

•

Build literacy. Evaluation is based upon a good foundation of reading ability and information literacy (AASL/AECT, 1998) or “ICT literacy” -- information and communication technologies literacy (Partnership for 21st Century Skills). Notably, these last two definitions of information literacy contain evaluation as part of an overall ability to negotiate information seeking situations. Learn to define the information task and context with questions such as these:

Skills: Fitzgerald

13

• • •

• • • • •

o For my purpose, how credible must the site be? o Even in casual information use, are there possibly harmful elements or outcomes if the information is wrong? o List criteria appropriate to the context (e.g., Paul & Elder, 2001). Apply them appropriately. Raise awareness that misinformation exists and that people lie in many different ways for many different reasons. Build prior knowledge by reading deeply about the topic (Stripling & Pitts, 1988). Identify personal emotions, values, and biases about the topic; consider their impact on the current problem. Write them down. Seek information that supports the opposite perspective; resist the urge to ignore these opposing items, especially for ill-defined problems (controversial or open-ended questions with no known right or wrong answer). Know the typical cognitive signals, and how to respond. Know the most common problem markers (Appendix C). Be able to identify the author’s purpose; what explanations or motivations can be found underlying web sites? Be able to choose and execute a number of evaluative strategies. Many are listed in Appendix B. Learn the skills of argument and logic; these help expose flawed arguments.

Questions The most interesting questions about evaluation are the most difficult to attack through standard research methodologies. A major weakness and challenge in most of the research that has been done in the area of user evaluative behavior involves imposed questions. Both Quarton (2003) and Gross (1999) have discussed the problems children have with research when topic choice is not theirs. One of the most important of these is lack of motivation. Conversely, it seems clear that motivation is one of the most important elements in the whole process. As Dresang asserted in 1999, research should describe young Internet users in very natural conditions, pursuing questions of their own. Only through these self-determined information contexts can a real understanding of user behavior grow. Even more difficult, the question of how people move to a critical state from a relaxed one is crucial but overlooked by most studies. Almost daily, the media reports a successful scam or hoax. Why do people continue to fall victim to these deceptions despite numerous public warnings? Why do tabloid publications, notorious for printing inaccurate, unsubstantiated, and sensational information, continue to sell issues? Why do email hoaxes, some of them almost as old as the Internet, continue to circulate? Technology marches on, and how do all of these factors apply to newer web formats, like the

Skills: Fitzgerald

14

author-constructed and self-policed blogs and wikis, called “collaborative information assembly” by Jesdanun (2004)? Scams and hoaxes are usually mild in import. However, they raise the question: could an entire society be fooled about matters of importance? The answer to this question, arguably at least, is yes. Successful, although small, deceptions reflect the possibility that wholesale and tragic deceptions can occur. These are matters for adults, and we will continue to argue the philosophical problems of protecting children from them. Still, it would seem that re-educating an adult population and educating the young about the dangers of misinformation remain essential projects for the 21st century.

Skills: Fitzgerald

15

References Abelson, R.P. (1986). Beliefs are like possessions. Journal for the Theory of Social Behaviour, 16 (3), 223-230. American Association of School Librarians, & Association for Educational Communications and Technology. (1998). Information power: Building partnerships for learning. Chicago: American Library Association. Ashcraft, M.H. (1994). Human memory and cognition (2nd ed.). New York: HarperCollins. Aycock, A., & Buchignani, N. (1995). The e-mail murders: Reflections on ‘dead’ letters. In S.G. Jones (Ed.), Cybersociety (pp. 184-231). Thousand Oaks, CA: SAGE. Baker, L. (1979). Comprehension monitoring: Identifying and coping with text confusions. Journal of Reading Behavior, 11 (4), 365-374. Baker, L. (1984). Children’s effective use of multiple standards for evaluating their comprehension. Journal of Educational Psychology, 76 (4), 588-597. Belenky, M.F., Clinchy, B.M., Goldberger, N.R., & Tarule, J.M. (1986). Women’s ways of knowing: The development of self, voice, and mind. New York: Basic Books. Belli, R.F. (1989). Influences of misleading postevent information: Misinformation interference and acceptance. Journal of Experimental Psychology: General, 118 (1), 72-85. Beyer, B.K. (1985). Critical thinking: What is it? Social Education, 49 (4), 270-276. Bilal, D. (2000). Children’s use of the Yahooligans! web search engine: I. Cognitive, physical, and affective behaviors on fact-based search tasks. Journal of the American Society for Information Science, 51 (7), 646-665. Bird, S.E. (1996). CJ’s revenge: Media, folklore, and the cultural construction of AIDS. Critical Studies in Mass Communication, 13 (1), 44-58. Bloom, B.S., Engelhart, M.D., Furst, E.J., Hill, W.H., & Krathworhl, D.R. (1956). Taxonomy of educational objectives: The classification of educational goals. New York: David McKay. Bodenhausen, G.V., Kramer, G.P., & Susser, K. (1994). Happiness and stereotypic thinking in social judgment. Journal of Personality and Social Psychology, 66, 621-632. Brown, A.L., Bransford, J.D., Ferrara, R.A., & Campione, J.C. (1983). Learning, remembering, and understanding. In J.H. Flavell & E.M. Markman (Eds.), Handbook of child psychology (Vol. III): Cognitive Development (pp. 77-166). New York: John Wiley & Sons. Brown, A.L., & DeLoache, J.S. (1978). Skills, plans, and self-regulation. In R.S. Siegler (Ed.), Children’s thinking: What develops (pp. 3-35). Hillsdale, NJ: Lawrence Erlbaum. Carey, S. (1985). Are children fundamentally different kinds of thinkers and learners than adults? In S.F. Chipman & J.W. Segal (Eds.), Thinking and learning skills, (Vol. 2, pp. 485-517). Hillsdale, NJ: Lawrence Erlbaum. Corporation for Public Broadcasting. (2002). Connected to the future: A report on children’s Internet use from the Corporation for Public Broadcasting. Retrieved February 15, 2005, from the Corporation for Public Broadcasting site: http://www.cpb.org/ed/resources/connected/ Cromwell, L.S. (1992). Assessing critical thinking. In C.A. Barnes (Ed.), Critical thinking: Educational imperative (pp. 37-50). San Francisco: Jossey-Bass. D’Angelo, E. (1971). The teaching of critical thinking. Amsterdam: B.R. Gruner.

Skills: Fitzgerald

16

Dewey, J. (1933). How we think: A restatement of the relation of reflective thinking to the educative process (rev. ed.). Boston: D.C. Heath. Dresang, E.T. (1999). More research needed: Informal information-seeking behavior of youth on the Internet. Journal of the American Society for Information Science, 50, 1123-1124. Dunn, K. (2002). Assessing information literacy skills in the California State University: A progress report. Journal of Academic Librarianship, 28(1/2), 26-35. Eisenberg, M.B., & Small, R.V. (1993). Information-based education: An investigation of the nature and role of information attributes in education. Information Processing & Management, 29 (2), 263-275. Ennis, R.H. (1987). A taxonomy of critical thinking dispositions and abilities. In J.B. Baron & R.J. Sternberg (Eds.), Teaching thinking skills: Theory and practice (pp. 9-26). New York: W.H. Freeman. Ennis, R. (1989). Critical thinking and subject specificity: Clarification and needed research. Educational Researcher, 18 (3), 4-10. Fallows, D. (2005). Search engine users. Washington, DC: Pew Internet & American Life Project. Retrieved March 10, 2005, from http://www.pewinternet.org. Fitzgerald, M.A. (1999). Evaluating information: An information literacy challenge. School Library Media Research, 2. Retrieved March 23, 2005, from http://www.ala.org/aasl/SLMR/vol2/evaluating.html. Fitzgerald, M.A. (2000). The cognitive process of information evaluation in doctoral students: A collective case study. Journal of Education for Library and Information Science, 41 (3), 170-186. Fitzgerald, M.A., & Galloway, C. (2001). Relevance judging, evaluation, and decision making in virtual libraries: A descriptive study. Journal of the American Society for Information Science & Technology, 52 (12), 989-1010. Flanagin, A.J. & Metzger, M.J. (2000). Perceptions of Internet information credibility. Journalism & Mass Communication Quarterly, 77 (3), 515-540. Flavell, J.H. (1981). Cognitive monitoring. In W.P. Dickson (Ed.), Children’s oral communication skills (pp. 35-60). New York: Academic Press. Fogg, B.J., Soohoo, C., Danielson, D.R., Marable, L. Stanford, J., & Tauber, E.R. (2003). How do people evaluate the credibility of web sites? A study with over 2,500 participants. Conference on Designing for User Experiences, San Francisco. ACM Press. Fogg, B.J., Swani, P., Treinen, M., Marshall, J., Laraki, O., Osipovich, A., Varma, C., Fang, N., Paul, J., Rangnekar, A., & Shon, J. (2001). What makes web sites credible? A report on a large quantitative study. SIGCHI Conference on Human Factors in Computing Systems, Seattle, WA. ACM Press. Fogg, B.J. & Tseng, H. (1999, May). The elements of computer credibility. Paper presented at the Conference on Human Factors and Computing Systems, Pittsburgh. Freund, T., Kruglanski, A.W., & Shpitzajzen, A. (1985). The freezing and unfreezing of impressional primacy: Effects of the need for structure and the fear of invalidity. Personality and Social Psychology Bulletin, 11 (4), 479-487. Fritch, J.W. (2003). Heuristics, tools, and systems for evaluating Internet information: Helping users assess a tangled Web. Online Information Review, 27 (5), 321-327. Garbarino, E.C. & Edell, J.A. (1997). Cognitive effort, affect, and choice. Journal of Consumer Research, 24 (2), 147-158.

Skills: Fitzgerald

17

Garett, K. & Wulf, K. (1978). The relationship of a measure of critical thinking ability to personality variables and to indicators of academic achievement. Educational and Psychological Measurement, 38, 1181-1187. Garner, R. (1980). Monitoring of understanding: An investigation of good and poor readers’ awareness of induced miscomprehension of text. Journal of Reading Behavior, 12 (1), 55-63. Gilovich, T. (1991). How we know what isn’t so: The fallibility of human reason in everyday life. New York: Free Press. Glaser, E.M. (1985). Critical thinking: Educating for responsible citizenship in a democracy. National Forum: Phi Kappa Phi Journal, 65 (1), 24-27. Gross, M. (1999). Imposed queries in the school library media center: A descriptive study. Library and Information Science Research, 21 (4), 501-521. Jesdanun, A. (2004, December 13). More information but less truth? When access is so easy, answers can be elusive. Associated Press (MSNBC). Retrieved January 25, 2005, from http://www.msnbc.msn.com/id/6645957/ King, P.M., & Kitchener, K.S. (1994). Developing reflective judgment: Understanding and promoting intellectual growth and critical thinking in adolescents and adults. San Francisco: Jossey-Bass. Kruglanski, A.W., & Freund, T. (1983). The freezing and unfreezing of lay-inferences: Effects on impressional primacy, ethnic stereotyping, and numerical anchoring. Journal of Experimental Social Psychology, 19 (5), 448-468. Mackinnon, J.L.. (1987). Relations among patient management problems, critical thinking abilities and professional knowledge levels attained by physical therapy students (Doctoral dissertation, North Carolina State University, 1987). Dissertation Abstracts International, 48-06A, 1441. Markman, E.M. (1979). Realizing that you don’t understand: Elementary school children’s awareness of inconsistencies. Child Development, 50, 643-655. Markman, E.M. (1981). Comprehension monitoring. In W.P. Dickson (Ed.), Children’s oral communication skills (pp. 61-84). New York: Academic Press. Markman, E.M., & Gorin, L. (1981). Children’s ability to adjust their standards for evaluating comprehension. Journal of Educational Psychology, 73 (3), 320-325. McGregor, J.H. (1994). Cognitive processes and the use of information: A qualitative study of higher-order thinking skills used in the research process by students in a gifted program. In C.C. Kuhlthau (Ed.), School Library Media Annual (pp. 124-133). Englewood, CO: Libraries Unlimited. Metzger, M.J., Flanagin, A.J., Eyal, K., Lemus, D.R., & McCann, R.M. (2003). Credibility for the 21st century: Integrating perspectives on source, message, and media credibility in the contemporary media environment. In Kalbfleisch, P.J. (Ed.). Communication Yearbook 27 (pp. 293-335). Mahwah, NJ: Lawrence Erlbaum Associates. Miller, S.A., & Lipps, L. (1973). Extinction of conservation and transivity of weight. Journal of Experimental Child Psychology, 16 (3), 388-402. Moskowitz, D. & Stroh, P. (1996). Expectation-driven assessments of political candidates. Political Psychology, 17 (4), 695-712. Osherson, D.N., & Markman, E. (1975). Language and the ability to evaluate contradictions and tautologies. Cognition, 3 (3), 213-226.

Skills: Fitzgerald

18

Osman, M., & Hannafin, M. (1992). Metacognition research and theory: Analysis and implications for instructional design. Educational Technology Research & Development, 40 (2), 83-99. Paris, S.G., & Myers, M. (1981). Comprehension monitoring, memory, and study strategies of good and poor readers. Journal of Reading Behavior, 13 (1), 5-22. Partnership for 21st Century Skills. Learning for the 21st century: A report and mile guide for 21st century skills. Retrieved February 21, 2005, from http://www.21stcenturyskills.org/default.asp Paul, R. & Elder, L. (2001). Critical thinking: Tools for taking charge of your learning and your life. Upper Saddle River, NJ: Prentice Hall. Petty, R.E., & Cacioppo, J.T. (1986). The elaboration likelihood model of persuasion. In L. Berkowitz (Ed.), Advances in experimental social psychology (Vol. 19, pp. 123-205). Orlando, FL: Academic Press. Pitts, J.M. (1994). Personal understandings and mental models of information: A qualitative study of factors associated with the information-seeking and use of adolescents (Doctoral dissertation, Florida State University, 1994). Dissertation Abstracts International, 5501A, 4616. . Quarton, B. (2003). Research skills and the new undergraduate. Journal of Instructional Psychology, 30 (2), 120-124. Ross, L., Lepper, M.R., & Hubbard, M. (1975). Perseverance in self-perception and social perception: Biased attributional processes in the debriefing paradigm. Journal of Personality and Social Psychology, 32, 880-892. Siegel, M., & Carey, R.F. (1989). Critical thinking: A semiotic perspective. Bloomington, IN: ERIC Clearinghouse on Reading and Communication Skills. Simonson, I., & Nye, P. (1992). The effect of accountability on susceptibility to decision errors. Organizational Behavior and Human Decision Processes, 51 (3), 416-446. Stanford Persuasive Technology Lab. (2004, April). Stanford Web Credibility Research: (Mostly) Evidence-based Articles on Web Credibility. Retrieved March 22, 2005, from http://credibility.stanford.edu/credlit.html Stripling, B.K., & Pitts, J.M. (1988). Brainstorms and blueprints: Teaching library research as a thinking process. Englewood, CO: Libraries Unlimited. Svenson, O. (1981). Are we all less risky and more skillful than our fellow drivers? Acta Psychologica, 47 (2), 143-148. Taylor, R.S. (1986). Value-added processes in information systems. Norwood, NJ: Ablex. Tenopir, C. (2003). Use and users of electronic library resources: An overview and analysis of recent research studies. Washington, DC: Council on Library and Information Resources. Retrieved February 22, 2005, from www.clir.org/pubs/reports/pub120/pub120.pdf Tetlock, P.E. (1983). Accountability and the perseverance of first impressions. Social Psychology Quarterly, 46 (4), 285-292. Wathen, C.N. & Burkell, J. (2002). Believe it or not: Factors influencing credibility on the Web. Journal of the American Society for Information Science, 53 (2), 134-144. Weisburg, H.K., & Toor, R. (1994). Learning, linking and critical thinking. Berkeley Heights, NJ: Library Learning Resources. Yinger, R.J. (1980). Can we really teach them to think? In R.E. Young (Ed.), Fostering critical thinking (pp. 11-31). San Francisco: Jossey-Bass.

Skills: Fitzgerald

19

Skills: Fitzgerald

20