Julie Greg Blog Handout

Julie Coiro, 9/11/06

New Literacies of the Internet

So, where are we headed? Supporting Students’ Critical Evaluation of Online Information NERA Conference Institute 2008 Julie Coiro University of Rhode Island [email protected]

Greg McVerry University of Connecticut [email protected]

Talking about aspects of critical evaluation with students Relevancy: the information’s level of importance to a particular reading purpose or stated information need Accuracy: the extent to which information contains factual and updated details that can be verified by consulting alternative and/or primary sources Reliability: the information’s level of trustworthiness based on information about the author and the publishing body [most and least reliable] Bias (Perspective): the position or slant toward which an author shapes information Commercial bias: the extent to which information appeared to be influenced by commercial interests for or against a certain product.

Findings from Recent Research: Study #2 Coiro, 2007

Review findings from recent research about how readers respond to different aspects of critical evaluation prompted by Internet texts and tasks Introduce a series of strategy lessons that address the challenges of online critical evaluation Evaluating relevancy Evaluating accuracy Evaluating reliability Evaluating bias/point of view Share and discuss in our follow up session For specific examples, visit http://www.lite.iwarp.com/CoiroVT2008.html

Findings from Recent Research - Study #1 (Leu et al, 2007)

• 87.5% (42 out of 48) reported the site was reliable; • Slightly more than half of our students (27 out of 48) reported it to be very reliable • 6 out of 48 reported the site was unreliable (just participated in a lesson that taught this site, showing students to be suspicious of information online)

Evaluating Accuracy of Online Information

Online Reading Comprehension Assessment - Scenario I (ORCA-1)

Total N=109

Only 20% have strategies for evaluating accuracy

3 = .02% N=2

2 = 19% N=21

1 = 26% N=28

0 = 54% N=58

I checked this information with www.____.com and they compared similarly. [checked with 2nd reliable source] I know this is accurate because I learned it in science class. [compared with prior knowledge] I know this is accurate because it’s made by a corporation and there is a place to contact them. [implicit trust] It seems right but you can never know; The website I think is always right; It had plenty of pictures; I checked it out with Ask Jeeves; Why would they lie? [misconceptions]

80% do not know how to evaluate accuracy or did not locate the page.

1

Julie Coiro, 9/11/06

Evaluating Least Reliable Source

New Literacies of the Internet

Total N=109

Evaluating Most Reliable Source

Only 32% correctly identified the least reliable source

3 = 14% N=15

2 = 18% N=20

1 = 21% N=23

0 = 47% N=51

There are a lot of misspellings, there is no way to contact the people who put out this site, and it’s a .com compared to a .org or .gov [at least one critical attempt] It’s a .com and they are trying to sell you something at the very top of the page. [surface procedures]

This is the least reliable source because the text is a little confusing to read. [readability, size of page, etc.]

3 = 13% N=14

2 = 16% N=18 N=37

0 = 37% N=40

This is most reliable because it is made by doctors from the American Lung Association [critically consider source] There are no spelling mistakes and the url is a .org. [surface procedures] I knew more about carbon monoxide than I knew from reading all of the other pages [only relevancy or interest] It’s really detailed and it has like 10 paragraphs of information. [readability, size of page, etc.]

71% considered only relevancy, text length, or did not know.

68% considered only relevancy or interest or did not know.

Evaluating Commercial Bias

Only 29% correctly identified the most reliable source

1 = 34%

It doesn’t really talk about anything specific - there’s not a lot of information about the topic [only relevancy or interest]

Total N=109

Total N=109

Overall, findings suggest…

Only 30% see reason to caution information with advertising

3 = 13% N=14

2 = 17% N=20

1 = 17% N=28

0 = 43% N=47

This site is probably one sided info - the information might be inaccurate or exaggerated [balanced consideration of bias] If a website is trying to sell you something then it is a bad source and doesn’t have good information [some suspicion or overgeneralization] I think websites that try to sell you things just want to get a lot of money because the Internet is really expensive [caution, but not in relation to the information] It makes me think they must really know what they’re talking about; I think that what they are telling you about those things is really true [ads make the information better]

Only 20-32% of these 7th graders (N=109) had at least one effective strategy for evaluating accuracy, reliability, or potential bias of online information That means 68-80% of students either did not have effective strategies for evaluating online information (35-45%) or could not even locate the information in the first place (33-35%).

71% are mildly cautious of the ads [not the info] or think ads make it better!

Critically Evaluating Information

Challenge 1: Evaluating Relevancy Reading search engine results

Evaluating relevancy: Does it meet my needs? Evaluating accuracy: Can I verify it with another reliable source? Evaluating reliability: Can I trust it? Evaluating bias: How does the author shape it?

So, how might we begin to teach students how to answer these questions…

Modeling effective online reading/skimming strategies: What clues do the words after the link give me? Are the results in any special order? Who sponsors the site? What’s missing from this list? Activity:

2

Julie Coiro, 9/11/06

New Literacies of the Internet

Challenge 1: Evaluating Relevancy

Challenge 2: Evaluating Relevancy -

Reading search engine results

Previewing a website

Question/Answer

How do you know? Why is it important to know?

Which site features information about heiroglyphics? Site 1

I read it in the description after the link.

The description after the link gives clues to what the site contains - it’s faster.

Who sponsors Site 3? California school

The URL tells me with http://…ca.edu

Knowing who sponsors the site might give clues about the author’s lens/stance

Modeling effective online previewing strategies STOP and THINK! Stop Preview left menu and top menu bars Anticipate where each link will lead Anticipate multiple levels (closer or further) Explore interactive mouseover functions Note the author/webmaster Understand website search features

&Think

Challenge 2: Evaluating Relevancy -

Challenge 3: Evaluating Accuracy

Previewing a website

Reading to verify online information

Activity Imagine... you want to get more involved with community service and network with other youth volunteers around the country. You will have one minute to preview each of the websites using the strategies above and rank each website according to its relevancy to your needs (0=lowest, 3=highest). Provide details to support your decision. Then, identify two parts of the “best” website that you believe would be most relevant for your needs. Site 1

Site 2

Site 3

Site 4

Model effective online verification strategies STOP and THINK! Cross check factual data with at least three other sources Ask probing questions: What claims is the author making? What evidence do I find elsewhere to support those claims? What evidence do I find elsewhere to refute those claims? Consider the context of where you find the evidence The challenge of confirming vs. disconfirming -- Adapted from IMSA 21st Century Information Fluency Project (2007)

Challenge 3: Evaluating Accuracy -

Challenge 4: Evaluating Reliability -

Reading To Verify Online Information

Investigating an author’s credentials

Activity Imagine you are selecting websites to use as part of your online research for your report on a strange animal. Apply the above questioning strategies to the websites below to verify or refute the accuracy of each author's claims. Be prepared to provide evidence for your decisions. • Rock Nest Monster • The Elephant Art Gallery (Related Video) • Armed and Dangerous Dolphins

Model effective online critical reading strategies: Understand the relationship between the information and the author/source to evaluate reliability Activity (Complete at least two websites) Locate the "About Us" link on each website below. Complete the activity in your handout by following these steps: (a) identify the name of hyperlink that led you to information about the website's creator; (b) tell one thing you learned about the author's background and level of expertise; (c) search off the website for the author's name and tell one other thing you learned about the author; (d) is there anything about the site that appears to increase or decrease the reliability of the information found there? (please explain your hunch) (e) rate each author's level of expertise from 1 (lowest) to 5 (highest) and be prepared to justify your decision with evidence.

3

Julie Coiro, 9/11/06

New Literacies of the Internet

Challenge 5: Integrating strategies for evaluating accuracy & reliability Activity 1. Work with your group to brainstorm strategies for critically evaluating the information at a website. Create a list of these ideas in your handout. 2. See if you can determine if each of these sites is real or not real (there are some of both). Avoid relying only on your prior knowledge - since often your students will not have this knowledge to rely on or it is inaccurate.

REAL = accurate AND reliable

If you determined that the information is real, what evidence do you have to prove it? If you think it is a hoax, how do you know for sure?

Challenge 6: Separating fact from opinion and author’s point of view Model effective online critical reading strategies: Detecting Fact Versus Opinion Detecting Bias in Relation to Author’s Affiliation Determining Author’s Purpose in Relation to Affiliation

3. Share with the larger group the strategies that helped you determine the quality of information at each website. Add to your own list any new strategies you learned from someone else in the group.

Challenge 6: Separating fact from opinion and author’s point of view Activity Explore each website related to the Iditarod dog sled races as you complete the three tasks described below... * Ray Redington's Dog Care * Racing for the Grave * Scholastic's Is the Iditarod for the Dogs? Level 1: Detecting Fact Versus Opinion Tell which website you think has the STRONGEST opinions about the use of sled dogs in the Iditarod. Tell whether you think the author of the website you chose is for or against racing sled dogs for competition. Select a quote from the website you chose and explain why you think it is an example of the author sharing strong opinions.

Challenge 7: Integrating strategies for evaluating accuracy, reliability & bias Developing an overall healthy skepticism Pulling it all together: Investigate one or more of the websites below while considering the answers to the following questions: * What is the purpose of this site? * Who created the information at this site? * When was the information at this site last updated? * Where can I go to check the accuracy of this information? * Why did this person, or group, put this information on the Internet? * Is there anyone that might be offended or hurt by the information at this site? * Does the website present only one side of the issue, or are multiple perspectives provided? * How is the information at this site shaped by the stance taken by the creator of the site?

Athletes.com or Martin Luther King.org

Challenge 6: Separating fact from opinion and author’s point of view Level 2: Detecting Bias and Considering the Author's Affiliation Tell which website (Site A, B, or C) gives opinions from more than one side of the issue. Who are the two people whose opinions are given in the website you chose in number 1? What factors make these two people feel the way they do about the treatment of sled dogs ? Level 3. Determining Author's Purpose in Relation to Affiliation Identify the general and specific purpose(s) of each site and provide at least two reasons to support your answer. For example - detecting evidence of bias: • Does the website provide factual information? • …try to persuade you to think or feel a certain way? • …try to sell you something? • …try to raise money or collect donations?

To sum up… Review findings from recent research Introduce a series of strategy lessons for supporting online critical evaluation Evaluating relevancy Evaluating accuracy Evaluating reliability Evaluating bias/point of view Share and discuss in small-group follow up session For specific examples, visit http://www.lite.iwarp.com/CoiroVT2008.html Thank you!

4