Trust, Learning, And Usability

Trust, Learning, and Usability Laurian C. Vega, Yeong-Tay Sun, D. Scott McCrickard Center for Human-Computer Interaction, Virginia Tech 2202 Kraft Drive Blacksburg, Virginia 24060 {Laurian, YTSun, McCrickS}@vt.edu by defining the domain we are examining: formal instruction.

STATEMENT OF TOPIC

In this poster presentation we will introduce the findings from a study that evaluated trust and its effects on usability and learning within online information repositories.

Education is a largely social endeavor guided by instructors and tools; lectures are given by individuals, books are written by individuals. Learning is an in situ activity that is guided by participation with the group. Therefore, trust is a social value that must be encapsulated in the design of educational tools like software. It is for this reason that Jones et al. [8] have cited trust as a key factor that must be accounted for in the continuing development of e-Learning.

SUMMARY

Designing software and online tools so that students find them not only usable, but also credible to learn from is a growing issue. In this poster presentation we will introduce the findings from a study that evaluated trust and its effects on usability and learning within online information repositories. We investigated what design element was being trusted by the e-learner and what aspects of usability (e.g., errors) influenced trust. Using a novel method of evaluation, we will present preliminary findings that lead to a model of trust and implications for designing e-Learning tools.

Approach & Uniqueness

To begin probing these questions we ran an exploratory study of twenty participants using two different e-Learning websites: the Usability Case Study Library (UCS), and Wikipedia. The UCS is an online repository of software design materials. Wikipedia is an online encyclopedic repository of user provided content. The participants spent twenty minutes within each website to complete four tasks.

SIGNIFICANCE AND RELEVANCE Problem & Motivation

To measure usability, our usability goals were efficiency, learnability, utility, and effectiveness. We measured the number of errors encountered, subjective beliefs, and the time users spent on each task.

As learning continues to moves from the classroom to the Internet, concerns over how students use and evaluate online materials becomes increasingly important [1]. Indeed, the assessment of credible online resources becomes a priority for evaluating and designing webmaterials.

To measure trust, we employed a unique method. Every time users clicked on the website, and if the click was at least thirty seconds apart from the previous click, a pop-up appeared asking how much they trusted the websites as a whole, the individual webpages, and the information they were using.

Prior work has suggested that the usability of a system affects how people will learn [2]. Also supported is the idea that students’ trust of online materials will affect their use [3, 4]. However, what has been minimally researched is the effect of usability on trust in e-Learning. We question, what is a user trusting in e-Learning situations? Is the learner trusting the creator of the information, as suggested by [5]? Or, is the learner trusting the online materials, as suggested by [6]? Would the trust in one affect the other? What effect does usability have when trust is examined under this lens?

Results & Contributions

These results are still preliminary; however, they are encouraging. In examining subjective measures of usability and using a Fit Least Squares Effect Test, there were a number of findings. The relationship between the UCS website as a whole and usability show that website instructions and feedback had a positive effect on trust. Frustration and the overall feel of the website had a negative effect on trust for the UCS. Also having a negative effect on the UCS was the belief that operations completed on the website related to the task at hand. Focusing on these aspects of usability can help when designing a trustworthy e-Learning website.

Background & Related Work

In a recent search for related work on trust and online materials, there were over two thousand results from peerreviewed publications. While most of these publications concentrate on the topics of trust and e-Commerce and how to increase trust in the buyer, the fact that trust has been researched and evaluated for the past fifty years is apparent. Therefore, we have limited out view of trust in this research 1

As illustrated by looking at Figure 1, the data is very rich and tells an interesting story. REFERENCES

1. Rogers, D. and K. Swan, Self-regulated learning and internet searching. Teachers College Record, 2004. 106(9): p. 1804-1824. 2. Rosson, M.B. and J.M. Carroll, Usability Engineering: Scenario-Based Development of Human-Computer Interaction. 2002, San Francisco: Morgan Kaufmann Publishers. Figure 1. One participant’s responses to trusting the entire website, a single webpage, or the information to guide them across one twenty minute interval.

Wikipedia is statistically trusted more then the UCS (UCS Trust = 495, Wikipedia Trust = 617, p= 1.1E-35). This finding holds across each individual question. Next, if we break this average down into its separate questions, we can see a trend: users trust the entire site more then they trust an individual page within that site. Second, users trust that the page they are on will lead them to their task answer less then both of the previous questions. (Correlation= .959). For a software designer, focusing on the interaction of the whole website can be a better place to start when designing for trust. In examining error rates and their effects on trust, using a Pearson’s ChiSquare test we found that usability errors had a detrimental effect on all measured aspects of trust (p<.0001, df=1). After encountering an error a users trust would decrease. One interesting interaction effect to be examined is the hypothesis that when trust levels are on average lower, they are more likely to be affected by interaction effects like usability. When trust levels are higher, usability specifically bad usability - may not cause such a large effect.

3. Fogg, B.J., et al. What makes web sites credible? A report on a large quantitative study. in Conference on Human Factors in Computing Systems CHI 2001 Anyone. Anywhere, Mar 31-Apr 5 2001. 2001. 4. Tschannen-Moran, M. and W.K. Hoy, A Multidisciplinary Analysis of the Nature, Meaning, and Measurement of Trust. Review of Educational Research, 2000. 70(4): p. 547-93. 5. Friedman, B., P.H. Kahn, Jr., and D.C. Howe, Trust Online, in Communications of the ACM. 2000. p. 34-40. 6. Nass, C.I., et al., Computers are Social Actors: A Review of Current Research, in Human Values and the Design of Computer Technology, B. Friedman, Editor. 1997, Cambridge University Press: Cambridge. p. 137162. 7. Jones, C., L. Dirckinck-Holmfeld, and B. Lindstrom, A relational, indirect, meso-level approach to CSCL design in the next decade. International Journal of Computer-Supported Collaborative Learning, 2006. 1(1): p. 35-56. 8. Rogers, Y., H. Sharp, and J. Preece, Interaction Design. 2007, Chichester, West Sussex: John Wiley & Sons Ltd.