Robles Final Test Construction




FINAL EXAM Richard Robles 18-EDST-633 December 11, 2008

Richard Robles, Page 2 FINAL TAKE HOME EXAM- 18-EDST-633 Answer each question on a separate sheet of paper. Responses should be typed in 12point font, single-spaced with one-inch margins in the form of short essays (i.e., no more than one page of text per answer). Because this is a take-home exam, you may use any resources available to you, including but not limited to your textbook, lecture notes, Internet; however, you must work independently as an individual. External sources must be cited appropriately (e.g., MLA, APA) to avoid plagiarism. 1.

Throughout the textbook, Stiggins connects the chapters to the concept of assessment FOR learning. Using the four guiding principles of reliable assessment and the power of student involvement, explain how assessment FOR learning is applied to your position as a teacher or administrator. Use direct examples of your experience to further illustrate your points. (25 points)

2.

Compare and contrast the use of portfolios and standardized tests as communicative forms of assessment? What common qualities need to be present for both to portray an assessment FOR learning? (15 points)

3.

Exhibit A provides background information on the University of Cincinnati’s first-year experience program. In evaluating the program’s effectiveness, a committee of faculty and administrators drafted a set of targeted learnings/standards (beginning on page 2, line 19). (40 points total) a. Deconstruct the Intellectual and Self-Management Skills standards (beginning on page 2, line 32) into student-friendly terms. (15 points) b. A college success skills class was designed to address the academic and taskmanagement skills on page 2, lines 35-38 (critical thinking, ethical behavior, goal setting, information literacy, listening, note-taking, problem-solving, project management, reading, self-assessment, self-reflection, study, test-taking, time management, writing, and use of technology). What kind of disposition assessment method would you use? (10 points) c. Considering these standards in student-friendly terms, identify and explain which one of the four types of portfolios would be most effective in presenting a student’s progress and/or accomplishment. (15 points)

Richard Robles, Page 3 4.

Exhibit B is a scoring rubric for speech presentations from the Schools of California Online Resources for Education**. (20 points total) a. Using the five attributes of good rubrics, evaluate the effectiveness of this rubric. Assume the appropriate and relevant achievement standards outlined in Exhibit A. What recommendations would you offer for improvement? (10 points) b. To promote the power of student involvement, what strategies would you employ to garner their participation and bring them to a deep level of assessment? (10 points)

























































 **
http://score.rims.k12.ca.us/sub_standards/alt_assessment_res_on_web.html


UC|Great Beginnings

1 2 3 4 5 6 7 8

PURPOSE UC|Great Beginnings is the integrated first-year experience curriculum at the University of Cincinnati (UC). The first-year curriculum is part of the very fabric of the university; it promotes the achievement of academic excellence (UC/21 Goal 3) and places students at the center (UC/21 Goal 1). UC|Great Beginnings emphasizes that higher learning extends beyond the walls of the university classroom to incorporate the greater community and workplace (UC/21 Goal 4) while concurrently providing students with a sense of belonging and connectedness to the university (UC/21 Goal 5), its mission, and its vision for the future.

9 10 11 12 13 14

The aims of the program are (1) to help students acquire intellectual and self-management skills needed for success in the university, as well as in life-long learning; (2) to get students engaged with the cultural life and diversity of this large, urban, research university; (3) to focus students’ attention on what their professional and civic responsibilities to the world might be as educated persons; and (4) to advance students’ capacity to make connections between knowledge gained from multiple sources and to apply their increasingly comprehensive understanding to new questions and situations.

15 16 17 18

Opportunities for student learning embedded in UC|Great Beginnings is achieved and reinforced through student participation in multiple academic and co-curricular experiences that address one or more of the program’s aims. Over the course of their first year of enrollment all matriculated students will experience a similar and comprehensive, cohesive, and well-integrated introduction to higher education at the University of Cincinnati.

19 20 21 22 23 24 25 26 27 28 29

EXPECTED OUTCOMES     

Increased student engagement, learning, and achievement in the first year of college and beyond especially as it relates to General Education. Progress toward increased graduation rates as measured by increased retention from the first to second years of college, a preliminary indicator of success. Clearly articulated and structured first-year experience requirements across all degree programs. Clearly articulated and structured first-year experience co-curricular opportunities. Creation of an identifiable and active network of faculty and staff who share with one another their knowledge and practice surrounding the first college year including scholarly activity related to teaching and learning.

STRUCTURE

30 31 32 33 34 35 36

UC|Great Beginnings is intended to be the cornerstone upon which each student’s university learning experience is built. Therefore, its components should be designed to reflect UC’s dynamic environment for undergraduate learning and introduce first-year students to the types of learning activities and level of accomplishment that graduating students are expected to demonstrate. For example, courses and experiences that use active, experiential, or community-based pedagogies—and, that incorporate purposeful and on-going reflection as an integral part of their design are desirable as these are signature features of UC’s undergraduate experience. Likewise, academic rigor must be maintained to protect the integrity of degree programs.

37 38 39 40

Although all students will be assured of participation in a first year experience, no particular structure for delivering UC|Great Beginnings is prescribed to divisions, colleges, departments, or units. Rather, these bodies will develop programs that, while incorporating the UC|Great Beginnings aims and objectives, are tailored to the unique needs of their students.

41 42

Structures that have proven useful and may serve as models include: freshman seminars—small courses designed to introduce collegiate level expectations and resources to support student learning; learning communities—sets 1 Revised 5/23/06

1 2 3 4 5 6 7

of courses in which students are co-enrolled and through which content may be linked; introductory-level academic or disciplinary courses that are well integrated with other aspects of the first-year experience or with other aspects of the undergraduate curriculum; courses and activities that foster out-of-class interaction between students, their faculty, and other professional mentors; required regularly-scheduled academic advising; the use of undergraduate teaching assistants or peer mentors; first-year student retreats and conferences, clubs and organizations, and other similar programs that foster student participation and leadership in university life. EVALUATION

8 9 10 11

UC|Great Beginnings is intended to be dynamic and responsive to student and university needs with even its aims and objectives being changeable. Multiple and varied assessments to improve UC|Great Beginnings and to confirm its effectiveness for eliciting desired processes and outcomes must be a core and regular part of its design and implementation, which follows an on-going continuous improvement model.

12 13 14

Each unit will evaluate its own UC|Great Beginnings content and structures. Units will use their data in conjunction with university-wide data to make ongoing enhancements to improve performance. (See appendix for possible examples).

15 16 17 18

University-wide assessments will include analysis of aggregate data reflecting such issues as student achievement, retention, and graduation; student engagement as measured by instruments such as the National Survey for Student Engagement (NSSE), and other comprehensive measures of outcomes including contributions toward achievement of UC|21 goals.

19

TARGETED STUDENT LEARNINGS

20 21 22 23 24 25 26 27 28 29 30 31

INTEGRATIVE LEARNING To advance students’ capacity to make connections between knowledge gained from multiple sources and experiences and to apply their increasingly comprehensive understanding to new questions and situations, students will:  Learn how to recognize academic and professional disciplines and their interrelationships.  Gain some appreciation for how knowledge is created and shared by learning and participating in specific methods for developing and disseminating knowledge appropriate to their field of study.  Begin learning how to utilize diverse and even contradictory points of view.  Begin to apply previously acquired skills and knowledge to practice in new and various settings.  Recognize how various courses and learning experiences contribute to mastery of the general education baccalaureate competencies: effective communication, critical thinking, social responsibility, and knowledge integration.

32 33 34 35 36 37 38 39 40 41 42 43 44 45

INTELLECTUAL & SELF-MANAGEMENT SKILLS To help students acquire intellectual and self-management skills needed for success in the university as well as in life-long learning, students will:  Gain some proficiency in academic and task-management skills including: critical thinking, ethical behavior, goal setting, information literacy, listening, note-taking, problem-solving, project management, reading, self-assessment, self-reflection, study, test-taking, time management, writing, and use of technology.  Gain some personal responsibility skills for setting academic and career development goals as well as for creating and instituting plans to achieve those goals.  Learn strategies to successfully manage change and adapt to new situations.  Gain some understanding of issues related to personal health and exhibit behaviors that indicate attention to personal well-being.  Acquire skills to effectively manage competing priorities like work, family life, and study by learning how to spend their time and energy in ways that support a balanced lifestyle. 2 Revised 5/23/06

1 2



Learn how to form effective relationships, learning partnerships, and teams with faculty, staff, and other students to learn about and engage in educational opportunities and university life.

3 4 5 6 7 8 9 10 11 12 13

PROFESSIONAL AND CIVIC RESPONSIBILITY To focus students’ attention on what their professional and civic responsibilities to the world might be as educated persons, students will:  Gain some understanding regarding the consequences that their actions have upon the community and incorporate learning from mistakes and building on successes as they learn to take responsibility for their behaviors.  Learn concepts of professional and civic responsibility to improve the world and the human condition.  Learn professional and practical ethical and behavioral skills to hold themselves accountable as well as assertiveness skills for voicing ethical concerns to others.  Exhibit behaviors that indicate they have some understanding of cultural competence and welcome opportunities for cross-cultural interaction and communication.

14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29

UNIVERSITY ENGAGEMENT To get students engaged with the cultural life and diversity of this large urban research university, students will:  Know how to find out about university services and practices and how to reach service representatives for assistance.  Stay abreast of current university policies and procedures as well as be familiar with campus facilities, academic and student service offices, governance structures, and other resources.  Participate in university sponsored co-curricular activities outside of class requirements including lectures, workshops, plays, concerts, student organizations, etc.  Have opportunities to participate in classroom and co-curricular activities with students from different nationalities, ethnic, religious, and cultural groups.  Extend to those they meet the same respect, cooperation, and caring they expect from others.  Work to build a learning environment that offers everyone an equal opportunity to grow, flourish, and contribute.  Have opportunities to take courses and participate in activities from colleges and academic departments outside of their primary program of study.

3 Revised 5/23/06

1 APPENDIX

BIBLIOGRAPHY Astin, A. (1993). What matters in college? Four critical years revisited. San Francisco, CA: Jossey-Bass. Donahue, L. (2004). Connections and reflections: Creating a positive learning environment for first-year students. Journal of the First-Year Experience, 16(1), 77-100. Elmore, D. E., Prentice, J. C., & Trosset, C. (2006). Do students understand liberal arts disciplines? Liberal Education, 92(1), 48-55. Gardner, J. N. (2006, February 28). The First-Year Experience’s First 25 Years—The Jury is In, But Out on the Next: You Can Determine the Verdict. Plenary address presented at the 25th Anniversary First-Year Experience Conference and Gala Celebration, Atlanta, GA. Retrieved May 22, 2006, from http://www.sc.edu/fye/events/presentation/FYEAnnualConf06PlenarySpeech.pdf Gardner, J. N. et al. (2006). Foundational dimensions, foundations of excellence in the first college year. Retrieved May 21, 2006 from http://www.fyfoundations.org/foundationalDimensions.aspx Huber, M. T., & Hutchings, P. (2004). Integrative Learning: Mapping the Terrain. Washington, DC: Association of American Colleges and Universities and The Carnegie Foundation for the Advancement of Teaching. Kuh, G.D. (2001). The National Survey of Student Engagement: Conceptual Framework and Overview of Psychometric Properties. Bloomington, IN: Indiana University Center for Postsecondary Research. Retrieved May 21, 2006 from http://nsse.iub.edu/pdf/conceptual_framework_2003.pdf Kuh, G. D., Kinzie, J., Schuh, J. H., & Whitt, E. (2005). Student success in college: Creating conditions that matter. San Francisco: Jossey-Bass. Lattuca, L. R., Voight, L. J., & Fath, K. Q. (2004). Does interdisciplinarity promote learning? Theoretical support and researchable questions. The Review of Higher Education, 28(1), 23-48. MacGregor, J. (2003). Integrating learning communities with service-learning. Olympia, WA: The Evergreen State College, Washington Center for Improving the Quality of Undergraduate Education, in cooperation with the American Association for Higher Education. Nathan, R. (2005). My freshman year: What a professor learned by becoming a student. Ithaca, NY: Cornell University Press. National Panel Report. (2002). Greater expectations: A new vision for learning as a nation goes to college. Washington, DC: Association of American Colleges and Universities. Retrieved May 21, 2006 from http://www.greaterexpectations.org/pdf/GEX.FINAL.pdf. National Resource Center for the First-Year Experience and Students in Transition. First-year resources. Retrieved May 21, 2006 from http://www.sc.edu/fye/resources/fyr/index.html Oconner, J., & with others. (2003). Learning communities in research universities. National learning communities project monograph series. Olympia, WA: The Evergreen State College, Washington Center for Improving the Quality of Undergraduate Education. 4 Revised 5/23/06

Pedersen, S. (2003). Learning communities and the academic library. National learning communities project monograph series. Olympia, WA: The Evergreen State College, Washington Center for Improving the Quality of Undergraduate Education in cooperation with the American Association for Higher Education and the Association of College and Research Libraries. Peters, J. R., & Stearns, D. E. (2003). Bringing educational relevancy to the first-year college experience by bearing witness to social problems. The Journal of Experiential Education, 25(3), 332-342. Skipper T. L., Argo R. (Eds.). (2003). Involvement in campus activities and the retention of first-year college students (monograph no.36) . Columbia, SC: University of South Carolina, National Resource Center for The First-Year Experience and Students in Transition. Swing, R. L. (2004). Introduction to first-year assessment. In R. L. Swing (Ed.), Proving and improving, volume II: Tools and techniques for assessing the first college year (monograph no. 37) (pp. xiii-xxvi). Columbia, SC: University of South Carolina, National Resource Center for The First-Year Experience and Students in Transition. Swing, R. L. (Ed.). (2004). Proving and improving, volume II: Tools and techniques for assessing the first college year (monograph no. 37). Columbia, SC: University of South Carolina, National Resource Enter for The First-Year Experience and Students in Transition. Taylor, K., with, Moore, W. S., MacGregor, J., & Lindblad, J. (2003). Learning community research and assessment: What we know now. Olympia, WA: The Evergreen State College, Washington Center for Improving the Quality of Undergraduate Education, in cooperation with he American Association for Higher Education. Tinto, V. (1997). Classrooms as communities: Exploring the educational character of student persistence. The Journal of Higher Education, 68, 599-623. Tobolowsky, B. F. (2005). The 2003 national survey on first-year seminars: Continuing innovations in the collegiate curriculum (monograph no. 41). Columbia, SC: University of South Carolina, National Resource Center for FirstYear Experience and Students in Transition.

Walker, A. A. (2003). Learning communities and their effect on students' cognitive abilities. Journal of the First-Year Experience, 15(2), 11-33.

5 Revised 5/23/06

Scoring Rubric for Speech

http://score.rims.k12.ca.us/sub_standards/scoring_rubric_for_sp...

Scoring Rubric for Speech Visual Organization | Analysis/ Coherence | Originality | Delivery 1: Novice

2: Beginning

3: Proficient

Visual Organization

Opening

Conclusion

Top No clear opening. Audience is unsure what the and/or theme is. No Basis for audience to predict or follow.

No clear conclusion. The speech just seems to stop.

Does not supply any words and/or phrases that link Transition Words/ ideas, such as Verbal Road Signs sequence markers, logial connectors, ect..., throughout.

Opening only orients the audience to the topic or theme but not both. Audience has some sense of what will follow.

Clear opening that orients audience to topic and theme but is not engaging. Audience has a good sense of what will follow. There is some Clear restatement restatement of the of main theme and main theme and ajor supports. major support. Audience is asked to action or belief. Supplies a few words and/or phrases that link ideas, such as sequence markers, logical connectors, ect..., throughout.

Inconsistently supplies words and/or phrases that link ideas, such as sequence markers, logical connectors, ect..., throughout.

Analysis/ Coherence

Logic

1 of 3

4: Expert

Engaging opening that orients the audience to the topic and theme. Audience knows what will follow.

Creative restatement of main theme and major supports. Strong call to action or beliefs. Supplies words and/or phrases that link ideas, such as sequence arkers, logical connectors. "As I said earlier..." ect..., throughout.

Top Does not connect ideas logically and makes few or no logical connection between statements.

Connects some ideas logically and creates a general sense of cumulatve statement.

Connects most ideas logically and builds a cummulative statement.

Connects all ideas logically and builds a compelling, cummulative statement.

12/11/08 3:23 AM

Scoring Rubric for Speech

Focus on Topic

Vocabulary

http://score.rims.k12.ca.us/sub_standards/scoring_rubric_for_sp...

Seems to stray Focuses on topic from topic and off with some major into side issues. tangents.

Uses vague and weak langage and/or uses language that violates school honor code.

Uses occational strong, accurate language that is appropriate to the theme of the audience.

Focuses on topicwith few major tangents.

Focuses tightly on topic and everything revolves about theme. Uses strong Uses precise, accurate language evocative language throughout that is throughout hat is appropriate to appropriate to th theme and theme and audience. audience.

Originality

Topic Choice

Top Chose uninspired topic that he/she does not seem to care about or chose a topic that violates school honor code.

Chose a hackneyed or cliched topic or topic that is inapproperiate for setting, time and audience.

Chose a traditional but approperiate topic hat speaker seems invested in.

Delivery

Eye Contact

Voice

Body Language

2 of 3

Chose novel topic or gives new or unusual perepective on old theme and seems enthusiastic.

Top Does not make eye Makes some eye Makes varied contact with the contact with some inconsistent eye audience. of the audience. contact with the entire audience. Speaker speaks in Uses some Uses much a monotone or a variance in pitch, variance in pitch, voice that is clearly rate, and volume to rate, and volume to uncontrolled. appropriately appropriately enhance their convey their meaning. Practice meaning. Exhibits and control are some practive and evident. control. Uses no body language that inappropriately reinforces his/her meaning.

Makes varied and substained eye contact with the entire audience. Uses variance in pitch, rate, and volume to appropriately convey their meaning throughout. Exhibits practiced purposeful control. Speaker uses some Speaker uses much Speaker uses body body language that body language that language that appropriately appropriately appropiately enhances their enhances their enhances their meaning. meaning. meaning throughout speech.

12/11/08 3:23 AM

Scoring Rubric for Speech

Frequently user filler words or phrases, or pauses Filler Words ("uh") inappropriately, pauses as if at a loss.

http://score.rims.k12.ca.us/sub_standards/scoring_rubric_for_sp...

Smooth flowing language, with occational filler words or verbal tics.

Smooth flowing language, with one or two filler words or verbal tics.

Smooth flowing language, totally devoid of any filler words or verbal tics.

Visual Organization | Analysis/ Coherence | Originality | Delivery | Top

3 of 3

12/11/08 3:23 AM

Richard Robles 18-EDST-633 December 10, 2008 Final Justification This final assumes that the students taking this final are graduate students with some work experience in an instructional and/or administrative role at a school. I felt the essay responses would be appropriate to evaluate the student’s application of the material and reasoning. •

The first question was meant to determine understanding of the guiding principles of assessment. Although Stiggins repeatedly went back to the idea of assessment FOR learning, the question probed how does the student identifies with the concept and where does s/he prioritize its value in her/his work philosophy.

•

Question two was attempting to ask the student to evaluate portfolios and standardized tests as a form of communication and reason how the two are common. Within higher education these two forms of assessment are becoming popular for a variety of reasons. The research project I am working on through the UC Provost’s Office is comparing the two as the basis for value-added learning.

•

Question three is a meant to gauge mastery of knowledge and understanding. This is a real situation where I am participating in an on-going discussion to devise an assessment plan for the first-year experience at the University. The questions posed were based on discussion strings in the overall conversation. This provides the student an example of how the concepts learned in class could be applied a realto-life scenario.

•

Question four is an example of a national conversation to standardize general education requirements and liberal arts education. Rubrics and checklists are becoming more commonly used to standardize performance and essay assessment.

Reflection With my background primarily in higher education, I was challenged to find connections between the concepts presented in the class with my career as a University administrator and instructor. Looking back at my first reflection, I still believe that my definition of assessment and its importance in higher education still rings true. My student affairs background approaches assessment from the idea of using surveys and first-order statistics to get a picture of what the students are saying. This course has provided insight to expand and operationalize assessment beyond surveys and data mining. As it relates to the student, I was always mindful of the assessment OF learning. But I went about viewing it through indirect measures of student success (i.e., National Survey of Student Engagement, Student Satisfactory Inventory) and coupling it with grade point averages to correlate learning. The idea of assessment FOR learning was

an idea I have read about since working with electronic portfolios, but have not had the opportunity to investigate further. I feel prompted to revisit my work to balance the two and how to teach my students to continuously reflect on their learning and draw connections across disciplines. The final takeaway from the course was how it helped me organize thoughts and ideas into a methodology for my instruction. Much of my education was focused on engineering, counseling, business and leadership development, and not teaching. I took on a teaching role at the University because I was good at delivering many onetime presentations for student organizations. As one of my students put it jokingly, going to class was getting a weekly motivational speech from me. This class helped me take my assessments up a notch and helped me provide a structure to the classes I teach. I can now see how standards, outcomes, and objectives can be threaded through a course or program and how it should be properly measured. Oddly, even though assessment has been around for a number of decades at the K-12 levels, higher education is now coming around to the idea of assessing its students OF and FOR learning. Within the last four years, positions at the University have emerged solely for student learning assessment. We are now seeing a more significant trend where assessing value-added learning will be driving performance-based budgeting. But every conversation will revolve around what are the learning targets supposed to be achieving and how are they measured over the students’ collegiate career.