Chapter 3 | Learning Assessment
“Learning requires active manipulation of the material to be learned and cannot occur passively.”
(Reigeluth & Carr-Chellman, 2009)
The purpose of this chapter is to provide readers with easy-to-implement tools and strategies for assessing learning in open pedagogy projects. There is a wide variety of assessments available to use within the classroom. This chapter includes both general and open pedagogy assessments that instructors can use as they adopt open practices. The primary focus of this chapter is on strategies to assess student learning. However, some of these techniques may inform instructors on their teaching, especially around open pedagogy techniques. The key take-away from this chapter is that learning assessment and open pedagogy are closely-linked, and that learning assessment can be viewed as an extension of the open classroom.
There are multiple types of assessments that can be used within the classroom to understand student learning. Common learning assessments include formative, summative, self-assessment, peer-assessment, and more (Elder, 2019). Formative assessments can be defined as assessments that can be conducted by instructors or students that inform future lessons during the course (Black & William, 2009). These assessments can be informal and are meant to gauge student learning of skills or concepts (Indiana University Bloomington Center for Innovative Teaching and Learning). Summative assessments are typically graded and used at the end of a section or course to assess student learning (Northern Illinois University Center for Innovative Teaching and Learning).
In exploring open pedagogy learning assessment, we found that they generally fall into two categories: Assessments that include open pedagogy practices like copyright and open licensing; or assessments that focus specifically on learning outcomes subject knowledge outcomes. The latter assessments are still used for open pedagogy assignments, but students may not be evaluated on skills and processes that may be developed through the use of open practices. In these cases, the outcome of the assignments are shared openly.
Assessments directly focused on open pedagogy are helpful, but other general assessments can be used for these projects. Iowa State University’s Center for Excellence in Learning and Teaching (CELT) has an excellent outline of common classroom and learning assessment techniques that can be applied to open pedagogy assignments (CELT). CELT has divided the assessments into three categories based on the information gathered from the assessment: 1) Student Knowledge and Skills, 2) Student Attitudes, Values, and Self-Awareness, and 3) Student Wellness and Well-Being. When introducing concepts related to open pedagogy such as copyright or creative commons licensing, the assessment examples outlined in Student Knowledge and Skills could be helpful to quickly gauge understanding of these topics and determine if additional instruction needs to be provided. Additionally, assessments that allow students to share questions, anonymously or not, can help identify areas within open pedagogy assignments where students need additional explanation to be successful. CELT’s list of learning assessments can be a helpful tool to quickly identify common assessments that can be added to open pedagogy projects and assignments.
Another type of assessment that can be used for open pedagogy assignments is “ungrading.” In an essay by Jesse Stommel (2021), “ungrading starts with teachers just talking to students about grades. Demystifying grades (and the culture around them) gives students a sense of ownership over their own education.” Applying upgrading in the classroom allows opportunities for students and faculty to collaborate on building assessments and rubrics to measure learning (Rapchak, Hands & Hensley, 2023). Additionally, upgrading can be an approach to creating a more equitable and inclusive classroom (Rapchak, Hands & Hensley, 2023). Ungrading and open pedagogy connect through goals of student agency and inclusiveness. In an open pedagogy assignment, students can co-create a shared assessment or create self-evaluations based on their own learning. Rapchak, Hands, & Hensley note that self-evaluations build skills for conducting future evaluations in the workplace.
Examples of General Learning Assessments Applied to Open Pedagogy Projects
Student-Developed Lesson Plans
(Education course, 4-year public university, USA)
Stacy Katz and Jennifer Van Allen developed a framework to approach re-designing course assignments to be more “renewable” or part of an open educational practice. Katz and Van Allen (2021) developed a Renewable Assignment Design Framework. In redesigning a course assignment to include open practices, Katz and Van Allen reviewed the original course rubric. The rubric is revised as their last step in re-designing their course assignment.
Van Allen’s revised assignment asked students to create an OER to share in OER Commons. The final rubric that accompanied the assignment can be found in their chapter, “Evolving into the Open: A Framework for Collaborative Design of Renewable Assignments.” In this example, the rubric ties to the content of the final project, but the deliverable is an open educational resource.
Co-Created Syllabus
(Humanities course, 4-year public university, USA)
Amy Nelson developed a more collaborative approach to assessing learning outcomes for a Soviet History course (Nelson, 2019). In this course, Nelson and her students co-created a course syllabus that included expectations for both the students and instructor. Nelson developed a general syllabus that met the institutional requirements and the class used that document to create a new syllabus. In this example, Nelson provided general guidance on the course assessments to facilitate the discussion with students. As a result of the “open” course syllabus, students prioritized respectful and engaging discussions, which comprised 25% of the final grade. The new syllabus breaks down major assignments in order to include a list of tools and skills needed as well as strategies to complete the assignments.
Collaborative Course Notes
(Science course; 4-year public university, USA)
In another open pedagogy assignment, students were asked to contribute to collaborative course notes using the tool “Hypothes.is” to create annotations to scholarly articles (Goller, Vandegrift, Cross, & Smith, 2021). As part of the rubric for the annotations, students were asked to focus on three areas related to each article annotation: overview, results, and summary. This assessment example focuses on students applying critical thinking skills as well as their understanding of scientific literacy. The rubric focuses on the skills and knowledge gained from creating annotations of scholarly literature and contributing it to a shared space. Additionally, the rubric reminds students that the collaborative notes will be posted publicly.
Student-Created Videos
(Science course, 4-year public university, USA)
Students in an agriculture course were required to create videos that included open access images and other materials as part of a course project. The assignment layered active learning with open pedagogy. The rubric focused on the substance and design of the videos (Shea, 2019). However, students encountered open pedagogical practices as they built their videos. Instruction was provided on how to locate and use materials under a Creative Commons License. Students used this education for the videos and also in understanding their own Creative Commons license attached to their video. In addition to the final rubric, students participated in a formative assessment at each point of the assignment (Shea). Assessments that focus on reflection is another option that can be used for open pedagogy assignments. “Reflective assessment emphasizes gathering feedback through observing, questioning, writing, illustrating, and discussing” (Bond, Denton, & Ellis, 2015, p3). Kristen McCleary (2023)developed a series of reflective questions for students to use to write essays on oral histories. Students discussed their reflections with their peers as well.
Examples of Open Pedagogy-Focused Learning Assessments
In another example of adapting rubrics to incorporate open practices, Nate Angell (2019) developed an Open Knowledge Practices Learning Experience Rubric. The rubric focuses on these five areas: portable skills, renewable work, engaging openness, ensuring inclusivity, and exploring roles and relationships. The rubric arranges these skills into 3 categories: 1) Emerging, 2) Developing, and 3) Transforming learning. The rubric explicitly evaluates assignments and projects that use an open pedagogy approach.
|
|
Emerging | Developing | Transforming |
| Portable skills | Training and practice in required skills are integrated into the experience. | Participants practice skills that would be valuable for them beyond the experience. | Participants apply skills in new situations outside the experience. |
| Renewable work | Participants produce works connected to existing works that have value in the world. | Participants modify and/or add to existing works that have value in the world. | Participants produce new works that have value in the world. |
| Engaging openness | Participants engage with open materials and/or activities (eg, open data, open educational resources, open-source technologies, etc). | Participants apply open practices and/or tools to closed materials and/or activities. | Participants engage directly and productively with new or existing open projects and/or communities. |
| Ensuring inclusivity | Materials and activities support and encourage access and use by diverse participants. | Participants modify/extend materials and activities to enable better access and use within the experience. | Participants modify, extend, and/or promote materials and/or activities to engage communities outside the experience. |
| Exploring roles & relationships | Roles and relationships shared among participants and broader communities are clearly surfaced and discussed. | Participants explore different roles and relationships like learning/teaching, receiving/giving, observing/making, consuming/producing, autonomy/interdependence, freedom/responsibility. | Participants empower themselves and others to take on new roles and relationships outside the experience. |
| Open Knowledge Practices Learning Experience Rubric 1.1 by Nate Angell, 8 Nov 2019. Licensed via Creative Commons Attribution 4.0 International. Additional contributors include Heather Angell, Twyla Angell. Comment/contribute and find current version and other formats at xolotl.org/okp-learning-experience-rubric. | |||
Currently, many instructors are using established assessments in their open pedagogy assignments. In some cases, instructors have developed new assessments that embrace the skills and values around open pedagogy. As open pedagogy continues to develop, more assessment strategies may be shared with the open community.
References
Angell, N. (2019). OKP Learning Experience Rubric 1.1 Accessed at https://xolotl.org/okp-learning-experience-rubric/
Black, P., & Wiliam, D. (2009). Developing the theory of formative assessment. Educational Assessment, Evaluation and Accountability, 21(1), 5-31. doi:https://doi.org/10.1007/s11092-008-9068-5
Bond, J., Denton, D. W., & Ellis, A. K. (2015). Impact of reflective assessment on student learning: Best-evidence synthesis from then quantitative studies. International Dialogues on Education, 2(2), 172-184.
Center for Excellence in Learning and Teaching (CELT), Iowa State University. (n.d.) Classroom Assessment Techniques. Accessed at https://www.celt.iastate.edu/instructional-strategies/evaluating-teaching/classroom-assessment-techniques-quick-strategies-to-check-student-learning-in-class/
Elder, A. (2019). “Assessing Course Outcomes.” in The OER Starter Kit. Retrieved from https://iastate.pressbooks.pub/oerstarterkit/chapter/assessing-outcomes/
Goller, C., Vandegrift, M., Cross, W., & Smyth, D. S. (2021). Sharing notes is encouraged: Annotating and cocreating with hypothes.is and Google Docs. Journal of Microbiology and Biology Education, 22 (1). https://doi.org/10.1128/jmbe.v22i1.2135
Hallam, S., Willingham, P., & Baranovic, K. (2021). A process of engagement: Using government documents in open pedagogy. The Journal of Academic Librarianship, 47(3), 1-8. https://doi.org/10.1016/j.acalib.2021.102358
Hoffman, K.D., Chierici, R., & Spence, A. (2020). “What if we were to go?”:Undergraduates simulate the building of an NGO from theory to practice. In A. Clifton & K. Hoffman (Eds.). Open pedagogy approaches: Faculty, library, and student collaborations. Milne Library. https://milnepublishing.geneseo.edu/openpedagogyapproaches/chapter/what-if-we-were-to-go-undergraduates-simulate-the-building-of-an-ngo-from-theory-to-practice/
Indiana University Bloomington Center for Innovative Teaching and Learning. (n.d.). Summative and Formative Assessment. Retrieved from https://citl.indiana.edu/teaching-resources/assessing-student-learning/summative-formative/index.html
Katz, S. & Van Allen, J. (2021). Evolving into the open: A framework for collaborative design of renewable assignments. In Open pedagogy approaches. https://milnepublishing.geneseo.edu/openpedagogyapproaches/
McCleary, K. (2023). Hist 150 Final Assessment Overview. In Emotional histories: Oral history assignment for social change. https://docs.google.com/document/d/1x72XWzzU7Wz-lj1IJpWP6rlecXKdr6f99tj5I4o0eAM/edit#heading=h.tx0xybkkyg6c
Nelson, A. (2019). Collaborative Syllabus Design: Students at the center. In A. Clifton & K. Hoffman (Eds.). Open pedagogy approaches: Faculty, library, and student collaborations. Milne Library. https://openpedagogy.org/course-level/collaborative-syllabus-design-students-at-the-center/
Northern Illinois University Center for Innovative Teaching and Learning. (2012). Formative and summative assessment. In Instructional guide for university faculty and teaching assistants. Retrieved from https://www.niu.edu/citl/resources/guides/instructional-guide
Rapchak, M., Hands, A. S., & Hensley, M. K. (2023). Moving toward equity: Experiences with ungrading. Journal of Education for Library and Information Science, 64(1), 89-98. doi:https://doi.org/10.3138/jelis-2021-0062
Reigeluth, C. M. & Carr-Chellman, A. A. (2009). Understanding instructional theory. In C. Reigeluth & A. Carr-Chellman (Eds.). Instructional-Design theories and models, volume III: Building a common knowledge base. Routledge.
Shea, A. (2019). Building a collection of openly licensed student-developed videos. In A. Clifton & K. Hoffman (Eds.). Open pedagogy approaches: Faculty, library, and student collaborations. Milne Library. https://milnepublishing.geneseo.edu/openpedagogyapproaches/chapter/building-a-collection-of-openly-licensed-student-developed-videos/
Stommel, J. (2021). Upgrading: an introduction. Retrieved from https://www.jessestommel.com/ungrading-an-introduction/.
