Equity-Minded Assessment Practices - Santa Clara University

Over the past few years, assessment researchers and practitioners have focused increasingly on the importance of culturally responsive and equitable approaches to assessment. While these conversations are ongoing and terms are evolving, assessment practices are adapting to become more equity-minded.

"Equity-minded assessment refers to ways we ensure assessment processes and practices are appropriate for all students and that we ultimately do no harm in the process. While it can be challenging to consider the vast differences and needs of our student populations in our practices, our task as educational providers is to strive to help every student succeed."

Montenegro & Jankowski summarize a list of actions that comprise equity-minded assessment (p.13):

Check biases and ask reflective questions throughout the assessment process to address assumptions and positions of privilege.
Use multiple sources of evidence appropriate for the students being assessed and assessment effort.
Include student perspectives and take action based on perspectives.
Increase transparency in assessment results and actions taken.
Ensure collected data can be meaningfully disaggregated and interrogated.
Make evidence-based changes that address issues of equity that are context-specific.

These recommendations apply to a number of elements in the assessment cycle, and are elaborated in the points below.

Equity-Minded Assessment

Equity-minded assessment practices can be embedded throughout the assessment process– from the way in which the program defines its mission and vision statements, to the way in which assessments are conducted, and the data are reported and analyzed.

Mission and Vision Programs and departments can align their mission and vision statements to reflect their JEDI-B goals and learning outcomes. For example, at the University of Rhode Island: The Sociology and Anthropology Department strives to provide meaningful educational experiences for our students, as well as contribute to building knowledge in our disciplines. Our curriculum, teaching, and interaction with students and colleagues are all rooted in a commitment to respect, equity, diversity, inclusion, and social justice. Our department is dedicated to creating a supportive community for our students and facilitating their academic and professional growth.
Program Goals and Learning Outcomes It’s important to invite student input into all program goals and learning outcomes as an equity-minded practice. Students provide valuable perspectives on what’s clear and meaningful, and can suggest goals and outcomes that may not have been considered. Programs and departments can develop specific program goals and student learning outcomes that address justice, equity, diversity, equity, inclusion, and belonging (JEDI-B), at times in response to student interest or demand. (More information is available on the JEDI-B goals and outcomes page.) Assessing these program goals and learning outcomes at regular intervals allows programs to track the progress of interventions and their impacts on student learning and experience.
Reflecting on Learning Experiences Whether or not a program’s goals or learning outcomes are specific to JEDI, equity-minded practitioners consider how they can develop learning experiences that utilize inclusive practices and work to accomplish program goals and learning outcomes that benefit all students. SCU’s Faculty Collaborative for Teaching Innovation has compiled a set of Digital Resources for Teaching (DRT) that can support curricular exploration and innovation. The Collaborative’s page on Transparent Assignment Design is particularly well suited to an equity-minded approach.
Assessment Processes Sampling To be able to meaningfully disaggregate work according to relevant groups of learners, programs may need to select student work for assessment strategically so that comparisons can be made between student subgroups of interest. First, determine the subgroups of interest and then it’s possible to select student work randomly from within the subgroup(s). Programs may need to use larger sample sizes so they have enough information from the relevant subgroups from which to draw conclusions. This might also require collecting samples from several sections, quarters, or even across years to accumulate enough data to reach meaningful conclusions. For more guidance on sampling, review EA’s Sampling page. Direct Assessment Methods Consider drawing on multiple types of assignments as assessment artifacts that allow students to demonstrate their competencies using more than one modality. An overview of types of assessment artifacts can be found on EA’s Direct Measures page. Some examples that illustrate the particular utility of ePortfolios for direct assessment are included on our ePortfolios page. E-Portfolios provide students with the ability to select multiple examples of their work to demonstrate their achievement of learning outcomes and with the opportunity to provide their own reflections and rationalization of their learning. Signature assignments (such as in capstones or other culminating experiences) that address DEI learning outcomes can be valuable for several reasons: signature assignments allow for meaningful assessment of program-level outcomes across course sections and cohorts; signature assignments in a capstone or other culminating experience indicate student mastery near or at time of exit from the institution; and these assignments invite students to demonstrate their learning specific to DEI learning outcomes either within a program or through an integration of learning from their various core, major, and co-curricular experiences.
Analyzing and Reporting Data It is important to disaggregate data by student identity and affiliation. This helps uncover differences in student experiences that are not visible in aggregate data. If you can't publish visual representations of data in order to protect individual student identities (see below), you can still use your results as a means of outreach and share data that highlights specific issues affecting minoritized groups with relevant campus offices to support these students. “When conducting statistical analysis and looking for differences across populations, do not hold white students as the bar for comparison. In addition, move the analysis beyond between group analysis and towards within-group analysis honoring the notion that not all students in one predefined, oversimplified, cultural placeholder designating group belonging have the same experience. In addition, the concept of statistical significance is different than finding data that is significant to a program or population.” (Levy, J., & Heiser, C. (2018, March). Inclusive assessment practice (Equity Response). Urbana, IL: University of Illinois and Indiana University, National Institute for Learning Outcomes Assessment (NILOA).)” If samples are small, it is important to take great care in reporting specific findings or data. Educational Assessment generally recommends caution in reporting data when the sample size is smaller than 10. If the sample is 5 or fewer on certain demographic variables, it is highly likely that students could be identified. This could pose harm. When you need to do some aggregation, you can still elevate the experiences of your smaller student groups in your reporting: You may be able to share open-ended data that help tell underrepresented student stories. You still may be able identify patterns in the data and then share the data that highlights inequities or negative experiences with relevant campus units. Remember, data that lacks statistical significance due to small sample sizes is still the authentic experience of the students who shared their insights. It’s still telling lived stories and may suggest a need to take action. When developing visual representations (e.g., charts, tables), Educational Assessment recommends presenting smaller samples by sample size (not by percentage). Comparing percentages of small samples and large samples within a single graphic can confuse an audience and even potentially mislead with data. When presenting results through graphics, it’s important to create visuals that are accessible in color and font size. Adobe has developed a site that allows the user to select a pallet of colors that will be easier to view online. UC Berkeley has created a website dedicated to digital accessibility. We recommend reviewing their recommendations for Google docs, pdfs, Qualtrics, etc.
Respectful Communication in Reporting Another dimension of inclusivity and equity-focused data reporting is to write about identity and affinity groups sensitively and respectfully. Michigan State University has developed an Inclusivity Guide that may be helpful when drafting and editing content. Their recommendations for general practice have been copied below: Use language in accordance with the individual’s identity. Be specific and avoid generalizing identity groups. Avoid the use of pejoratives. However, exceptions can be made for quotations if relevant to the content. The origins of seemingly innocuous idioms or words may be racist, sexist or ableist in nature, such as “cakewalk” or “grandfather clause.” Consider the origins of everyday language before freely using it in communications. Avoid reinforcing deficit narratives that place people as victims of societal problems and myths that ignore systemic barriers, as both prop up negative stereotypes. Instead use truth-telling about those deficits and barriers. We all make mistakes. Give yourself grace, reflect and consider ways to acknowledge any unintentional harm that may have resulted. For tools on inclusion, respect and accountability, visit Building Inclusive Communities.

Mission and Vision

Programs and departments can align their mission and vision statements to reflect their JEDI-B goals and learning outcomes. For example, at the University of Rhode Island:

The Sociology and Anthropology Department strives to provide meaningful educational experiences for our students, as well as contribute to building knowledge in our disciplines. Our curriculum, teaching, and interaction with students and colleagues are all rooted in a commitment to respect, equity, diversity, inclusion, and social justice. Our department is dedicated to creating a supportive community for our students and facilitating their academic and professional growth.

Program Goals and Learning Outcomes

It’s important to invite student input into all program goals and learning outcomes as an equity-minded practice. Students provide valuable perspectives on what’s clear and meaningful, and can suggest goals and outcomes that may not have been considered.

Programs and departments can develop specific program goals and student learning outcomes that address justice, equity, diversity, equity, inclusion, and belonging (JEDI-B), at times in response to student interest or demand. (More information is available on the JEDI-B goals and outcomes page.) Assessing these program goals and learning outcomes at regular intervals allows programs to track the progress of interventions and their impacts on student learning and experience.

Reflecting on Learning Experiences

Whether or not a program’s goals or learning outcomes are specific to JEDI, equity-minded practitioners consider how they can develop learning experiences that utilize inclusive practices and work to accomplish program goals and learning outcomes that benefit all students. SCU’s Faculty Collaborative for Teaching Innovation has compiled a set of Digital Resources for Teaching (DRT) that can support curricular exploration and innovation. The Collaborative’s page on Transparent Assignment Design is particularly well suited to an equity-minded approach.

Assessment Processes

Sampling

To be able to meaningfully disaggregate work according to relevant groups of learners, programs may need to select student work for assessment strategically so that comparisons can be made between student subgroups of interest. First, determine the subgroups of interest and then it’s possible to select student work randomly from within the subgroup(s). Programs may need to use larger sample sizes so they have enough information from the relevant subgroups from which to draw conclusions. This might also require collecting samples from several sections, quarters, or even across years to accumulate enough data to reach meaningful conclusions. For more guidance on sampling, review EA’s Sampling page.

Direct Assessment Methods

Consider drawing on multiple types of assignments as assessment artifacts that allow students to demonstrate their competencies using more than one modality. An overview of types of assessment artifacts can be found on EA’s Direct Measures page. Some examples that illustrate the particular utility of ePortfolios for direct assessment are included on our ePortfolios page. E-Portfolios provide students with the ability to select multiple examples of their work to demonstrate their achievement of learning outcomes and with the opportunity to provide their own reflections and rationalization of their learning.

Signature assignments (such as in capstones or other culminating experiences) that address DEI learning outcomes can be valuable for several reasons: signature assignments allow for meaningful assessment of program-level outcomes across course sections and cohorts; signature assignments in a capstone or other culminating experience indicate student mastery near or at time of exit from the institution; and these assignments invite students to demonstrate their learning specific to DEI learning outcomes either within a program or through an integration of learning from their various core, major, and co-curricular experiences.

Analyzing and Reporting Data

It is important to disaggregate data by student identity and affiliation. This helps uncover differences in student experiences that are not visible in aggregate data. If you can't publish visual representations of data in order to protect individual student identities (see below), you can still use your results as a means of outreach and share data that highlights specific issues affecting minoritized groups with relevant campus offices to support these students.

“When conducting statistical analysis and looking for differences across populations, do not hold white students as the bar for comparison. In addition, move the analysis beyond between group analysis and towards within-group analysis honoring the notion that not all students in one predefined, oversimplified, cultural placeholder designating group belonging have the same experience. In addition, the concept of statistical significance is different than finding data that is significant to a program or population.”

(Levy, J., & Heiser, C. (2018, March). Inclusive assessment practice (Equity Response). Urbana, IL: University of Illinois and Indiana University, National Institute for Learning Outcomes Assessment (NILOA).)”

If samples are small, it is important to take great care in reporting specific findings or data. Educational Assessment generally recommends caution in reporting data when the sample size is smaller than 10. If the sample is 5 or fewer on certain demographic variables, it is highly likely that students could be identified. This could pose harm.

When you need to do some aggregation, you can still elevate the experiences of your smaller student groups in your reporting:

You may be able to share open-ended data that help tell underrepresented student stories.
You still may be able identify patterns in the data and then share the data that highlights inequities or negative experiences with relevant campus units.
Remember, data that lacks statistical significance due to small sample sizes is still the authentic experience of the students who shared their insights. It’s still telling lived stories and may suggest a need to take action.

When developing visual representations (e.g., charts, tables), Educational Assessment recommends presenting smaller samples by sample size (not by percentage). Comparing percentages of small samples and large samples within a single graphic can confuse an audience and even potentially mislead with data.

When presenting results through graphics, it’s important to create visuals that are accessible in color and font size. Adobe has developed a site that allows the user to select a pallet of colors that will be easier to view online.

UC Berkeley has created a website dedicated to digital accessibility. We recommend reviewing their recommendations for Google docs, pdfs, Qualtrics, etc.

Respectful Communication in Reporting

Another dimension of inclusivity and equity-focused data reporting is to write about identity and affinity groups sensitively and respectfully. Michigan State University has developed an Inclusivity Guide that may be helpful when drafting and editing content. Their recommendations for general practice have been copied below:

Use language in accordance with the individual’s identity.
Be specific and avoid generalizing identity groups.
Avoid the use of pejoratives. However, exceptions can be made for quotations if relevant to the content.
The origins of seemingly innocuous idioms or words may be racist, sexist or ableist in nature, such as “cakewalk” or “grandfather clause.” Consider the origins of everyday language before freely using it in communications.
Avoid reinforcing deficit narratives that place people as victims of societal problems and myths that ignore systemic barriers, as both prop up negative stereotypes. Instead use truth-telling about those deficits and barriers.
We all make mistakes. Give yourself grace, reflect and consider ways to acknowledge any unintentional harm that may have resulted. For tools on inclusion, respect and accountability, visit Building Inclusive Communities.

References

–Montenegro, E., & Jankowski, N. A. (2020, January). A new decade for assessment: Embedding equity into assessment praxis. (Occasional Paper No. 42). Urbana, IL: University of Illinois and Indiana University, National Institute for Learning Outcomes Assessment (NILOA).