Surveys are a valuable and widely used tool to support the assessment of JEDI-B goals and learning outcomes. They can be used as the primary way to assess particular program goals and learning outcomes; and they also supplement other forms of assessment to deepen understanding of results and possible program changes.
Common JEDI-B Surveys
Climate Surveys
Climate surveys ask questions about individuals’ experiences of belonging and inclusion; bias, harassment, and discrimination within an organization (usually a department, program, or the university overall). Comprehensive climate surveys are often completed by students, faculty, and staff to capture a view of all stakeholders’ experiences and their perceptions of how their institution is doing to support diversity and equity. These surveys help campuses’ understand the extent to which individuals and groups feel welcomed, valued, and supported in their work, studies, research, or co-curricular activities. They can drill down into types of experiences, such as general academic experience, student-faculty interaction, peer-peer interaction, academic advising, and engagement and involvement patterns.
Institutions or programs use information from climate surveys to develop a better understanding of the extent to which their campus climate supports diversity and equity, and to inform and improve support, policies, and practices at their institution related to diversity and equity.
Climate surveys are typically administered anonymously in order to capture an accurate and truthful representation of climate. A wide range of demographic questions can be included, so that the results can be broken out by gender, race/ethnicity, sexual identity, disability status, age (traditional versus ‘nontraditional’), socioeconomic status, first-generation student versus non-first generation, and U.S. versus international residency status. Other factors relevant to a particular institution can also be included, such as academic major, graduate vs. undergraduate affiliation, students’ transfer status, or commuter status. Refer to “More on Demographic Questions” below when selecting which demographics to collect.
Even with the assurance of anonymity, care must be taken when analyzing climate survey results to protect the identities of individuals for groups that are small in number. Especially, as analyses include the intersection of multiple identities, the chances become greater that some individuals can be identified (e.g., an older person with a particular underrepresented race or ethnicity who identifies as gay).
The frequency by which climate surveys are administered can vary. Often a campus or department will capture a baseline view, and then readminister every few years, particularly if programs or the campus have made some changes and they wish to see the impact of those changes.
Resources
At SCU, since 2020, the Office of Diversity and Inclusion has helped different academic and non-academic units conduct and implement a departmental climate study. Contact Ray Plaza (rplaza@scu.edu) for more information.
The Communication Department recently drafted a department climate survey for students and faculty. The surveys were developed drawing upon other SCU department climate surveys, the SCU Belonging Survey, as well as a review of other campus’s climate surveys. The student survey was reviewed by a task force of students in the department. Contact Chris Bachen (cbachen@scu.edu) for a copy of the faculty or student climate surveys, if interested.
Question Sets
Programs can embed particular questions in existing surveys to monitor elements of the climate in classrooms, programs, or departments. Classroom climate can be assessed through a small number of questions in Mid-quarter surveys and in End-quarter Course Narrative Surveys. Co-curricular programs can similarly ask climate questions to gain information about how students experienced a particular program, learning experiences, and/or interactions among participants.
Embedding a small number of relevant climate or JEDI-B questions in program completion surveys or senior exit surveys gives programs a chance to take a yearly look into students’ experiences and perceptions and enables ongoing changes as needed.
Survey questions can be used to supplement other assessments. It is common for programs to have questions once they review assessment results. Why did some students respond one way or another? Why did some meet the learning goals, but others did not? Even with disaggregating data, not all questions may be answered. Short surveys or embedded questions can tease out some of the reasons behind the results a program obtained.
More on Demographic Questions
Including some demographic questions in your surveys allows you to disaggregate data and explore group-level differences in students’ experiences.
How many questions?
How does a program decide which demographics to include? One approach is to review academic literature for demographic variables that are likely to be most pertinent to your survey topic(s).
When you’re not sure which background characteristics will be most relevant, it can seem like a good idea to include as many demographic variables in your survey as possible. However, there are several reasons to think carefully about this:
- Too many demographics can make the survey too long (and lower your response rate)
- Respondents may be put off by the amount of personal information that’s being collected; they may question the relevance of asking these questions given what they understood the focus of the survey to be
- Odds are much of the data won’t be relevant or get analyzed
- For some units, the sample sizes will be so small that it will challenge anonymity expectations to report all the quantitative disaggregated data
A more tempered approach is to start with a small, carefully selected set of demographic questions. This can make the survey more approachable for students. Adding a few open ended questions within the survey can also help surface future demographics to include.
Which questions? Asking for information about demographics and social identities
The way students are asked to characterize their identities in the demographic questions in university-sponsored surveys can elicit strong reactions from students, both positive and negative.
While there is no definitive list of best ways to ask demographic questions sensitively and respectfully, we have compiled a set of recommendations (and included the rationales and provenance for the wording where applicable). We have developed a list of Demographic Survey Questions as a living document that evolves over time.
Other useful resources
Vanderbilt University, Lesbian, Gay, Bisexual, Transgender, Queer, & Intersex Life, “How to Ask About Sexuality/Gender”
University of Waterloo, “Guidelines for Collecting Demographic Information from Study Participants”
Oregon Social Learning Center, “Asking Demographic Questions in Research”, 2021
General Recommendations for Survey Development
Good practice in general is to draft surveys as economically as possible. Don’t ask anything in a survey that you don’t need to know or you can locate elsewhere; that you’ve recently asked elsewhere; or that you don’t plan to use. Respecting participants’ time and effort to provide feedback is the ethical way to proceed and will yield the best results.
- Clearly communicate at the beginning of the survey whether it is anonymous, confidential, or identifiable. Survey participants should know the purpose of the survey, how their data will be used, how any personally identifying information will be used, and who to contact for more information
- Be attentive to asking questions in ways that are culturally inclusive (especially any demographic questions) and accessible to all participants
- Consider adding a few open-ended questions that allow students to share individual perspectives