Direct Measures of student learning consider actual student work or behavior as evidence of student learning outcomes.
Direct measures are most often drawn from student work embedded in a course (such as individual assignments, exams, or projects that are part of the course.) Course-embedded assignment allows for demonstration of learning in the classroom and also, when collected from a sample of students or across courses, how students within a program are generally meeting the learning outcome(s) associated with the program’s curriculum.
Direct assessment can also take place in co-curricular learning. Most commonly, students’ learning in these experiences can be measured by performance, assigned reflection, or situational observations.
| Direct Assessment of Student Learning - Data Options & When to Use | |||
| Data | Student Work Products | Assessment Tools | Learning Assessed |
| Product- based Student Academic Work | Capstone course products; course embedded content; Signature assignments;Portfolios; | Scoring rubrics (administered by faculty and/or external reviewers such as site supervisors, mentor teachers, etc.) |
Knowledge, skills, behaviors, or values represented by the Learning Outcome |
| Performance-based Student Academic Work | Presentations;Performances (music, theater, art show, poster session, dance, recitals, etc.) | Scoring rubrics (administered by faculty and/or external reviewers such as site supervisors, mentor teachers, etc.) | Knowledge, skills, behaviors, or values represented by the Learning Outcome |
| Note: Rather than collecting grades, evaluate the student work products for evidence of learning specific to the Learning Outcome (essays, final projects, multimedia content, performances, etc.) Rubrics are the most typical assessment tool. | |||
| Achievement Tests | Embedded questions on exams; Locally developed exams; Standardized tests | Test score analysis Scoring Rubrics Pre/Post Tests(administered by faculty and/or external testing providers) | Knowledge, skills, behaviors, or values represented by the Learning Outcome |
| Note: Rather than collecting grades, evaluate student test response(s) specifically related to the Learning Outcome under consideration. Individual test items can be evaluated on a 4-point scale (1-does not meet; 2-partially meets; 3- meets proficiency; or 4-exceeds proficiency) | |||
| Co-curricular learning experiences | Situational observations; Performances; Quizzes; Reflections; Short essays; Portfolios | Scoring rubrics (administered by staff and/or external reviewers such as site supervisors, etc.) Pre/post tests Student evaluations Advisor observations Shift reports | Knowledge, skills, behaviors, or values represented by the Learning Outcome |
| Note: When possible, integrate direct assessment into co-curricular experiences. It yields the richest data. Indirect assessment may be a necessary supplement. | |||
Direct Measures: Making them Effective
- Assignments/exams/student work products should be able to provide evidence directly related to the knowledge or skills represented by the Learning Outcome. Review the assignment prompt for alignment with the Learning Outcome.
- Evidence should reflect cumulative learning (i.e., should not be from work completed early in the quarter or experience).
- Faculty/staff should be confident that the assignment/exam/activity gave students the opportunity to demonstrate their learning fully. The format of the assignment may limit students' ability to demonstrate their learning if it asks solely for a brief response (e.g., a short answer question on an exam). Learning outcomes that reflect more complex cognitive outcomes (application/analysis/synthesis) will be best assessed through assignments that allow students to fully demonstrate their learning of these processes, such as longer papers or projects.
- Ideally, scoring takes place by a small group of faculty/staff (not just one faculty/staff) so that the focus of assessment stays on the program (not on individual students or course sections.) Sampling of student work is another key component.
- When rubrics are used to score student work, faculty/staff should be trained to use the rubric with a norming/calibration session.
- If the program is interested in assessing student attainment over time, it is important to include a preliminary assessment and a follow-up assessment that are comparable.
- Consider whether the same piece of evidence (e.g., research paper) can be used to assess more than one student Learning Outcome. Capstones are particularly conducive for this.
- Will the type of evidence gathered help the program understand what it can do to improve?
More Details and Examples of Commonly Used Direct Assessments Within Academic
Programs may elect to draw from multiple forms of direct assessment in their review of students’ progress on particular Learning Outcomes or groups of Learning Outcomes.
1.Course-embedded assignment assessment (assignments or other projects that are part of a course)
Course-embedded assessment is very common within departments for the assessment of program-level outcomes. Department faculty may select various forms of student coursework [assignment/exam/activity (see table above)] as source material for the assessment.
2. Collaboratively built and applied Signature Assignments (a common course-embedded assignment shared by multiple courses)
A signature assignment is one that has been adopted by program faculty to assess students’ learning toward one or more program-level learning outcomes. The “signature” of the assignment is that it has been developed and adopted collaboratively by faculty teaching within a program; while individual faculty may adapt the assignment for their courses, all faculty maintain core elements of the assignment that allow those elements to be assessed across courses, sections, or time using a common rubric or standard.
3. Evaluation of performance or presentations (live or on videotape or audiotape)
Students’ performances or presentations can often be a rich source of information about their learning, especially related to outcomes about oral communication. It is entirely appropriate to assess students’ work on these live or recorded. Often it is more convenient to record these presentations to allow for assessment at a later date and by a faculty committee.
4. Capstone course assessment
In many cases, capstones, as a culminating experience for majors, provide an excellent opportunity to assess multiple program learning outcomes near the end of majors’ degree programs. Often, these measurable outcomes include written and oral communication.
5. Portfolio Evaluation (through paper or ePortfolio, this can include multiple pieces of student work and can be used to measure growth over time)
Portfolio assignments generally involve students’ completion and curation of multiple pieces of student work over a period of time (a quarter or longer.) Ideally, portfolio assignments are designed to allow students to demonstrate their learning as well as produce reflections about their learning to further embed the content knowledge--and self development.
6. Tests and examinations (locally/faculty designed and/or commercially produced standardized tests).
An assessment approach that is generally quite easy to integrate is embedding assessment of program-level learning outcomes into exams students take as part of their classes or programs. These may be given (usually during the senior year) as a summative assessment. Just as with other forms of course-embedded program-level assessment, faculty can easily draw upon student performance scores on test items that are closely aligned with a particular program-level learning outcome.
7. Co-curricular Direct Assessment
The co-curriculum refers to learning activities and programs that reinforce the institution’s mission and values and complement the formal academic curriculum. Just as it is in the academic programs, co-curricular units and programs develop learning outcomes for students, assess these outcomes, design program improvements based on findings, and report on progress. While staff may think first of surveys or focus groups (both types of indirect data) for their programmatic assessments, there are many opportunities for direct assessment in co-curricular programs. The best approach to data collection depends on the learning outcome(s) to be measured and the nature of the activity.
Once your direct measures have been developed, an important next step is to determine your Targets and Benchmarks.
For programs with regular assessment cycles, please see Educational Assessment’s overview of annual program assessment and the annual assessment reporting template.