Division of Student Affairs

How do we measure learning outcomes?

Measuring learning outcomes can take many forms. As such, it is important to consider the research question before selecting outcome measures. Once the question is asked, the research design should be constructed in the best possible way to answer the question. When selecting measures, assessment practitioners should have a clear sense of the research design, including information concerning the length of the assessment period, the targeted sample of the assessment, and of course, the resources available to execute the assessment.

These elements of the research design (e.g., time of assessment, number of students, available resources) drive decisions concerning measurement selection. Some measures of student learning have been theoretically-derived and empirically-validated; however, these measures are often too long, too expensive, or too cumbersome to administer. Alternatively, many assessment practitioners have chosen to develop their own instruments. Often, these measures are not psychometrically sound: they lack any theoretical basis, have not been pilot tested, are unreliable and not valid. So, what's the solution?

Adopting a "mixed measure" approach to assessing student learning outcomes may be the best solution for measurement selection. For some projects, using theoretically-derived and empirically-validated measures of student learning makes sense; for others, it may be more efficient and cost-effective to develop "home-grown" measures. Concerning the latter, it is important that these measures be developed based on theory and with a basic understanding of psychometrics.

At UNCW, we are in the process of using this "mixed measure" approach to measuring student learning. Our theoretically-derived and empirically-validated measures of student learning include the Reasoning about Current Issues Test (RCI; King and Kitchener, 1994), the Need for Cognition Scale (NCS; Cacioppo and Petty, 1982), the Intercultural Development Inventory (IDI: Bennett 1986; 1993), the Socially Responsible Leadership Scale (SRLS; Tyree, 2004), the Defining Issues Test (DIT-2; Rest, Narvaez, Bebeau, & Thoma, 1999) and the Ryff Psychological Well-being Scale (Ryff; 1984). In addition, we are in the process of pilot testing some of our own instruments designed to measure the learning outcomes developed for the Student Affairs Strategic Plan.