General Education Assessment
- Philosophy and rationale
- Work product collection
- Work product review
- Dissemination of results
- Program Improvement
UNCW has adopted an approach to assessing its Learning Goals at the University Studies level that uses assignments that are a regular part of the course content. A strength of this approach is that the student work products are an authentic part of the curriculum, and hence there is a natural alignment often missing in standardized assessments. Students are motivated to perform at their best because the assignments are part of the course content and course grade. The assessment activities require little additional effort on the part of course faculty because the assignments used are a regular part of the coursework. An additional strength of this method is faculty collaboration and full participation in both the selection of the assignments and the scoring of the student work products.
The student work products collected are scored independently on a common rubric by trained scorers. The results of this scoring provide quantitative estimates of students’ performance and qualitative descriptions of what each performance level looks like, which provides valuable information for the process of improvement. The normal disadvantage to this type of approach when compared to standardized tests is that results cannot be compared to other institutions. This disadvantage is mitigated in part by the use of the AAC&U VALUE rubrics for many of the Learning Goals. This concern is also addressed by the regular administration of standardized assessments, giving the university the opportunity to make such comparisons. back to top
The eight Learning Goals are assessed on a rotating basis. The schedule has flexibility to increase the frequency of assessment of any learning goals, for example, one with inconclusive or confusing results. back to top
The sampling method used lays the foundation for the generalizability of the results. No one part of the University Studies curriculum, nor for that matter no one part of the university experience, is solely responsible for helping students to write well, think critically, or conduct responsible inquiry and analysis. These skills are practiced in many courses. Therefore, a matrix approach to sampling is taken, so that, over time, work products will be selected from all general education components that are aligned to each UNCW Learning Goal. The University Studies Curriculum Map illustrates this alignment. Once courses are selected for sampling, section selection is done to insure a representative mix of course offerings (for example, by in class or distance methods, and by instructor type – tenure-line, lecturer, or part time).
Prior to the start of each semester, the General Education Assessment Director meets with the selected course instructors to familiarize them with the VALUE rubrics. Instructors are asked to review their course content and assignments, and to select one assignment that they believe fits the dimensions of learning goal being assessed and the corresponding rubric. Instructors should include the Statement for Students provided by General Education Assessment in the syllabus for their course to be sampled.
During the analysis of the scores, the name and student ID information on the cover sheets is matched with student demographic information in university records for the purpose of analysis based on demographic and preparedness variables. back to top
Reviewers are recruited from UNCW faculty to include reviewers from a broad spectrum of departments. The intent is to give all faculty the opportunity to see the work being done by students in the general education courses and to learn more about the Learning Goals and rubrics. It is also important to have a least one faculty member from each of the departments from which student work products are being reviewed (for some content-heavy assignments, scorers are only selected from the discipline). back to top
Metarubrics, such as the VALUE rubrics, are constructed so that they can be used to score a variety of student artifacts across disciplines, across universities, and across preparation levels. But their strength is also a weakness: the generality of the rubric makes it more difficult to use than a rubric that is created for one specific assignment. To address this issue, a process must be created that not only introduces the rubric to the scorers, but also makes its use more manageable.
Volunteer scorers initially attend a two and a half hour workshop on the rubric they will be using. During the workshop, scorers review the rubric in detail and are introduced to the general assumptions adopted for applying the rubrics. After reviewing the rubric and initial assumptions, the volunteers read and score 3 – 4 student work products. Scoring is followed by a detailed discussion, so that scorers can better see the nuances of the rubric and learn what fellow scorers saw in the work products. From these discussions, scorer norming begins and assumptions are developed for applying the rubric to each specific assignment.
Scoring of the student work products is done during a scoring event. These events are either a half day or a whole day. Pairs of scorers are assigned a packet of student work on the same assignment. Scorers read the first paper, which is the same in each packet, score it, and then discuss their scores, coming to concensus on that score and specific interpretation of the rubric as it relates to the specific assignment. The packets also contain addition common papers which the scorers score individually. These are used to measure interrater reliability. back to top
A general education assessment report is written annually by the General Education Director. This report is first presented to the Learning Assessment Council, which may make recommendations to the Provost Office based on results. The report is also presented to the University Studies Advisory Committee, which may also make recommendations to the Faculty Senate. The reports, with recommendations, are published. Findings categorized by Learning Goal are also published on the web. back to top
Bodies responsible for implementing program improvements based on general education assessment results include the Provost's Office, the Faculty Senate, the University Studies Advisory Committee, and the faculty in departments that offer University Studies courses. back to top