Evaluating Course Evaluations: An Empirical Analysis of a Quasi-Experiment at the Stanford Law School, 2000-2007


58 Journal of Legal Education 388 (2009)

Daniel E. Ho & Timothy H. Shapiro


Abstract

In Fall 2007, the Stanford Law School implemented what were thought to be innocuous changes to its teaching evaluations. In addition to subtle wording changes, the law school switched the default format for all second and third year courses from paper to online. No efforts to anchor the new evaluations to the old were made, as the expectation was that the ratings would be comparable. Amassing a new dataset of $34,328$ evaluations of all 267 unique instructors and 350 courses offered at Stanford Law School from 2000-07, we show they are not. This unique case study provides an ideal opportunity to shed empirical light on long-standing questions in legal education about the design, improvement, and implementation of course evaluations. Using finely-tuned statistical methods, we demonstrate considerable sensitivity of evaluation responses to wording, timing, and implementation. We offer suggestions on how to maximize information and comparability of evaluations for any institution contemplating reform of its teaching evaluation system.


Download Manuscript