Specifications Grading & Ungrading

The passion I had for teaching was being killed by the dread I continuously faced with grading and worse, handing back the graded assignments. I found the after-grade encounters often antagonistic and more often related to winning back points than understanding the lessons to be learned from the experience. That is a quote from Linda Nilson's book Specifications Grading: Restoring Rigor, Motivating Students, and Saving Faculty Time. Along with several colleagues at Rose-Hulman, I read through the book and decided to implement Nilson's method for student assessment. Given the increase in the quality of work my students have produced, I will never go back! Since then, I have continued to explore non-traditional assessment structures in my class, implementing both Specifications Grading and more "pure" Ungrading in my courses. Additionally, I enjoy discussing assessment with anyone willing to have the conversation. This page contains various resources related to those discussions. As with any endeavor, I continue to learn and refine my approaches, and I do not claim to have all the answers!

Presentations/Publications

• Specifications Grading: An Overview and Example
  I was invited to write two posts for the Statistics Teaching and Learning Corner. The first post provides an introduction to the framework of Specs Grading; the second post is a high-level overview of an implementation of Specs Grading in one of my upper level courses. These articles provide a good starting point for those deciding whether they want to consider Specs Grading for their courses.
• Specifications Grading in a Statistics Classroom
  I led a half-day workshop in 2017 at USCOTS introducing the topic and providing tips for getting started. The workbook for participants, based largely on Nilson's text, could help those interested in designing a course with Specs Grading. While it might be helpful to a general audience, it is specifically written with statistics educators in mind.
• Specifications Grading at Rose-Hulman
  Dr. Sylvia Carlisle and I led a workshop for faculty at Rose-Hulman introducing specifications grading. A copy of our slides and the accompanying workbook may help those at Rose-Hulman get started with designing a course with Specs Grading.

Course Materials

• Engineering Statistics (Introductory Course)
  This is an introductory statistics course aimed at engineers and scientists; while it has a calculus pre-requisite, my version of the course does not have computations which rely on that material. To get a feel for the level of the course, you can review the course text, which is a culmination of previous course notes for the class. While I tinker with this course every term, the basic structure remains the same. This is hybrid-specs system as it relies on points. It is a class which is coordinated, but not all instructors use specs grading; so, we give a common final exam.
  • Course Syllabus: provides the course-level objectives of the course, as well as my introduction of specs grading for students. This gives you a high-level view of the course structure.
  • Rubric for Class Questions: provides the specifications for each type of open-ended question asked in the class. While I have moved to providing specs for each question on an exam, I still find it helpful to have a single document that provides the specs for various types of questions for consistency across the course.
  • Module-Level Objectives: this is a complete list of my module-level objectives which support the course objectives listed in the Course Syllabus. This also illustrates the module-structure of the course. It is helpful to note that the class is divided into three units of three modules (1-sample inference, regression, comparing groups).
• Statistical Programming (Intro to Data Science Type Course)
  This course requires an introductory exposure to statistics, including randomization-based inference. This course is aimed at students understanding the pipeline of an applied project - from data collection (scraping) through presentation of results in a reproducible fashion. By far, this is the class that I think I have felt most successful with specs grading. Students have little trouble adapting to the framework in this class.
  • Course Syllabus: provides the course-level objectives of the course, as well as my introduction of specs grading for students. This gives you a high-level view of the course structure.
  • Planning Sheet: when you are first designing a course, it is a good idea to layout how the activities align with the objectives of the course. I admit that I have not always done this well, but this class is an exception. In addition to laying things out, I tried to document the process, which is presented here. It was meant for my personal records, but it might help you think about how to document a course build yourself. I liked doing this in OneNote as it is an infinitely large workspace; for those who have never done something like this, it is important to remember that this is a living document. I created it with just the objectives first, but I updated it regularly as the course took shape.
• Biostatistics (Intermediate Course)
  This second course in statistics focuses on statistical modeling, particularly models common in the biological sciences, including the linear model, repeated measures, nonlinear models (including logistic regression), and survival analysis. This course is at the top with regard to my satisfaction with the structure.
  • Course Syllabus: provides the course-level objectives of the course, as well as my introduction of specs grading for students. This gives you a high-level view of the course structure. Note: this syllabus was from the Spring of 2020; in response to the pandemic, changes were made to the syllabus, which are shown in this document.
  • Specs for a Homework Assignment: used as group practice toward the computational and literacy with course concepts.
  • Specs for an Article Review: used to ensure article is read prior to class discussion, which is where the real learning occurs.
  • Specs for a Module Quiz: this is an automated assessment of computational skills and literacy with concepts.
  • Specs for a Concept Check: this is to assess mastery of explaining key concepts from a module in a multi-disciplinary team.
  • Specs for an Analysis Task: assess mastery of computational solutions and communication in a setting in which the components of the analysis are explicitly asked for. This is similar to an exam; so, this illustrates how specs might be combined with "points" in order to determine a satisfactory assignment.
  • Specs for a Case Study: a capstone type experience at the individual level, students must replicate a study and write a report. This is a very detailed set of specifications as it pertains to a multi-page report. Therefore, examples of this type of successful report are not easy to demonstrate on the page but are still given to students (note, these examples are from general reports but they do not replicate an existing work). Each assignment is annotated to help students understand the examples.
    • Successful Example: a really high quality submission that would have received a passing score.
    • Successful Example, but not Perfect: a submission that would have received a passing score but has areas of improvement (did not meet all specifications).
    • Unsuccessful Example: a low quality submission that would not have received a passing score.
  • Course Plan: my syllabus and other course documents are the result of a planning effort involving backward design. I capture this in a OneNote document, which is illustrated for the Biostat course here. The document is a draft of how the elements of a course fit together to meet the learning outcomes.
• Mathematical Statistics (Advanced Course)
  A course for those considering graduate school, this course has both introductory statistics and probability as pre-requisites. In addition, I can assume students have seen multivariable calculus and had some exposure to computing. This course is generally taken by seniors and is a good mix of theory and computation. I have only taught this class with specs grading once, but I was happy with this iteration. I am very grateful to Dr. Sylvia Carlilse's work with turning a "Real Analysis" course into a specs graded course; that was the inspriration behind many of the components of this class.
  • Course Syllabus: provides the course-level objectives of the course, as well as my introduction of specs grading for students. This gives you a high-level view of the course structure.
  • Course Rubric: this provides a rubric for various types of questions given in the class. This is somewhat of a hybrid class as the questions are not graded individually pass/fail. Thesese rubrics give a sense of how students can improve on their work. Thanks to Dr. Sylvia Carlisle for the inspiration of these requirements.
  • Guidelines for Collaboration: in theoretical courses like this, I want students to work together, but I also want them to understand the difference between collaboration and academic misconduct. These are the guidelines I give my students. Thanks to Dr. Sylvia Carlisle for these suggestions.
• Bayesian Data Analysis
  With only a Probability pre-requisite, this course acts as both an introductory statistics course for a few majors on campus and an advanced course for those considering a career in statistics. This course is generally taken by 3rd and 4th year students and provides a good mix of by-hand and computational components. In its most recent delivery, I have converted to a pure ungrading framework in which students propose their grade in the course and justify their position based on a portfolio of work.
  • Course Syllabus: provides the course-level objectives of the course, as well as my introduction of ungrading for students. This gives you a high-level view of the course structure.
  • Portfolio: this provides a guide for students to reflect on their work in the course prior to proposing their course grade. This is submitted prior to having a conversation with me about their work; during this discussion, we agree on the course grade that will be assigned.
• Social Justice and Statistical Concepts
  This course was jointly developed and delivered with Dr. Jessica Livingston in the Department for Humanities, Social Sciences, and the Arts at Rose-Hulman. The course (profiled here) was designed to teach students the power of combining stories with data to tackle issues of inequity in our society. Requiring introductory statistics as a pre-requisite, the course introduced modeling concepts (e.g., analysis of complex surveys and interactions) alongside social justice concepts (e.g., intersectionality). As a way of modeling equitable frameworks in the course delivery itself, we opted to use an ungrading scheme in which students propose their grade in the course and justify their position based on a portfolio of work.
  • Course Syllabus: provides the course-level objectives of the course, as well as our introduction of ungrading for students. This gives you a high-level view of the course structure.
  • Capstone Project Criteria: the course built towards a single capstone project. Expecting most students to use this project to demonstrate their learning, clear expectations were provided similar to a specs graded system.
  • Portfolio: this provides a guide for students to reflect on their work in the course prior to proposing their course grade. This is submitted prior to having a conversation with me about their work; during this discussion, we agree on the course grade that will be assigned. We opted for a very simplistic reflection in this course given the amount of writing they had done previously.
• Capstone Experience
  The senior capstone experience could take the form of a project or thesis. I am grateful to Dr. Ella Ingram for her template on a contract for senior research that was the basis for the following syllabus.
  • Course Syllabus: this is a contract for students choosing to work on a seniore capstone experience with me. This also outlines the specifications for various grades; note that some components reflect department-wide requirements.
  • Capstone Committee Agreement: this document is for students to give to prospective committee members.