Home - Centers and Institutes - Center for Learning, Teaching, and Research - Center for Learning, Teaching, and Research Updates
Center for Learning, Teaching, and Research Updates


Desirable Difficulty: Helping Students Obtain the Benefits of Retrieval Practice (“The Testing Effect”)

By Doug Johnson on January 27, 2016

A number of Colgate faculty (some examples below) have been taking advantage of empirical findings from applied cognitive science.  One class of findings, “Desirable Difficulties“, refers to conditions where a learner feels that an approach is difficult or ineffective, but that approach actually leads to better learning.  While at first this content area may seem to be oriented to the learner (our students), there are implications for us as faculty.

An example of a desirable difficulty is retrieval practice.  Students tend to like to “re-study” material versus testing themselves on it, as forcing themselves to try and retrieve information is effortful, not always successful, and feels premature.  The data, however, are clear that retrieval practice (whether self-testing, low-stakes quizzes, or “real” tests) has a powerful impact on learning (and not just memorization).  A comprehensive review of this literature may be found here (Roediger, H., & Karpicke, J., 2006), and the reference and resource section below has other related links.

Three Colgate faculty, Spencer Kelly and Neil Albert in Psychology, and Liz Marlowe in Art and Art History,  were kind enough to share with me how they use frequent assessment techniques in their courses.  As you will see, Spencer, Liz and Neil report other benefits to their approaches in addition to the learning effects predicted from the literature.

Spencer wrote:

In my Language and Thought and Cognitive Neuroscience seminars, I ask daily cumulative exam questions at the start of almost every class that test students’ comprehension of the day’s readings and their ability to integrate readings and discussions across the semester. The questions range from 5 minutes to 30 minutes, with most of them being around 8 to 10 minutes. These questions (after dropping the two lowest scores) are worth roughly 35% of the student’s total grade.
The main benefit is that students come prepared every day not only to discuss the readings for that day but to explore them in the context of everything that has come before them. This creates a coherence to the discussion that drives home (on a daily basis) the main points and themes of the class. Another benefit is that students get immediate feedback on where they are in the class, and this allows them to make necessary adjustments. Not to mention it helps me know when I also need to adjust!

Thus in addition to potential learning effects, Spencer notes his approach helps students come to class prepared, better integrate the information, and receive early and frequent feedback on their performance.  Spencer also uses the results of the frequent assessments to adjust his teaching during the semester, rather than waiting for end-of-semester Student Evaluations of Teaching (SETs) and adjusting his course the next time it is offered.

Liz uses a graded mini-exam every class period, and wrote:

Students in my ARTS 101 class answer an exam question during the first 10 minutes of class (75-minute period) almost every day, for a total of 22-24 questions over the course of the semester. I drop their lowest score. After the first week of classes, the questions are always cumulative and usually comparative, drawing on something from earlier in the semester plus either the reading from the night before or the previous lecture. For example, I might show a vase with a scene of a ritual offering from Mesopotamia that we studied in week 2 and a plaque with a scene of a ritual offering from Crete that we examined the day before (in week 5), and ask them something like “How do the differences in the iconography of these two scenes reveal larger differences between the respective cultures that created them?” This is a very standard type of question in art history, but usually students are only asked to think in this sort of integrative way twice a semester, at the midterm and at the final exam. Having them do this more frequently not only keeps all the material fresh in the active part of their brain (I leave it to my colleagues in Neuroscience to put that in more precise language), but also shows them on a daily basis what art history is for, how studying a culture’s art allows us to develop a very big picture of the various ways human society has been organized as well as the differing values and ideals that various cultures have embraced over time. In other words, they see they pay-off of this discipline in a relatively low-stress context every day, rather than just at high-stakes exams twice a semester. Having them think cumulatively (and put their cumulative thinking into writing) every day increases not only how much they learn in the course but also their understanding of the value of learning such material.

Like Spencer, Liz notes that the frequent assessments help students integrate information from various parts of the course, and to treat their learning as incremental and connected as opposed to packed knowledge acquisition and a “brain dump” during a high-stakes exam.

Neil also uses student work to adjust his teaching (“Just in Time Teaching”, see reference and resource section below), and wrote:

Much of the focus of my Cognitive Neuroscience seminar is on reading primary source material for a biweekly theme (e.g., perception or learning). Class discussions focus on describing conceptual frameworks, main findings and the degree to which papers complement and contradict each other – both within and across themes. For each article we read, students complete a brief article response that includes the main findings of the paper and its greatest strengths and weaknesses; article responses are submitted in Moodle and due a little over an hour before class, but are not graded. The primary impact of these short responses is that students have thought carefully about keys aspects of a paper before coming to class to discuss it – and they do so within a structured framework. I spend an hour or so before class going over these responses as I tailor my discussion plan based on the students’ mastery of the material. In class, we discuss these same aspects of each paper and students benefit from a reflective presentation of the strengths and weaknesses their peers have identified. Over the course of the semester, students become dramatically better readers and they are able to identify recurring strengths and weaknesses that occur across themes. Ultimately, students come to recognize the strengths and weakness of the relatively immature field of Cognitive Neuroscience and how to be a productive force in an imperfect world.

Thus, Neil’s approach to frequently asking students to reflect on the material indirectly requires retrieval practice by its very design, while also encouraging coming to class prepared and allowing the adjustment of his approach to the material based on where the students are struggling.

It is important to note that helping students utilize retrieval practice does not have to depend upon graded assignments nor use a lot of class time.  For another example, in my Psychology 309 course, I utilize a number of low-stakes quizzes that take place out of class and that are submitted electronically. Each student has a separate data set, so they can share concepts with each other, but each must do their own calculations and they cannot “check” answers with each other, since each will be unique.  Setting up the quizzes takes some time, but I can then use spreadsheets and a form plug-in in Google to quickly grade responses and provide feedback.

If you have been experimenting with more frequent assessments or finding other ways to enhance learning related to the “testing effect”, please do share your approach, as well as any consequences that you noticed (pro and/or con) in the comments section.

Happy teaching and learning.

References and Further Reading

An enormous THANK YOU to Professors Albert, Kelly and Marlowe!


Colgate Faculty:  If you have ideas for future posts related to teaching at Colgate, please contact Doug Johnson in the CLTR.


  • Doug Johnson said:

    Note: Just read a paper demonstrating the testing effect is effective with college students with Attention-Deficit/Hyperactivity Disorder….

    Knouse, L. E., Rawson, K. A., Vaughn, K. E., and Dunlosky, J. (2016). Does testing improve learning for college students with attention-deficit / hyperactivity disorder? Clinical Psychological Science, 4(1), 136-143.

  • Peter Tagtmeyer said:

    Nate Kornell in the psychology department at Williams College, a protégé of Robert Bjork continues to research the relationships between testing, learning and memory. His work can be found at http://sites.williams.edu/nk2/publications/

Leave a comment

Comments: Please make sure you keep your feedback thoughtful, on-topic and respectful. Offensive language, personal attacks, or irrelevant comments may be deleted. Responsibility for comments lies with each individual user, not with Colgate University. Comments will not appear immediately. We appreciate your patience.