Improving Students’ Study Practices Through the Principled Design of Research Probes

A key challenge of the learning sciences is moving research results into practice. Educators on the front lines perceive little value in the outputs of education research and demand more “usable knowledge”. This work explores the potential instead of usable artifacts to translate knowledge into practice, adding scientists as stakeholders in an interaction design process. The contributions are two effective systems, the scientific and contextual principles in their design, and a research model for scientific research through interaction design.

College student study practices are the domain chosen for the development of these methods. Iterative ethnographic fieldwork identified two systems that would be likely to advance both learning in practice and knowledge for applying the employed theories in general. Nudge was designed to improve students’ study time management by regularly emailing students with explicit recommended study activities. It reconceptualizes the syllabus into an interactive guide that fits into modern students' attention streams. Examplify was designed to improve how students learn from worked example problems by modularizing them into steps and scaffolding their metacognitive behaviors though problem-solving and self-explanation prompts. It combines these techniques in a way that is exceedingly easy to author, using existing answer keys and students' self-evaluations.

Nudge and Examplify were evaluated experimentally over a full semester of a lecture-based introductory chemistry course. Nudge messages increased students’ sense of achievement and interacted with students’ existing time management skills to improve exam grades for poorer students. Among students who could choose whether to receive them, 80% did. Students with access to Examplify had higher exam scores (d=0.26), especially on delayed measures of learning (d=0.40). A key design decision in Examplify was not clearly resolvable by existing theory and so was tested experimentally by comparing two variants, one without prompts to solve the steps. The variant without problem solving was less effective (d=0.77) and less used, while usage rates of the variant with problem solving increased over time.

These results support the use of the design methods employed and provide specific empirical recommendations for future designs of these and similar systems for implementing theory in practice.