This book documents a data analysis workflow that might be useful for people in the Learning and Instruction Lab at ETH Zürich. It might also be useful for researchers in education research more generally, as well as anyone else who analyses behavioural data from human participants.
After conversations with several lab members — Martina R., Ipek, Charlotte, Martina V., and Christian — a set of common themes emerged across otherwise distinct projects. Each project used a broadly similar design, collected similar types of data, and faced similar questions about planning and analysis. Rather than repeat my comments and feedback, I decided to outline my thoughts here in the form of a general workflow. The hope would be that folks in the lab will find it a useful resource for planning experiments and analysing data.
The book follows the natural sequence of a study from planning to inference, split into two parts.
Part 1: Before data collection: how to use pilot data or simulations to justify a sample size (Planning), and what and how to preregister (Preregistration).
Part 2: After data collection: how to process and visualise data (Data processing), fit a multilevel Bayesian regression model (Modelling), and generate and communicate inferences (Inference).
This is deliberately a workflow document, not a methods textbook. The most important scientific decisions — framing the theoretical context, specifying research questions, choosing appropriate measures — are not covered here. Those decisions should come first, and nothing in this workflow substitutes for them.
All code is written in R and uses a tidyverse + CmdStan/brms stack. Slow-running chunks (model fitting, simulations) are pre-compiled; the book renders from saved outputs by default. See the reproducibility note in the README for details on refitting from scratch.
The data are simulated based on the structure and parameter estimates of a dataset collected by a lab member as part of their PhD research. The simulated data preserve the key features of the original — sample size, design structure, effect magnitudes, and variability — but contain no real participant data.
Simulated data are used for two reasons. First, they allow the book to be fully self-contained and publicly shareable without raising data privacy concerns. Second, because the true parameter values are known by construction, they make it easier to check that models are recovering what they should — a useful property when the goal is to outline a data analysis and modelling workflow.
This is a first draft version of the document, and therefore I welcome constructive feedback on things that could be improved.
© Rich Ramsey, ETH Zürich. Licensed under CC BY 4.0.
@book{RamseyliLabworkflow2026,
title = {A Workflow for Experiment Planning and Data Analysis in Education Research},
author = {Ramsey, Richard},
year = {2026},
month = {10},
edition = {version 0.1},
url = {https://rich-ramsey.github.io/li-lab-workflow/}
}Many thanks to Isabel Nigsch for helpful feedback on a previous version of this project.