Case study · Research
Reading cognitive overload from eye movements
When interfaces overwhelm people, performance drops and learning stops, and surveys only catch it after the fact. This project asked whether eye movements alone could flag the moment of overload, and grounded the answer in Cognitive Load Theory.
01 · The signal
Eyes are honest about mental strain
Ten oculomotor features feed a Random Forest that classifies each time segment as high or low-medium load. Saccade rate dominates at 39.4% importance with horizontal gaze dispersion at 21%, exactly the scanning signatures Cognitive Load Theory predicts for design-induced burden, while pupil metrics track the difficulty of the task itself.
Design preview of the dashboard rebuild. Metrics shown are from the published proof of concept.
>>> scipy.io.loadmat("colet_data_v3.mat") NotImplementedError: Please use HDF reader for matlab v7.3 files >>> from mat73_reader import load >>> load("colet_data_v3.mat")["data"] [1220 rows x 14 columns] # unlocked
02 · The wall, then the key
The training data was locked in an unreadable format
The COLET corpus ships as MATLAB v7.3 tables no Python tool could decode, so the original study trained on a synthetic stand-in that preserved the corpus statistics, stated plainly as its biggest limitation. Solving that blocker properly produced mat73-reader, and with it the path to training on the real 1,220 trials.
The tool that opened the format →03 · In motion
From proof of concept to publication
The revision retrains on the real COLET data, adds the theoretical grounding reviewers asked for, and targets a research venue alongside an interactive poster. After that: the same model on edge hardware, classifying load in real time from a camera feed.
The applications pull in three directions: usability tests that watch confusion happen instead of hearing about it afterward, tutoring systems that adapt before frustration wins, and monitoring for jobs where overload is dangerous.
Status
- Proof of concept complete: dashboard, model, paper
- Data unlocked: mat73-reader decodes the full corpus
- In progress: real-data retraining and paper revision
- Next: interactive poster, then on-device inference
The theory underneath
Three kinds of mental load, two of them measurable here
Cognitive Load Theory, from John Sweller’s work on why people learn poorly when problem-solving demands exceed working memory, splits mental burden three ways. The model’s feature importances map onto that split, which is why it begins to explain load rather than just flag it: confusion caused by design is distinguishable from genuine task difficulty.
Intrinsic load
The difficulty of the task itself. Calculus is harder than arithmetic no matter how it is presented. Pupil diameter tracks this kind of effort, and the model detects it even through a proxy measure.
Extraneous load
The burden added by poor presentation: the checkout button you cannot find. Rapid saccades and scattered gaze are its signature, and they carry 60% of the model's predictive weight.
Germane load
The productive effort of building lasting understanding. The goal of measuring the other two is protecting room for this one.
Does it generalize?
Tested on a corpus it never trained on
Beyond the held-out test set (5-fold cross-validation averaged 0.746 before the final 0.897), the model processed recordings from EM-COGLOAD, a separate corpus of 276 recordings from 52 participants doing different tasks, treated as unlabeled to simulate real deployment. It surfaced three distinct workload signatures, each matching what the task history said should be there.
Gradual climb
11 high-load episodes, 55% average
A classic learning curve: green start, orange middle, red finish as task difficulty escalated.
Sustained calm
0 episodes, 10.5% average
A comfortable participant end to end. The model's feature weights adapted, with saccade importance dropping to 18%, consistent with calmer eyes on easier tasks.
Intermittent spikes
6 episodes, peaking at 100%
Intense focus on hard subtasks with recovery between, which is how real work actually feels.
By the numbers
01234567890123456789.012345678901234567890123456789012345678901234567890123456789
macro F1
on the held-out test set for high-load detection
01234567890123456789.0123456789012345678901234567890123456789
target
the bar the project set before training
0123456789012345678901234567890123456789
oculomotor features
velocity, fixation, saccade, dispersion, and pupil metrics
0123456789012345678901234567890123456789
participants
in the COLET corpus the revision trains on
The finding
The model leans hardest on saccade rate and gaze dispersion, exactly the signatures the theory predicts. We taught a computer to recognize the visual pattern of an overwhelmed mind.