Case study · Research
Reading cognitive overload from eye movements
When interfaces overwhelm people, performance drops and learning stops, and surveys only catch it after the fact. This project asked whether eye movements alone could flag the moment of overload, and grounded the answer in Cognitive Load Theory.
01 · The signal
Eyes are honest about mental strain
Twelve oculomotor features feed a Random Forest that classifies each task segment as high or low load. On the real corpus, with within-person normalization and participant-grouped validation, pupil dynamics lead and the importances spread across the feature set. The tidy story the synthetic proof of concept told, saccade rate dominating at 39.4%, did not survive contact with real data; its importance fell to 0.108, and that failed replication became a finding in its own right.
Design preview of the dashboard rebuild. Metrics shown are from the published proof of concept.
>>> scipy.io.loadmat("colet_data_v3.mat") NotImplementedError: Please use HDF reader for matlab v7.3 files >>> from mat73_reader import load >>> load("colet_data_v3.mat")["data"] [1220 rows x 14 columns] # unlocked
02 · The wall, then the key
The training data was locked in an unreadable format
The COLET corpus ships as MATLAB v7.3 tables no Python tool could decode, so the original study trained on a synthetic stand-in that preserved the corpus statistics, stated plainly as its biggest limitation. Solving that blocker properly produced mat73-reader, and with it the path to training on the real 1,220 trials.
The tool that opened the format →03 · In motion
From proof of concept to publication
The rebuild retrained on the real COLET data and the numbers moved: honest grouped validation lands near 0.73, not the synthetic 0.897, and that gap became the paper’s subject. The revision adds the theoretical grounding reviewers asked for and targets a research venue alongside an interactive poster. After that: the same model on edge hardware, classifying load in real time from a camera feed.
The applications pull in three directions: usability tests that watch confusion happen instead of hearing about it afterward, tutoring systems that adapt before frustration wins, and monitoring for jobs where overload is dangerous.
Status
- Proof of concept complete: dashboard, model, paper
- Data unlocked: mat73-reader decodes the full corpus
- Rebuild done through label robustness; protocol freeze next
- Next: paper on the synthetic-real gap, interactive poster
The theory underneath
Three kinds of mental load, two of them measurable here
Cognitive Load Theory, from John Sweller’s work on why people learn poorly when problem-solving demands exceed working memory, splits mental burden three ways. The model’s feature importances map onto that split, which is why it begins to explain load rather than just flag it: confusion caused by design is distinguishable from genuine task difficulty.
Intrinsic load
The difficulty of the task itself. Calculus is harder than arithmetic no matter how it is presented. Pupil diameter tracks this kind of effort, and the mental-demand subscale is where the model scores highest (F1 0.751).
Extraneous load
The burden added by poor presentation: the checkout button you cannot find. Theory says rapid saccades and scattered gaze are its signature; on real data that signature is present but far weaker than the synthetic stand-in suggested.
Germane load
The productive effort of building lasting understanding. The goal of measuring the other two is protecting room for this one.
What the real data changed
The synthetic stand-in was hiding the story
The proof of concept trained on a synthetic substitute because the real corpus was locked in an unreadable format. Once mat73-reader opened the actual recordings, the rebuild reran everything with participant-grouped validation and label-robustness checks across seven labeling schemes. Some claims fell, one survived, and the gaps between the synthetic and real results are now the subject of the paper.
The inflation
0.897 fell to 0.665
The proof of concept's headline was measured on a synthetic stand-in. Honest validation on the real corpus starts at 0.665, and the decomposition experiment showed the gap is the data itself, not the split: the stand-in fabricated structure that real recordings do not have.
The artifact
saccade importance 0.394 to 0.108
The feature story that made the theory mapping look clean was a property of the synthetic data. On real recordings the importances flatten and pupil dynamics lead.
The survivor
0.751 mental, 0.497 physical
Within-person pupil change carries real signal. The model scores highest on the mental-demand subscale and near chance on physical demand, the negative control a cognitive model should fail.
By the numbers
01234567890123456789.012345678901234567890123456789012345678901234567890123456789
macro F1
participant-grouped cross-validation on the real corpus
01234567890123456789.012345678901234567890123456789012345678901234567890123456789
mental demand
the strongest subscale target, where a cognitive model should lead
01234567890123456789.012345678901234567890123456789012345678901234567890123456789
physical demand
near chance on the subscale a load model should fail: the negative control
012345678901234567890123456789012345678901234567890123456789
labeled instances
47 participants by 4 tasks, decoded from the real COLET corpus
The finding
The synthetic stand-in told a tidy story: saccades dominate, theory confirmed. The real data tells a subtler one, and the gap between those two stories is what the paper is about.