Case study
mat73-reader: opening MATLAB’s undocumented table format
A research dataset was locked inside a file format no Python tool could read. Instead of hacking around it, I reverse-engineered the format and shipped the fix for everyone.
The format
Where the table hides
A v7.3 file is HDF5, so h5py opens it happily. Then the trail goes cold: the table variable is a group holding opaque integer references into a hidden group called #refs#, where MCOS, MATLAB’s internal object system, serialized the real data. Which reference holds the column names, which holds the types, and which holds the payload is the undocumented part.
mat73-reader implements that mapping. It follows the references, reassembles the columns, and hands back a pandas DataFrame. For everything MCOS does not touch, the right tools already exist, and the README says so: scipy for pre-7.3 files, mat73 for plain v7.3 arrays. This library exists for the one case nothing else covers.
>>> f = h5py.File("colet_data_v3.mat") >>> list(f.keys()) ['#refs#', 'data'] >>> f["data"] <HDF5 group (2 members)> # not your table >>> f["data"]["MCOS"] <HDF5 dataset, uint32> # opaque object refs
01 · The wall
Every existing tool fails on MATLAB tables
MATLAB v7.3 stores tables through MCOS, an undocumented internal object system layered over HDF5. scipy raises. The mat73 library returns None. The data is right there in the file, and nothing in the Python ecosystem can reach it.
>>> scipy.io.loadmat("colet_data_v3.mat") NotImplementedError: Please use HDF reader for matlab v7.3 files >>> mat73.loadmat("colet_data_v3.mat")["data"] None # table object silently dropped
>>> from mat73_reader import load >>> data = load("colet_data_v3.mat") >>> data["data"].head(3) participant trial task load_label 0 1 1 1 high 1 1 2 1 low 2 1 3 2 medium >>> # pandas DataFrame, straight from MCOS
02 · The key
Reverse-engineered, then made ordinary
The library decodes MCOS table structures into pandas DataFrames with one call, and handles the numeric arrays, structs, cells, and character data around them. A CLI covers inspection and conversion to CSV or JSON for people who never want to leave the shell.
03 · Built to outlive its origin
The tool knows nothing about eye tracking
The dataset that motivated it is a cognitive science corpus, and the library contains not one line about it. Researchers in neuroscience, signal processing, and physics hit the same wall; the fix is domain-free by design. Apache 2.0, on PyPI, with an honest comparison table pointing at the right tool for every other .mat flavor.
The research it unlocked →$ mat73-reader inspect colet_data_v3.mat colet_data_v3.mat (MATLAB v7.3, HDF5) └─ data: table [1220 rows x 14 cols] MCOS └─ meta: struct {participants: 47, tasks: 4} $ mat73-reader convert colet_data_v3.mat --to csv wrote colet_data_v3.csv
By the numbers
012345678901234567890123456789012345678901234567890123456789
tables decoded
from the dataset that motivated the tool
01234567890123456789.01234567890123456789
GB validated
a full research corpus, end to end
0123456789012345678901234567890123456789
tests
across standard operations and edge cases
01234567890123456789
alternatives
no other Python tool decodes MATLAB tables
Why it matters
A blocker can be a workaround or a contribution. This one became the only tool of its kind, and the research that needed it is back in motion.Read the code →