BSc thesis · Leiden (LIACS) · 2025

Self-supervised models for human activity recognition

The work

A full self-supervised learning pipeline benchmarking six architectures for recognising physical activity from a wrist sensor.

Given movement data from a wearable sensor, can a model tell walking from cycling from sitting? I benchmarked six approaches, from a twenty-year-old method to a modern transformer, three of them self-supervised, on the same dataset with the same protocol. Each was tested on people it had never seen during training, which is the only version of the question that matters if you want to ship something.

The scores ranged from 0.880 down to 0.843, which looks like a ranking. It is not one. A statistical test designed for exactly this comparison says the differences are indistinguishable from chance, so I did not run the follow-up test that would have ranked them — there was nothing to rank. The practical consequence is the interesting part: the state-of-the-art model the project was built around took about seven hours and finished fourth, while a conventional method took about twenty-three minutes and was statistically tied with it.

The self-supervised pipeline A model learns structure from a large pile of unlabelled sensor data, is then fine-tuned on the small labelled set, and is finally tested on people it has never seen. learn the shape of the data without labelsunlabelled sensor datapretraining labels are expensive; only a small set existssmall labelled setfine-tuning tested on a person never seen in training — the only question that mattersheld-out personleave-one-subject-out repeated for each of 22 people in turn, then pooled six architectures compared this way: SimCLR · DeepConvLSTM · SelfHARModel · SelfPAB · XGBoost · MOMENT spread 0.880 to 0.843 — a Friedman test says that ordering is noise
What you're looking at: how each of the six models was trained and judged. Three of them learn from unlabelled sensor data first, then fine-tune on the small labelled set. All six are finally tested on a person held out of training entirely, which is the only version of the question that matters for a product. Protocol · HARTH v1.2, 22 subjects · data 2025
Six architectures under leave-one-subject-out cross-validation Pooled macro-F1 spans 0.843 to 0.879, but the per-subject ranges overlap almost completely. A Friedman test over 22 subjects gives p = 0.296, so the ranking is not statistically significant. SimCLR SimCLR: pooled macro-F1 0.8795; per-subject range 0.362–0.944 0.879 DeepConvLSTM DeepConvLSTM: pooled macro-F1 0.8706; per-subject range 0.372–0.94 0.871 SelfHARModel SelfHARModel: pooled macro-F1 0.8622; per-subject range 0.441–0.903 0.862 SelfPAB SelfPAB: pooled macro-F1 0.8601; per-subject range 0.332–0.922 0.86 XGBoost XGBoost: pooled macro-F1 0.8509; per-subject range 0.295–0.881 0.851 MOMENT MOMENT: pooled macro-F1 0.8432; per-subject range 0.425–0.876 0.843 0.3 0.5 0.7 0.9 accuracy score, higher is better ● overall · one person Friedman χ² = 6.104, df = 5, p = 0.296 — not significant

Chart scrolls sideways on a narrow screen — or open “Show the numbers” below.

Show the numbers
modelpooled macro f1mean per fold macro f1min foldmax foldmean friedman rank
SimCLR0.87950.76090.36190.9442.636
DeepConvLSTM0.87060.74640.37190.94033.591
SelfHARModel0.86220.74720.44080.9033.5
SelfPAB0.86010.73770.33210.9223.727
XGBoost0.85090.73920.29540.88083.864
MOMENT0.84320.73340.42470.87573.682
What you're looking at: six models, one row each. The large dot is the model's overall score; every small dot is one person it was tested on. The rows overlap almost completely, and the spread within any one model dwarfs the gap between models — which is why the apparent ranking does not survive a statistical test. If the models really differed, the rows would separate. n = 22 subjects × 6 models on HARTH v1.2 (a public wearable-sensor dataset) · data 2025 · not pre-registered · held-out, leave-one-subject-out · models-macro-f1.csv, models-per-fold.csv
Per-class F1 across six architectures Every architecture fails on the same two minority classes, which is the class imbalance the thesis records as its persistent limitation. c1 4,862 c2 1,174 c3 769 c4 662 c5 1,999 c6 3,034 c7 11,612 c8 1,719 SimCLR c1: 0.9180.92 c2: 0.9730.97 c3: 0.5260.53 c4: 0.7940.79 c5: 0.9630.96 c6: 0.8870.89 c7: 0.9930.99 c8: 0.9830.98 DeepConvLSTM c1: 0.9010.9 c2: 0.9710.97 c3: 0.5030.5 c4: 0.7870.79 c5: 0.9650.96 c6: 0.8530.85 c7: 0.9940.99 c8: 0.9910.99 SelfHARModel c1: 0.8980.9 c2: 0.9780.98 c3: 0.460.46 c4: 0.7780.78 c5: 0.9660.97 c6: 0.8630.86 c7: 0.9880.99 c8: 0.9670.97 SelfPAB c1: 0.8860.89 c2: 0.9790.98 c3: 0.4960.5 c4: 0.7380.74 c5: 0.9620.96 c6: 0.870.87 c7: 0.990.99 c8: 0.960.96 XGBoost c1: 0.9080.91 c2: 0.980.98 c3: 0.4220.42 c4: 0.6850.69 c5: 0.9430.94 c6: 0.8950.89 c7: 0.9930.99 c8: 0.9810.98 MOMENT c1: 0.9450.95 c2: 0.9930.99 c3: 0.5730.57 c4: 0.9040.9 c5: 0.9670.97 c6: 0.8370.84 c7: 0.9380.94 c8: 0.5890.59 column = activity class, with its sample count below the label bar length = F1 for that class; full width would be 1

Chart scrolls sideways on a narrow screen — or open “Show the numbers” below.

Show the numbers
classsupportSimCLRDeepConvLSTMSelfHARModelSelfPABXGBoostMOMENT
148620.91780.90070.89810.88580.9080.9454
211740.97280.97130.97750.97880.97970.9928
37690.52630.50280.45950.49650.42240.5725
46620.79410.78750.77830.73810.68510.9041
519990.96330.96460.96570.9620.94340.9671
630340.88660.85330.86340.86980.89460.8368
7116120.99280.9940.9880.98980.9930.9383
817190.98250.99050.9670.960.98130.5889
What you're looking at: how well each model handles each type of activity. Columns are activities, with how many examples each had underneath. Every model collapses on the same two columns — the two with the fewest examples. That is a property of the data, not of any one model, and it is the limitation the thesis records. 8 classes, 25,831 pooled samples · data 2025 · not pre-registered · held-out, pooled across folds · models-per-class-f1.csv

Read the full BSc thesis (PDF, 4.6 MB, 30 pages). The timings above are from its results chapter.

The technical version

The load-bearing finding is negative

Macro-F1 (an accuracy score that weights rare activities equally) under leave-one-subject-out cross-validation on HARTH v1.2 — a public research dataset of wearable-sensor recordings — across 12 activity classes: SimCLR 0.880, DeepConvLSTM 0.871, SelfHARModel 0.862, SelfPAB 0.860, XGBoost 0.851, MOMENT 0.843. A Friedman test failed to reject the null at α = 0.05; the Nemenyi post-hoc was therefore not run. The thesis states the spread "could reasonably be attributed to random chance."

Recomputed, not quoted — tools/make_figures.py re-derives the test from the committed per-fold logs: χ² = 6.104, df = 5, p = 0.296. Note the headline scores are the pooled report across all folds; the mean of the per-fold scores is a different and lower quantity (SimCLR 0.761), and the two are not interchangeable.

The practical finding

SelfPAB cost about 7 hours for the full cross-validation against 1.7 hours for SimCLR and 23 minutes for XGBoost, and finished fourth. Recorded honestly rather than claimed: SelfPAB scored 0.860 here against 0.813 in its original paper on the same dataset, attributed to setup variation and not claimed as an improvement. Known persistent limitation: class imbalance.