Glossary

Every technical term this site uses, in plain words. Where a term has a number attached to it in my own work, that number is the example.

Bootstrap (and why “clustered”)

A way of asking how much an answer wobbles. You resample your own data thousands of times, recompute the result each time, and look at the spread. Clustered means resampling whole markets rather than individual trades, because two trades in the same market are not independent of each other. Treating them as independent would make a result look far more certain than it is.

The strategy results use 2,000 clustered resamples with the market as the unit.

Credible interval

The range a true value plausibly sits in. It is an honesty device: it says how precise the estimate is not. Read one out loud rather than skipping it — “[3,256, 23,292]” means somewhere between about three thousand and twenty-three thousand, and no more precise than that.

Dwell time

How long the market stays in one mode before switching to the other.

About 37 minutes in the quiet mode, about 8 minutes in the busy one.

Friedman test

A way of comparing several methods across many test cases at once, while allowing for the fact that some test cases are simply harder than others. It answers “is any of these genuinely better?” rather than “which scored highest?”.

Across 22 people, six activity-recognition models gave p = 0.296 — the ranking between them is indistinguishable from noise.

Hawkes process (self-exciting)

A model where every event makes the next one more likely for a while — earthquakes and aftershocks. Nothing hidden is driving it. The activity is its own cause.

The central question of my thesis was whether the market has a hidden mood that turns busy before the prices move, or whether price moves simply cause more price moves. The answer was the second.

Hazard rate

The chance that something ends in the next minute, given that it has not ended yet. A rising hazard means the longer it has run, the sooner it will stop. A falling hazard means the opposite.

Mine falls, from 0.325 to 0.167 per minute. The longer a burst of activity has already lasted, the more likely it is to keep going. Young bursts die young.

Held out (out-of-sample)

Tested on data the model never saw while it was being built. A model checked against the data it was fitted to will always look good; only held-out results tell you anything.

Isotonic calibration

Adjusting a model so its confidence means what it says: when it claims 70% confidence it is right about 70% of the time. A model can rank things well and still be badly calibrated.

Leave-one-out

Train on everything except one case, test on the case you left out, then repeat for every case in turn. Nothing is ever tested on itself.

For the wearable-sensor work the unit left out is a person, which makes the question “does this work on someone it has never seen?” — the only question that matters for a product. For the market work the unit is a market.

Macro-F1

An accuracy score that treats a rare category as just as important as a common one. It stops a model scoring well by quietly ignoring everything unusual.

It is the headline number in my BSc thesis, where six models scored between 0.843 and 0.880.

MMPP (Markov-modulated Poisson process)

A model in which the market flips between a quiet mode and a busy mode, and you cannot see which mode it is in. You only see the price moves, and have to infer the mode from them — like working out whether a room is in a meeting or on a coffee break by listening through the door.

Mine recovered a real quiet/busy split. It just could not use it to predict anything.

Nats (log-evidence)

A unit of how unsurprised a model was by data it had never seen. Higher is better. The absolute number means little; the comparison between two models on the same data is the point.

Partial pooling improved this on all 34 of 34 markets, by a total of 11,400 nats.

Order book, and L2 depth

The list of buy and sell offers resting at each price, waiting to be filled. L2 depth is how much is stacked up at each level. It says where the pressure is, which a single quoted price does not.

My historical data does not contain it. One strategy needed it, so it refused to run rather than substitute a rough stand-in — which is why its result is “not applicable” instead of a number.

p-value

If there were genuinely no effect at all, how often would pure luck alone produce a result at least this good? A small p-value means luck is an unlikely explanation. A large one means there is nothing here.

My strategies came out at p = 0.498, 0.601 and 0.613. The first is a coin flip; none is evidence of anything.

Partial pooling (hierarchical model)

Letting each market learn from all the other markets instead of being estimated alone. A market with barely any data borrows the shape of the typical market, so its estimate is sensible rather than wild. See also shrinkage.

It improved held-out prediction on every one of 34 markets.

Point process

The branch of maths for “when do things happen in time” — earthquakes, buses, price moves. You model the timing of events rather than their size.

Poisson process

The simplest point process: events arrive at random, at a steady average rate, and each one is independent of the last. It is the baseline that more interesting models have to beat.

Pre-registration

Writing down exactly what you are going to test, and what would count as success, before you look at the answer — and committing it to version control so the timestamp proves the order. It is what stops you moving the goalposts after seeing the result, because moving them afterwards would be visible in the history.

My thesis carries 14 pre-registered components. It is the reason I can report that my own primary test failed rather than quietly re-cutting the measurement until something passed.

Prediction market

A market where you buy a contract that pays $1 if some event happens and nothing if it does not. A contract trading at 63¢ means the crowd collectively thinks there is roughly a 63% chance. The price is the forecast.

Repricing event

A moment when the market’s price actually moves. The raw material of this whole project is not what the price is, but when it changes.

Self-supervised learning

Let a model learn the structure of a mountain of unlabelled data first, then fine-tune it on the small labelled set you actually have. Labels are expensive; raw data usually is not.

Three of the six models I benchmarked work this way.

Shrinkage

How hard a pooled model pulls a noisy estimate toward the group average. Estimates built on very little data should be pulled hard; estimates built on a lot should barely move.

Mine pulls the data-poor markets 40.5 times harder than the data-rich ones, which is exactly the behaviour you want.

Truncation artifact

A measurement error caused by which cases you were able to see in the first place. If you only survey people who have already been at a party for twenty minutes, you will conclude that nobody ever leaves early — not because it is true, but because the early leavers were never in your sample.

This is exactly what happened to me. A model said bursts of activity get more likely to end as they age. It was an artifact of only counting bursts above a minimum length, so I rejected it and rebuilt the result without that floor.

Walk-forward testing

Only ever train on the past and test on the future, stepping forward through time. It is the honest way to test a trading idea, because it never lets the model see information that would not have existed yet.

Weibull

A standard mathematical shape for how the risk of something ending changes as it gets older. Its shape parameter says whether that risk rises or falls with age.

Mine came out saying the risk rises. That was a truncation artifact, and the corrected result says the opposite.

XGBoost

A pile of simple decision rules, each one built to correct the mistakes of the ones before it. Well understood, unfashionable, and often extremely hard to beat.

In my benchmark it was statistically tied with a transformer that took eighteen times longer to train — about 23 minutes against about 7 hours.

λ (lambda) — the rate

How often something happens, in events per minute. Always readable as a waiting time: a rate of 1 per minute is one event every 60 seconds.

The quiet mode runs at 0.041 moves per minute — one price move about every 25 minutes. The busy mode runs at 1.141 — one about every 53 seconds.