LevijoyResults
How well does it actually work?
We'd rather show you the honest numbers than the flattering ones. Here's how Levijoy performs across real competition footage — including footage of horses and riders it had never seen before.
Evaluated tier by tier, not just on its best day.
Levijoy was scored by F1 score — a single number that balances catching real jumps against not raising false alarms. The “Development” tier is where thresholds were tuned, so treat it as a best case. The two hold-out tiers, on riders and horses the system had never encountered, are the more honest measure of how it performs on a yard’s own footage.
| Tier | Detail | Videos | Precision | Recall | F1 score |
|---|---|---|---|---|---|
| Early prototype | SAM2 approach, since abandoned | 8 | 37.3% | 21.6% | 27.3% |
| Development | in-sample tuning | 5 | 69.0% | 89.1% | 77.8% |
| Partial hold-out | unseen riders | 4 | 58.3% | 60.9% | 59.6% |
| Full hold-out | fully unseen | 6 | 48.9% | 69.7% | 57.5% |
The early prototype (shown for context, not in use today) tried to outline the horse’s silhouette frame by frame. Switching to tracking three fixed points on the body roughly tripled F1 — see how it works for why.
What drives accuracy
Tracking quality is the real bottleneck.
On footage where the horse was tracked reliably more than 65% of the time, Levijoy reached 74.5% F1. Below 65% reliable tracking, that dropped to 52.3%. In other words: the detection logic itself is solid — the biggest lever for improving results further is clearer footage and better pose-tracking, not a smarter detector.
- F1 gap between good and poor tracking quality
- 22.2pp
Where it still struggles
We'd rather show you the failure cases than hide them.
Three patterns account for most of the errors we see today — each one is a concrete target for the next round of improvement.
Missed jump (rear view)
Jumped straight towards or away from the camera, the vertical motion barely registers.
False positive (label switch)
In a crowded arena, tracking briefly jumps onto another horse and reads it as motion.
Ambiguous peak (canter stride)
An unusually pronounced canter stride occasionally clears the bar for a real jump.
Scope, honestly stated
What Levijoy is — and isn't — today.
Viewpoint sensitivity
Rear- and front-on jumps compress the vertical motion the system relies on, accounting for around 10% of missed jumps.
Depends on tracking quality
Footage with poor lighting, low resolution, or difficult angles gives the pose-tracking model less to work with, and accuracy drops with it.
Trained on a modest dataset
25 videos across 5 horses, all from one country. It generalizes well within that scope, but hasn't yet been tested at wider scale.
Counts jumps — nothing more, yet
Levijoy currently detects that a jump happened and when. It doesn't yet identify which fence, whether there was a fault, or a competition score.