Performance¶
Measured with benchmarks/performance.py, which runs the
same series and parameters against this version, the previous release
(1.1.0), the first PyTorch release (1.0.0) and the original NumPy
implementation (0.4), each in a fresh process, and scores every run against
the true changepoints (F1, margin 5). Apple M1, CPU, 4 threads, PyTorch 2.14,
Python 3.12; median of up to five runs. Raw results, with more sizes:
benchmarks/results/2026-09-23-apple-m1-cpu.json.
| Workload | 1.2.2 | 1.2.0 | 1.1.0 | 0.4 (NumPy) | 1.0.0 (PyTorch) |
|---|---|---|---|---|---|
Offline StudentT, 1 000 points |
0.61 s | 0.53 s | 3.1 s | 24 s | 108 s, misses changes (F1 0.50) |
Offline StudentT, 2 000 points |
2.3 s | 2.0 s | 24 s | 101 s | not run (predicted 426 s) |
Offline MultivariateT, 5-D, 1 000 points |
0.97 s | 0.94 s | 3.4 s | not available | 30 s |
Online StudentT, 1 000 points |
0.14 s | 0.16 s | 0.13 s | 0.12 s | 39 s |
Online StudentT, 5 000 points |
1.5 s | 1.9 s | 1.8 s | 2.0 s | not run (predicted 825 s) |
Online MultivariateT, 5-D, 1 000 points |
0.34 s | 0.40 s | 0.37 s | crashes (NameError) |
51 s, wrong (F1 0.04) |
OnlineChangepointDetector, 50 000 points, max_run_length=1000 |
7.2 s (144 µs per point) | 7.9 s (160 µs per point) | not available | not available | not available |
The 1.2.2 column is a later run of the same benchmark on the same machine
(benchmarks/results/2026-11-05-apple-m1-cpu.json);
the other columns come from one earlier file, so compare the 1.2.2 column with
them only as a guide. The offline code did not change in 1.2.2, so its
differences in the offline rows are machine load, not a slowdown.
1.2.0 and 1.2.2 find every change in each of these series (F1 1.00) except the streaming one (F1 0.94, 199 changes). In short: the offline detector is 31–51x faster than the NumPy original, 206–360x faster than 1.0.0 (which also misses changes) and faster than 1.1.0 by a factor that grows with length (1.5x at 250 points, 6x at 1 000, 12x at 2 000); the online detector runs at the speed of the NumPy original (both are a Python loop over time), and its multivariate model is correct only since 1.1.0.
Detection quality on real data, measured with benchmarks/tcpd.py
on the Turing Change Point Dataset (van den Burg and Williams, 2020; 30
annotated real series, TCPDBench's F1 and covering metrics, higher is
better):
| Method, default settings | F1 | cover |
|---|---|---|
| No changepoints at all (baseline) | 0.668 | 0.575 |
BOCPD as published by TCPDBench (R package ocp) |
0.696 | 0.636 |
This library, online, viterbi_changepoints |
0.694 | 0.637 |
| This library, offline | 0.739 | 0.664 |
The online model reproduces the published BOCPD (identical F1 on 27 of the
31 series both score, which add the 2-D run_log to these 30; also with
tuned settings, 0.887 against 0.890); the offline
detector beats it without tuning. For a finished series, use the offline
detector, or read the online posterior with viterbi_changepoints. The
online readout get_map_changepoints reports changes as data arrive,
without hindsight, and scores 0.571 F1 on the same series.
Complexity: the online recursion is O(T²) in time and memory (the
run-length posterior R is (T+1)² float32); OnlineChangepointDetector
with max_run_length=K is O(K) per observation. The offline recursion is
O(T²) (vectorized per start point), and so, in practice, is the table of
changepoint locations Pcp: it is O(J T²) for J rows, and rows stop once
the probability of that many changepoints drops below exp(-1000) (about
190 rows for a series with three clear changes, whatever its length; up
to 19x faster than 1.1.0 at 4 000 points); memory is about
16 T² bytes (see docs/devices.md).
Accelerators: see the FAQ; MPS is slower than the CPU on all of these, CUDA is unmeasured (issue #43). Only measured numbers appear in this README.