Precision and recall

Description

A true changepoint is declared “detected” (or positive) if there is at least one computed changepoint at less than “margin” points from it. Formally, assume a set of change point indexes \(t_1,t_2,\dots\) and their estimates \(\hat{t}_1, \hat{t}_2,\dots\) In the context of change point detection, precision and recall are defined as follows:

\[\text{precision}:=|\text{TP}|/|\{\hat{t}_l\}_l| \quad \text{and}\quad\text{recall}:=|\text{TP}|/|\{t_k\}_k|\]

where, for a given margin \(M\), true positives \(\text{TP}\) are true change points for which there is an estimated one at less than \(M\) samples, i.e

\[\text{TP}:= \{t_k\,|\, \exists\, \hat{t}_l\,\, \text{s.t.}\, |\hat{t}_l - t_k|<M \}.\]

Schematic example: true segmentation in gray, estimated segmentation in dashed lines and margin in dashed areas. Here, precision is 2/3 and recall is 2/2.

Usage

Start with the usual imports and create two segmentations to compare.

from ruptures.metrics import precision_recall
bkps1, bkps2 = [100, 200, 500], [105, 115, 350, 400, 500]
p, r = precision_recall(bkps1, bkps2)
print((p, r))

The margin paramater \(M\) can be changed through the keyword 'margin' (default is 10 samples).

p, r = precision_recall(bkps1, bkps2, margin=10)
print((p, r))
p, r = precision_recall(bkps1, bkps2, margin=20)
print((p, r))

Code explanation

ruptures.metrics.precisionrecall.precision_recall(true_bkps, my_bkps, margin=10)

Calculate the precision/recall of an estimated segmentation compared with the true segmentation.

Parameters

• true_bkps (list) – list of the last index of each regime (true partition).

• my_bkps (list) – list of the last index of each regime (computed partition).

• margin (int, optional) – allowed error (in points).

Returns

(precision, recall)

Return type

tuple