Fruit Metric (Average Precision, AP)

Task provided by the Scientific and Educational Center for Cognitive Modeling, MIPT

A small educational project: calculating Average Precision (AP) for the task of classifying fruits as Ripe / Unripe at fixed probability thresholds from 0.0 to 1.0 (exclusive) with step 0.1.

Formulas

Precision at threshold t
$Precision(t) = \frac{TP(t)}{TP(t) + FP(t)}$,

where $TP(t)$ is the number of true positives at threshold $t$,
and $FP(t)$ is the number of false positives.

Average Precision

$$ AP = \frac{1}{n} \sum_{i=1}^{n} Precision(t_i), \quad t_i \in {0.0, 0.1, \ldots, 0.9} $$

Data

The file data.csv contains 10 observations with columns:

• id - photo number
• probability - probability of the Ripe class
• label - ground truth label (Ripe or Unripe)

Usage

py fruit_metric.py --csv data.csv

The script will output a Threshold,Precision table and the final AP value.

It is assumed that if probability >= threshold, then the prediction is Ripe.

Notebook

The notebook main.ipynb shows the same calculation step by step with a small visualization of the Precision vs threshold plot.