EP01_03 — 📈 Mean Average Precision (mAP) — Precision-Recall Curve

1.18.6.1 🧠 Why is mAP the standard metric?

In EP01_02, you saw that the choice of threshold significantly alters Precision and Recall. The mAP (Mean Average Precision) addresses this: it evaluates the model at multiple thresholds (each threshold should generate a different confusion matrix) and summarizes performance by the area under the Precision-Recall (P-R) curve.

While the F1-Score examines a single equilibrium point, mAP considers the entire curve. The closer to 1.0, the better the detector across all thresholds and classes (e.g., coins of 25, 50, and 1 real).

References: Roboflow — mAP · Explanatory video

1.18.6.2 🔢 How mAP is calculated — step by step

1. Fixed thresholds (always use this list):

   limiares = [0.00, 0.09, 0.21, 0.31, 0.39, 0.52, 0.60, 0.71, 0.81, 0.89, 1.00]

2. For each threshold (t), classify the samples: predito = 1 if confiança ≥ t, else 0.
   Compute TP, FP, FN, TN and obtain Precision((t)) and Recall((t)).

3. Build the P-R curve: pairs (Recall((t)), Precision((t))), ordered by increasing Recall.

4. Monotonize Precision: \[P_{\text{mono}}[i] = \max_{j \ge i} P[j]\]

5. Compute the AP (area under the monotonic curve) using the trapezoidal rule (a more accurate approximation than the simple Riemann sum): \[AP = \sum_{i=1}^{m-1} \frac{P_{\text{mono}}[i-1] + P_{\text{mono}}[i]}{2} \cdot (S[i] - S[i-1])\]

6. mAP = average of the APs across all classes. In this assignment, there is only 1 class, so mAP = AP.

1.18.6.3 📋 Task

Read an integer n (number of samples). Then read n lines, each containing: true (0 or 1) and confidence (float 0.0–1.0).

Calculate and print, for the threshold 0.85 (index 9 in the list):

• Confusion Matrix (TP, FN, FP, TN)
• Accuracy, Precision, Recall, and F1-Score

Then, for all thresholds, print:

• Raw Precisions, monotonic Precisions, and Recalls, separated by ,
• Final mAP

1.18.6.4 📌 Important

• Fixed threshold for the individual metrics: 0.85
• Safe division: if the denominator is zero, use 0
• Formatting: two decimal places
• Monotonize from back to front
• Figure 1.13 presents a simulation of this problem

1.18.6.5 📌 Example Run

1.18.6.6 🐍 Tip for calculating AP (with trapezoid rule)

def calcular_AP(verdades, confiancas, limiares):
    m = len(limiares)
    precisoes = [0.0] * m
    sensibilidades = [0.0] * m
    for i in range(m):
        p, s = calcular_metricas(verdades, confiancas, limiares[i])
        precisoes[m-1-i] = p
        sensibilidades[m-1-i] = s
    prec_mono = precisoes.copy()
    for i in range(m-2, -1, -1):
        if prec_mono[i] < prec_mono[i+1]:
            prec_mono[i] = prec_mono[i+1]
    AP = 0.0
    for i in range(1, m):
        # Trapezoid rule: average of heights times the base
        area_trapezio = (prec_mono[i-1] + prec_mono[i]) / 2.0
        AP += area_trapezio * (sensibilidades[i] - sensibilidades[i-1])
    return precisoes, prec_mono, sensibilidades, AP

# your solution

TestSuite("EP01_03.py").run()

✔️ EP01_03.cases already exists in casos/
📋 5 case(s) loaded from casos/EP01_03.cases
💥 File EP01_03.py not found.