EP07_03 — 🟡 Evaluation via Confusion Matrix

7.18.3 EP07_03 🟡 Evaluation via Confusion Matrix

A binary weld quality classifier was trained and tested on a production line. For each inspected part, the system recorded the actual label (obtained by an expert) and the predicted label from the classifier, where 1 represents “defective” and 0 represents “conforming.”

Quality management wants to know not only the system’s accuracy but also its precision (when the system flags a defect, how often is it correct?) and its recall (of all truly defective parts, how many did the system manage to identify?)—the distinction discussed in the classifier evaluation section of the chapter.

7.18.3.1 📋 Implementation Guidelines

Quantity: Read the integer \(N\) (number of inspected parts).
Data for each part: For each of the \(N\) parts, read two integers—the actual label \(y\) and the predicted label \(\hat y\) (both \(\in \{0, 1\}\)).
Confusion matrix: Considering class 1 (defective) as positive, count:
- \(VP\) (True Positive): \(y=1\) and \(\hat y=1\);
- \(FP\) (False Positive): \(y=0\) and \(\hat y=1\);
- \(FN\) (False Negative): \(y=1\) and \(\hat y=0\);
- \(VN\) (True Negative): \(y=0\) and \(\hat y=0\).
Metrics: Calculate \[ \text{Accuracy} = \frac{VP+VN}{N}, \quad \text{Precision} = \frac{VP}{VP+FP}, \quad \text{Recall} = \frac{VP}{VP+FN}. \]
Degenerate cases: If \(VP+FP=0\) (no positive predictions), print Precisao: indefinida. If \(VP+FN=0\) (no actual positive cases), print Revocacao: indefinida.
Rounding: All numerical metrics must be rounded to 4 decimal places (round half away from zero) only for display.

7.18.3.2 📌 Computational Constraints

Fixed positive class convention: class 1 is always the positive class in this exercise, regardless of its relative frequency.
Division by zero protection: implement the degenerate cases from item 5 before performing the division.
Output order: follow exactly the order specified in the output section, even in degenerate cases.

7.18.3.3 🧠 Theoretical Foundation

Metric	Question It Answers	Sensitive to Imbalance?
Accuracy	What fraction of parts was classified correctly?	Yes—can mask errors in the minority class
Precision	Of the parts flagged as defective, how many truly are?	Penalizes false positives
Recall	Of the truly defective parts, how many were detected?	Penalizes false negatives

In an industrial context, low recall is often more severe than low precision: letting a defective part pass (false negative) tends to be costlier than manually inspecting a good part flagged by mistake (false positive).

7.18.3.4 📦 Input and Output Specification (VPL)

Input:

Line 1: Integer \(N\).
Next \(N\) lines: two integers per line—\(y\) and \(\hat y\), separated by spaces.

Output (in this exact order):

VP=<int> FP=<int> FN=<int> VN=<int>
Acuracia: <value or undefined metric>
Precisao: <value or undefined>
Revocacao: <value or undefined>

7.18.3.5 📌 Examples

Input	Output	Observation
4 1 1 0 1 1 0 0 0	VP=1 FP=1 FN=1 VN=1 Acuracia: 0.5000 Precisao: 0.5000 Revocacao: 0.5000	One error of each type.
3 0 0 0 0 0 0	VP=0 FP=0 FN=0 VN=3 Acuracia: 1.0000 Precisao: indefinida Revocacao: indefinida	No actual or predicted positive cases.

🎮 EP07_03 Simulator: Precision x Recall Production Line

Choose an inspection scenario and observe how Accuracy, Precision, and Recall react differently.

–

Figure 7.23: Simulator EP07_03: Precision x Recall

%%writefile EP07_03.py
# Python code

Overwriting EP07_03.py

TestSuite("EP07_03.py").run()

✔️ EP07_03.cases already exists in casos/
📋 5 case(s) loaded from casos/EP07_03.cases

🔍 Testing Python: EP07_03.py
⚠️ EP07_03.py: Empty file (fewer than 3 lines). Tests skipped.