DIP+CV · Programming Exercise

EP09_05 — 🟠 Segmentation Evaluation: Pixel-wise IoU and Dice

9.10.5 EP09_05 🟠 Segmentation Evaluation: Pixel-wise IoU and Dice

Block 2 of the “Semantic Segmentation with U-Net Architecture” section defines, in a few lines, the iou_mascaras function, used to measure the quality of the classic morphological baseline (smoothing + Otsu + opening) and, later, of the trained U-Net itself. Unlike the IoU from EP09_04 — computed over bounding boxes (rectangular regions described by four numbers) —, the segmentation IoU is computed pixel by pixel: each image position is compared individually between the predicted mask and the reference mask.

You have been tasked with generalizing this evaluation, implementing not only the pixel-wise IoU but also the Dice coefficient, another overlap metric widely used in medical segmentation (including in the perda_dice function, mentioned in the same block of the chapter as the basis for the loss function used to train the U-Net).

9.10.5.1 📋 Implementation Guidelines

  1. Input: Read the dimensions \(H \times W\) of the masks.

  2. Predicted mask: Read \(H\) lines with \(W\) integer values (0 or 1) each — for example, the output of a U-Net after thresholding at \(0.5\) over the sigmoid, as in Block 4 of the chapter.

  3. Reference mask: Read another \(H\) lines with \(W\) integer values (0 or 1) each — the ground truth.

  4. Intersection and union: Considering each pixel as belonging to the object when its value is different from zero, \[ \text{intersection} = \sum_{i,j} \mathbb{1}[P_{ij}=1 \wedge R_{ij}=1], \qquad \text{union} = \sum_{i,j} \mathbb{1}[P_{ij}=1 \vee R_{ij}=1]. \]

  5. Pixel-wise IoU: \[ \text{IoU} = \frac{\text{intersection}}{\text{union}}. \]

  6. Dice coefficient: \[ \text{Dice} = \frac{2 \cdot \text{intersection}}{|P| + |R|}, \] where \(|P|\) and \(|R|\) are the total number of object pixels in each mask.

  7. Convention for empty masks: if both masks have no object pixels (union \(= 0\) and \(|P|+|R|=0\)), consider the correspondence trivially perfect: \(\text{IoU} = \text{Dice} = 1.0\).

  8. Output: Two lines, IoU: X.XXXX and Dice: X.XXXX, each value with 4 decimal places.

9.10.5.2 📌 Computational Constraints

  • Any non-zero value counts as an object: treat values different from \(0\) (not just \(1\)) as belonging to the mask, replicating the predita > 0 check used in iou_mascaras in the chapter.
  • Same dimensions: the two masks always have exactly \(H \times W\) elements.
  • Empty convention: apply the rule from item 7 only when both masks are completely empty; if only one is empty, the intersection is \(0\) and the resulting IoU/Dice will also be \(0\).

9.10.5.3 🧠 Theoretical Foundation

Element Role in segmentation evaluation
Pixel-wise IoU Generalizes the metric from EP09_04 to arbitrarily shaped regions — not just rectangles — by comparing predicted and reference masks position by position
Dice coefficient Metric related to IoU (always \(\text{Dice} \ge \text{IoU}\)), more sensitive to small intersections and widely used as a loss function in segmentation (perda_dice function from the chapter)
Empty mask convention Avoids division by zero and recognizes that “no predicted object, no real object” is, by definition, a correct match
Classic vs. U-Net comparison The chapter uses exactly this type of metric to numerically justify why the U-Net surpasses the morphological baseline in low-contrast scenarios

9.10.5.4 🧩 Methods from morph.py that may help

  • mm.readImg(h, w, dtype='uint8') — directly reads each binary mask \(h \times w\) from standard input (the values \(0/1\) fit perfectly in the standard integer type).
  • The iou_mascaras function itself, defined in Block 2 of the U-Net section of the chapter (not part of morph.py, but of the chapter’s code), is the direct inspiration for this exercise — it is worth re-reading those few lines before coding.
  • For an optional extension (not required by this EP), mm.connectedComponents or mm.label0 (seen in the context of connected component analysis) would allow labeling each nodule individually and computing the IoU per component, instead of over the entire mask.

9.10.5.5 📦 Input and Output Specification (VPL)

Input:

  • Line 1: Integers \(H\) and \(W\).
  • Next \(H\) lines: \(W\) integer values (0 or 1) — predicted mask.
  • Next \(H\) lines: \(W\) integer values (0 or 1) — reference mask.

Output:

  • Line 1: IoU: X.XXXX.
  • Line 2: Dice: X.XXXX.
Tip💡 Illustrative Example

Consider a predicted mask with a \(2\times2\) square of active pixels and a reference shifted by one column, overlapping on only half of the area:

Predicted       Reference
0 0 0 0         0 0 0 0
0 1 1 0         0 0 1 1
0 1 1 0         0 0 1 1
0 0 0 0         0 0 0 0

Intersection \(=2\) pixels, union \(=6\) pixels (\(4+4-2\)), therefore \(\text{IoU}=2/6\approx0.3333\) and \(\text{Dice}=2\cdot2/(4+4)=0.5000\) — note that Dice is always equal to or greater than IoU for the same overlap.

9.10.5.6 📌 Examples

Input Output Observation
4 4
0 0 0 0
0 1 1 0
0 1 1 0
0 0 0 0
0 0 0 0
0 0 1 1
0 0 1 1
0 0 0 0
IoU: 0.3333
Dice: 0.5000
\(4\times4\) masks with partial overlap of 2 pixels.
🎮 Simulator: IoU and Dice Pixel by Pixel 🟠 Segmentation
Intersection (TP) Only predicted (FP) Only reference (FN) Background (TN)
5×5
Square
🔵 Predicted Mask
🟡 Reference Mask
🎯 Visual Comparison
📊 Calculations and Formulas
Figure 9.47: Simulator EP09_05: Segmentation Evaluation — IoU and Dice Pixel by Pixel
%%writefile EP09_05.py
# Python code
Overwriting EP09_05.py
TestSuite("EP09_05.py").run()
✔️ EP09_05.cases already exists in casos/
📋 4 case(s) loaded from casos/EP09_05.cases

🔍 Testing Python: EP09_05.py
⚠️ EP09_05.py: Empty file (fewer than 3 lines). Tests skipped.