EP08_12 — 🔴 Instance Segmentation on Real Images: Geometric Objects

8.14.12 EP08_12 🔴 Instance Segmentation on Real Images: Geometric Objects

The classic segmentation example in this chapter separated coin “instances” by spatial disconnection in the binary mask resulting from Otsu’s thresholding. In this exercise, you will apply the same idea—but now on a real image with varied geometric objects—by chaining together preprocessing, binarization, contour extraction (cv2.findContours), and validation of the result against a ground truth of bounding boxes.

Unlike the previous exercise (labeling on an already-prepared mask), here you start from the original image: the quality of your segmentation depends directly on the preprocessing choices (filtering, thresholding, morphological operations) made before labeling the components.

8.14.12.1 📋 Implementation Guidelines

Input: use the image 00000.jpg.
Preprocessing and segmentation: apply the necessary steps (filtering, binarization, and morphological operations) to automatically separate the objects from the background, without manual cropping.
Labeling and measurement: for each segmented object, determine:
- area;
- center of mass (centroid);
- type, according to the obj2 set.
Visual annotation: write, inside each object, its area and the type abbreviation (obj2).
Validation (IoU): compute the Intersection over Union (IoU) between the detected bounding box (cv2.boundingRect) and the ground truth bounding box of the corresponding type. An object is considered correctly segmented only if there is exactly one bounding box of the correct type with IoU ≥ 0.5.
Output: print, for each detected object, its identifier, type, and whether it was successfully validated (acertou=1) or not. The output must follow the order of the obj2 classes (0=Tria … 8=Cruz); within the same class, sort the objects by the vertical coordinate of the centroid (cy) in increasing order. Finally, print the overall accuracy.

8.14.12.2 📌 Computational Constraints

No manual cropping: all segmentation must be performed on the full image.

Fixed set of classes:

obj  = ['Triangulo','Quadrado','Pendagono','Hexagono','Heptagono','Circulo',
        'Elipse','Estrela','Cruz']
obj2 = ['Tria','Quad','Pent','Hexa','Hept','Circ','Elip','Estr','Cruz']

Image dimensions: 608 × 608 pixels—used to denormalize the coordinates from the TXT file.
Center of mass validation: an object is only considered correctly segmented if its centroid lies strictly inside the ground truth bounding box corresponding to the same object type.

8.14.12.3 🧠 Theoretical Background

Element	Role in instance segmentation
Preprocessing (filtering, thresholding)	Stage that produces the binary mask from the original intensity image
`cv2.findContours`	Extracts the contours of connected components in the binary mask
Geometric moments (`cv2.moments`)	Allow computing the center of mass (centroid) of each contour
`approxPolyDP` / vertices	Aids in classifying the object type (approximate number of sides)
Bounding box validation	Confirms whether the segmented instance spatially corresponds to a ground truth object, measuring the method’s accuracy

8.14.12.4 📌 Example of Expected Output

Objeto 1: tipo=Tria, validado=True
...
Acurácia: 88.89%

Fixed parameters for reproducibility: so that the output matches the automatic grading rubric, use exactly: minimum area filter of 300 pixels; cv2.approxPolyDP with epsilon = 0.02 * perimeter; solidity threshold of 0.92 and vertex count ≥ 9 (with ≥ 11 to distinguish Cruz from Estrela) for concave shapes; aspect ratio of 1.15 to distinguish Círculo from Elipse; IoU threshold of 0.5 for validation.

8.14.12.5 📌 Reference Files (`.jpg` and `.txt`)

For local debugging, two reference files are provided (included in this submission; when integrating them into the chapter repository, save them in all/cap08/dados/EP08/):

📥 Image (00000.jpg): image of geometric objects used as the exercise input. The goal is to automatically segment each object, determine its type, and compute its measurements.
📥 Ground truth (00000.txt): file containing the normalized bounding boxes of the objects in the image. Each line contains the class identifier and the normalized coordinates of the upper-left and lower-right corners, used to automatically validate the segmentation.

Figure 8.26 shows the input image and the same image with the bounding boxes drawn from the ground truth file.

import os
import urllib.request
from morph import mm

def garantir_e_baixar(nome):
    pasta = "dados/EP12"
    caminho = os.path.join(pasta, nome)

    os.makedirs(pasta, exist_ok=True)

    if not os.path.exists(caminho):
        url = (
            "https://raw.githubusercontent.com/"
            "fzampirolli/pdi-vc/master/all/cap08/dados/EP12/"
            + nome
        )
        print(f"Downloading {nome}...")
        urllib.request.urlretrieve(url, caminho)

    return caminho

img_arq = garantir_e_baixar("00000.jpg")
txt_arq = garantir_e_baixar("00000.txt")

img = mm.read(img_arq)
img_bb = mm.showBoundBox(img, txt_arq, fmt="yolo", show=False)

mm.show(
    [img, img_bb],
    titles=[
        "Original image",
        "Answer key bounding boxes"
    ],
    cols=2,
    figsize=(10,5)
)

Figure 8.26: Simulator EP08_12: Image used in EP08_12. On the left, the original image. On the right, the image with the *bounding boxes* from the answer key file.

🧮 Simulator EP08_12: Segmentation Accuracy on Multiple Objects 🟢 hit if IoU ≥ threshold AND correct type

Each shape has a bounding box ground truth (dashed rectangle, tight around the shape) and a bounding box detected (solid rectangle, offset/noisy). Adjust the noise, bias, and IoU threshold to see the validation change.

segmentation noise (px, max jitter per side)0

systematic bias in x (px)0

IoU threshold0.50

simulate classification error (2 objects with swapped types)

id	true type	detected type	IoU	≥ threshold	hit

Figure 8.27: EP08_12 Simulator: Segmentation Accuracy in Multiple Objects (IoU)

%%writefile EP08_12.py
# Python code

Overwriting EP08_12.py

TestSuite("EP08_12.py").run()

✔️ EP08_12.cases already exists in casos/
📋 1 case(s) loaded from casos/EP08_12.cases

🔍 Testing Python: EP08_12.py
⚠️ EP08_12.py: Empty file (fewer than 3 lines). Tests skipped.