EP02_11 — 🏆 Perspective Correction (Homography) in a Real Image

2.17.11 EP02_11 🏆 Perspective Correction (Homography) in a Real Image

In this activity, the goal is to apply perspective transformation (homography) to “rectify” a tilted object in a real photograph. You will work with an image of a newspaper, where the grid of a Sudoku puzzle is distorted due to the angle at which the photo was taken.

Your program must read input parameters from the terminal, load the image, compute the \(3 \times 3\) homography matrix, apply the geometric transformation, and display a global validation indicator.

Read two integers L and C, representing the row and column dimensions (height and width) that the rectified output image should have.
Read four coordinate pairs \((x, y)\) from the terminal, representing the four corners of the source quadrilateral (the distorted Sudoku in the original image).
Automatically compute the four destination coordinate pairs using the provided dimensions \(L\) and \(C\), mapping the corners to the edges of the new image: \((0,0)\), \((C-1, 0)\), \((0, L-1)\), and \((C-1, L-1)\).
Load the local image sudoku.png and convert it to grayscale.
Compute the homography matrix and apply the spatial transformation to the image.
Output: Calculate and print the sum of all pixels of the resulting image.

📌 Important:

Input file: The image sudoku.png must be in the same folder as the script. The program must read it directly from disk (e.g., using mm::read("sudoku.png") or cv2.imread).
Point Order: Ensure that the reading of the 4 source points and the generation of the 4 destination points strictly follow the same corner order: Top-Left (TL), Top-Right (TR), Bottom-Left (BL), and Bottom-Right (BR).
Dimensions in OpenCV: Remember that functions such as cv2.warpPerspective expect the output image size in the format (width, height), which is equivalent to (C, L).
Interpolation: To ensure mathematical consistency of the pixel sum with the automatic grader, use the default bilinear interpolation (flags=cv2.INTER_LINEAR).
Credits: The image used is “Sudoku en periódico” by Héctor Rodríguez, licensed under CC BY 2.0.

2.17.11.1 🧠 Problem Context

The homography has 8 degrees of freedom, requiring at least 4 point correspondences to be computed. Unlike affine transformations, it maps any quadrilateral to another quadrilateral, allowing lines that converge to vanishing points to become parallel again:

Operation	Characteristic	Typical Application
Homography	Projection between planes	Document rectification, scanning of plates and QR Codes.
Inverse Mapping	Scanning from destination to source	Avoids “holes” or empty pixels in the final rectified image.
Warping	Spatial resampling	Correction of lens distortion and panorama stitching.

2.17.11.2 📌 Examples

Input	Output	Observation
500 500 100 120 420 95 80 440 450 460	32982820	The first two inputs are the output dimensions (\(L\) and \(C\)). The following 4 lines are the \((x, y)\) coordinates of the Sudoku corners in the original image + PAD. The output is the total sum of pixels of the rectified image.
200 200 100 120 420 95 80 440 450 460	5277150	Same source points as the previous example, but generating a smaller output image (\(200 \times 200\)). The pixel sum decreases proportionally due to the scale.

2.17.11.3 Sudoku image acquisition and conversion to grayscale

Figure 2.22 shows the reading of the original image followed by its conversion to grayscale and resizing to a \(500 \times 500\) pixel matrix, preparing the data for the next step.

The perspective correction, applied at Figure 2.23 via the homography matrix, eliminates distortions caused by the camera angle and produces a frontal, regular view of the Sudoku grid.

%%writefile tmp/fig_02_sudoku_original.cpp
#define MM_OUT "tmp/fig_02_sudoku_original.png"
//| label: fig-02-sudoku-original
//| fig-cap: "Aquisição da imagem de um Sudoku à esquerda. À direita, conversão para tons de cinza e redimensionamento. Crédito: Héctor Rodríguez de Guardamar, Espanha (CC BY 2.0)."
//| echo: false
//| output: true

#include "morph.hpp"
#include <string>
#include <vector>
#include <filesystem>

int main() {
    std::string url = "https://upload.wikimedia.org/wikipedia/commons/e/e7/Sudoku_en_peri%C3%B3dico.jpg";
    mm::Image sudoku_rgb = mm::read(url);
    mm::Image sudoku_gray = mm::gray(sudoku_rgb);
    mm::Image sudoku_small = mm::resize(sudoku_gray, 500, 500);
    mm::write(sudoku_small, "sudoku.png");   // consumida pela célula fig-02-sudoku2

    mm::show(
        std::vector<mm::Image>{sudoku_rgb, sudoku_small},
        MM_OUT,
        std::vector<std::string>{"Original RGB", "Cinza 500x500"},
        2
    );
    
// [pdi:panel-io] auto-generated — do not edit by hand
std::filesystem::create_directories("tmp");
mm::write(sudoku_rgb, "tmp/fig_02_sudoku_original_0.png");
mm::write(sudoku_small, "tmp/fig_02_sudoku_original_1.png");
// [pdi:panel-io:end]
return 0;
}

Overwriting tmp/fig_02_sudoku_original.cpp

!g++ -I. -std=c++17 tmp/fig_02_sudoku_original.cpp -o tmp/fig_02_sudoku_original \
  && ./tmp/fig_02_sudoku_original \
  && test -f "tmp/fig_02_sudoku_original.png" \
  || echo "⚠ mm::show não gravou tmp/fig_02_sudoku_original.png"

[1] Original RGB
[2] Cinza 500x500

try:
    mm.show(
        [
            mm.read("tmp/fig_02_sudoku_original_0.png"),
            mm.read("tmp/fig_02_sudoku_original_1.png"),
        ],
        titles=[
            'Original RGB',
            'Cinza 500x500',
        ],
        cols=2,
    )
except Exception as _e:
    print("figura indisponivel nesta trilha (C++): " + repr(_e) + " tmp/fig_02_sudoku_original_0.png (ver a versao Python)")

Figure 2.22: Aquisição da imagem de um Sudoku à esquerda. À direita, conversão para tons de cinza e redimensionamento. Crédito: Héctor Rodríguez de Guardamar, Espanha (CC BY 2.0).

%%writefile tmp/fig_02_sudoku2.cpp
#define MM_OUT "tmp/fig_02_sudoku2.png"
#include "morph.hpp"
#include <iostream>
#include <vector>
#include <array>
#include <string>
#include <filesystem>

int main() {
    // 1. Image saved by the previous cell (sudoku.png, 500×500, gray)
    mm::Image img = mm::read("sudoku.png");

    // 2. Padding so the grid corners aren't cut off (mm::pad fills with 0)
    const int PAD = 60;
    mm::Image img_pad = mm::pad(img, PAD);

    // 3. Grid corners in the expanded image — TL, TR, BL, BR
    std::vector<std::array<double, 2>> pts1 = {{100, 160}, {390, 45}, {200, 580}, {570, 420}};

    // 4. Destination corners: front view SIZE×SIZE
    const int SIZE = 500;
    std::vector<std::array<double, 2>> pts2 = {{0, 0}, {SIZE, 0}, {0, SIZE}, {SIZE, SIZE}};

    // 5. Homography pts1 -> pts2 and rectification
    mm::Image img_rect = mm::perspective_transform(img_pad, pts1, pts2, SIZE, SIZE);

    // 6. Display
    mm::show(
        std::vector<mm::Image>{img_pad, img_rect},
        MM_OUT,
        std::vector<std::string>{"Original (with padding)", "Rectified front view"},
        2
    );

    
// [pdi:panel-io] auto-generated — do not edit by hand
std::filesystem::create_directories("tmp");
mm::write(img_pad, "tmp/fig_02_sudoku2_0.png");
mm::write(img_rect, "tmp/fig_02_sudoku2_1.png");
// [pdi:panel-io:end]
return 0;
}

Overwriting tmp/fig_02_sudoku2.cpp

!g++ -I. -std=c++17 tmp/fig_02_sudoku2.cpp -o tmp/fig_02_sudoku2 \
  && ./tmp/fig_02_sudoku2 \
  && test -f "tmp/fig_02_sudoku2.png" \
  || echo "⚠ mm::show não gravou tmp/fig_02_sudoku2.png"

[1] Original (with padding)
[2] Rectified front view

try:
    mm.show(
        [
            mm.read("tmp/fig_02_sudoku2_0.png"),
            mm.read("tmp/fig_02_sudoku2_1.png"),
        ],
        titles=[
            'Original (com padding)',
            'Vista frontal retificada',
        ],
        cols=2,
    )
except Exception as _e:
    print("figura indisponivel nesta trilha (C++): " + repr(_e) + " tmp/fig_02_sudoku2_0.png (ver a versao Python)")

Figure 2.23: Correção de perspectiva por homografia 3×3: imagem com *padding* e a vista frontal retificada, via *mm::perspective_transform* (solve 8×8 dos 4 pontos + mapeamento inverso projetivo).

🎮 Simulator EP02_11: Sudoku Perspective 3×3 Homography · CC BY 2.0

📷 Original (Expanded) — Drag the Corners

Photo: Héctor Rodríguez · CC BY 2.0

✅ Corrected (400×400) — Frontal View

      Loading Sudoku image...
    

Figure 2.24: EP02_11 Simulator: Sudoku Perspective Correction (3×3 Homography with Bilinear Resampling)

%%writefile EP02_11.cpp
// your solution

Overwriting EP02_11.cpp

TestSuite("EP02_11.cpp").run()

✔️ EP02_11.cases already exists in casos/
📋 4 case(s) loaded from casos/EP02_11.cases

🔍 Testing C++: EP02_11.cpp
⚠️ EP02_11.cpp: Empty file (fewer than 3 lines). Tests skipped.