🎮 Simulator EP07_01: k-NN Classifier Step by Step
Majority Voting
Adjust k and see which training examples (ordered by distance) participate in the voting for the fixed query (★ at x = 3, y = 3).
–
The KNeighborsClassifier from scikit-learn, used throughout the chapter, hides behind a single call (.fit / .predict) a quite simple decision rule: for each new observation, compute the distance to all training examples, select the \(k\) closest ones, and vote by the majority class among them.
Before relying on the library, you have been tasked with implementing this rule from scratch, for a two-dimensional feature space, exactly as the chapter’s interactive decision boundary simulator does internally with each user click.
1.| Element | Role in k-NN |
|---|---|
| Feature space | Set of all possible vectors \((x, y)\) |
| Euclidean distance | Measure of similarity between observations |
| Small \(k\) | Irregular boundary, high variance |
| Large \(k\) | Smooth boundary, high bias |
| Majority voting | Decision rule \(\hat y = \operatorname{mode}\{y_i : x_i \in N_k(x)\}\) |
Input:
Output:
0 or 1) for the respective query, in input order.Total class 1: X.| Input | Output | Observation |
|---|---|---|
| 4 3 0 0 0 1 0 0 5 5 1 6 5 1 1 1 1 |
0 Total class 1: 0 |
Query close to the class 0 cluster. |
| 4 1 0 0 0 1 0 0 5 5 1 6 5 1 2 0.9 0.1 5.5 5.1 |
0 1 Total class 1: 1 |
With \(k=1\), each query inherits the class of its closest neighbor. |
%%writefile EP07_01.py
# Python codeOverwriting EP07_01.py
TestSuite("EP07_01.py").run()✔️ EP07_01.cases already exists in casos/
📋 5 case(s) loaded from casos/EP07_01.cases
🔍 Testing Python: EP07_01.py
⚠️ EP07_01.py: Empty file (fewer than 3 lines). Tests skipped.