🧮 Simulator: Kernel Weight Gradient
∂Loss / ∂K[kr][kc] = Σ dZ · X
INPUT PATTERN (IMAGE 6×6)?
Choose which image 6×6 feeds the convolution.
LEARNING RATE (η)?
Learning rate. Adjust to see the difference between smooth convergence (0.002) and collapse from overshooting (0.02).
Example reduced: image 6×6 and filter 3×3 generating maps 4×4. Click the tabs "🔍 How is it calculated?" below each matrix to understand the calculations step by step. Hover over any cell in X, Z, A, dZ, or K to see the exact calculation for that value, with the elements used in the related layers highlighted with a dashed/blue outline.
1. KERNEL WEIGHT?
Select which kernel weight you want to analyze individually.
CURRENT KERNEL (K)?
Filter 3×3 values. The selected weight is highlighted in blue. Hover over a weight to see where it is used.
🔍 How is it updated?
Gradient Rule:
• ∇K = sum of the 16 dZ votes × X.
K ← K − η · ∇K
• η = learning rate.• ∇K = sum of the 16 dZ votes × X.
INPUT X (6×6)?
Image 6×6. Blue pixel = overlap with the selected kernel weight K in the current window. Hover over a pixel to see which Z positions it is used in.
🔍 How does X work?
Input matrix. At position (r,c), weight K multiplies the pixel:
X[r + kr][c + kc]
PRE-ACTIVATION Z (4×4)?
Convolution result before ReLU: Z = Σ K · X. Hover over a cell to see the 9 terms of the sum, highlighting the window in X and the entire kernel K.
🔍 How is Z calculated?
Cross-correlation:
Pointwise multiplication of filter 3×3 over X:
Pointwise multiplication of filter 3×3 over X:
Z[r][c] = Σ K · X
ACTIVATION A (4×4)?
Post-ReLU result: A = max(0, Z). If Z ≤ 0, the activation is zeroed. Hover over a cell to highlight the corresponding Z.
🔍 How is A calculated?
ReLU function:
A[r][c] = max(0, Z[r][c])
Global Sum (S):S = Σ A[r][c]
ERROR dZ (4×4)?
Propagated error: dZ = (S - target) · I(Z > 0). Where A=0, the error dZ is also 0. Hover over a cell to see the full calculation, highlighting the corresponding Z and all 16 cells of A that form S.
🔍 How are dZ, S, and Loss calculated?
1. Loss (Loss L):
L = ½ (S − target)²
2. Propagated error dZ:dZ = (S − target) · deriv_ReLU(Z)
2. CALCULATION AND SUM OF EACH POSITION'S "VOTES"?
Each position (r,c) generates a vote = dZ[r][c] × X[r+kr][c+kc]. The sum of all 16 votes forms the weight gradient.
CALCULATION OF THIS POSITION?
Displays the local error (dZ) and input pixel (X) multiplied at the current sliding window position.
ACCUMULATED SUM (GRADIENT)?
The accumulated value of the products dZ × X from all positions already visited. When it reaches 16/16, this is the final weight gradient.
HISTORY OF THE 16 POSITIONS (COLUMNS c=0, c=1, c=2, c=3)?
Follow the list of all 16 positions organized in 4 columns to match the window movement over the output image.
3. USE THE GRADIENT TO UPDATE THE KERNEL?
Applies the Gradient Descent rule (K ← K − η · gradient) to all 9 weights.
LOSS OVER UPDATES?
Evolution of error L = ½(S − target)²:
• Objective: L → 0 (S → target).
• If stuck at L = 40.5: "Overshooting" occurred (exaggerated leap). The weights became very negative, generating Z ≤ 0 (ReLU death). With S = 0, the loss freezes at ½(0 − 9)² = 40.5.
• Objective: L → 0 (S → target).
• If stuck at L = 40.5: "Overshooting" occurred (exaggerated leap). The weights became very negative, generating Z ≤ 0 (ReLU death). With S = 0, the loss freezes at ½(0 − 9)² = 40.5.
Description: Figure 9.12: Interactive simulator for computing the gradient of a weight from the convolutional kernel .