⚡ Simulator: Optimization with Contour Lines (SGD vs. Adam)
Non-Convex Terrain: Local vs. Global Minimum
💡 How to read this map: a yellow arrow points in the direction of descent (−∇L), which is the opposite of the gradient vector (∇L). The optimizer moves in this direction to reduce the loss L(w1, w2) until reaching the deepest regions (darker shades).
MAIN ALGORITHM (solid line)
LEARNING RATE (η)
Loss Heatmap L(w₁, w₂) — click to choose the start
Global Minimum
Local Minimum
Gradient (↓ descent)
Main trajectory
Ghost (other optimizer)
OPTIMIZATION STATE:
w₁ = 1.80, w₂ = 0.20
Loss L = 2.450
STEP
0
|∇L| (MAGNITUDE)
0.000
Status: Starting Point
Description: Figure 9.9: Interactive simulator of Optimization Algorithms: compare the trajectory of SGD and Adam over a non-convex loss surface with a heatmap and contour lines. The solid line shows the selected optimizer advancing step by step; the dashed line shows, for instant comparison, the full path that the other optimizer would take from the same starting point. Notice how SGD gets stuck in the Local Minimum on the right, while Adam may or may not cross the central barrier depending on accumulated momentum and learning rate. Click anywhere on the plot to reset the starting point of the weights.