Autograd

Published on
August 23, 2026
Micrograd, Line by Line — How Andrej Karpathy's Tiny Autograd Engine Explains Neural Nets
Deep-Learning Neural-Networks Autograd Python
Notes on Andrej Karpathy's micrograd — a ~150-line autograd engine that builds backpropagation, gradient descent, and a full neural net (Value, Neuron, Layer, MLP) from first principles. Still the best free, intuitive explanation of how neural nets actually learn.

Micrograd, Line by Line — How Andrej Karpathy's Tiny Autograd Engine Explains Neural Nets