Deep Learning: Stacking Simple Things Until They Get Smart
A neural network is layers of "multiply, add, squash." Backpropagation figures out which numbers to change. Transformers add one trick: attention.
// foundations
Layers, backprop, transformers and attention: the machinery behind modern models.
A neural network is layers of "multiply, add, squash." Backpropagation figures out which numbers to change. Transformers add one trick: attention.