Deep Learning: Stacking Simple Things Until They Get Smart
A neural network is layers of "multiply, add, squash." Backpropagation figures out which numbers to change. Transformers add one trick: attention.
14 published · newest first
A neural network is layers of "multiply, add, squash." Backpropagation figures out which numbers to change. Transformers add one trick: attention.
The whole field compresses to a loop: guess, measure error, adjust, repeat. Everything else is engineering around that loop.