Learning Machines

Find the bottom of a bowl you cannot see: measure the slope where you stand, step against it, repeat. The learning rate decides whether you crawl, arrive, bounce, or fly off.

One neuron learns a line: weigh the inputs, fire or not, and on every mistake turn the line toward the point it missed. It always works when a line exists, and never when one does not.

One training step, node by node: run the example forward, measure the error, send the blame back through the weights with the chain rule, and move every weight against it.

Turn words into arrows so that a dot product measures meaning: words seen together are pulled to point the same way, strangers are pushed apart, and similarity becomes an angle.

Every token looks at every other: a query scored against each key by a dot product, a softmax that turns the row into weights, and an output that blends the values. Six tokens, one six by six matrix.

One block, six stops: embed with a position code, attend with every head, add and normalize, feed forward, add and normalize again, predict a word at every position. Untrained weights, real flow.

Decoding one token at a time, every step needs every earlier token's key and value, and those never change. Keep them and a step projects one token, not the prefix: a ledger counts the saving.

Corrupt a shape into noise one small step at a time, train a network to guess the noise in any corrupted point, then walk the corruption backward from pure noise and watch the shape come out.