#engineering

Attention from scratch: a step-by-step tutorial in NumPy

How a transformer works out that 'it' means the cat: dot products, softmax and a weighted sum, built in about a dozen lines of NumPy and worked by hand, number by number.