How Transformers Work

A step-by-step walkthrough of the encoder transformer architecture

Simulated modeSimulated8-dim embeddings · 2 heads · real math, tiny scale

➤ input_text:

Try:

Use ← → arrow keys to navigate

Step 0:Input

Everything starts with raw text. The transformer doesn't see words — it needs to convert text into numbers. Type a sentence to begin.

The cat sat on the mat