The Stack
A token is the chunk of text an AI model actually reads and writes — not a word, not a character, but a piece of a word: "un", "believ", "able". Every language model, from the smallest phone-side model to the biggest frontier system, does
The Frontier
A mixture of experts is a way of building an AI model where, instead of one giant network doing all the work, you split it into many smaller "expert" sub-networks — and a tiny router decides, for each piece of input, which two or three of those experts
The Stack
Reinforcement learning from human feedback (RLHF) is the training technique that teaches an AI model what people actually prefer — by using human ratings, not just text, as the signal for what counts as a good answer. It is the reason a chatbot says "I'm not sure, let
The Frontier
Continual learning — also called lifelong learning — is the ability of an AI model to keep learning new information over time without losing the skills it already has. Instead of being frozen after training, or wiped and rebuilt whenever the world changes, a continually learning model gets smarter as it goes,
The Stack
A vector database is a database purpose-built to store and search "embeddings" — long lists of numbers that capture the meaning of a piece of text, an image, or audio — so it can find things that are semantically similar rather than things that merely match your keywords. If
The Stack
Knowledge distillation is a way of training a small "student" AI model to imitate a big "teacher" model — so the student picks up much of the teacher's skill while being far smaller, faster, and cheaper to run. If you have read tech news lately,
The Guardrails
A sandbox escape is when an AI program that was supposed to run only inside a sealed, controlled environment breaks out and gets access to systems or networks it was never meant to touch. A "sandbox" in computing is exactly what it sounds like: a fenced-in area
The Frontier
A learned kernel is the low-level math software that runs on a GPU and was designed — or tuned by trial and error — by an AI program, rather than hand-written by a human engineer. When a large language model does its math, it doesn't just "think&
The Frontier
An AI hallucination is a statement a chatbot or model produces with total confidence that simply isn't true. The model isn't lying, glitching, or behaving maliciously — it is doing exactly what it was trained to do, which is to generate the next most plausible string of
The Stack
A large language model is a giant bag of numbers. Quantization is the trick of shrinking those numbers so the model takes up less memory and runs faster — usually with only a small, barely-noticeable drop in quality. If you have ever tried to run a powerful open-weight model
The Stack
Reasoning tokens are the hidden text a model generates to think through a problem before giving you its answer — invisible, billed to your account, and the reason modern AI can suddenly solve hard questions. When you ask a regular chatbot a question, it reads your words and starts writing a
The Stack
Synthetic data is information generated by a computer — not collected from the real world — that mimics the patterns, structure, and statistical properties of real data well enough to train an AI model. It is one of the most important tools in modern AI development, and one of the least understood.