AI 101 — What is a learned kernel?

A learned kernel is the low-level math software that runs on a GPU and was designed — or tuned by trial and error — by an AI program, rather than hand-written by a human engineer.

When a large language model does its math, it doesn't just "think" in a clean way: every layer boils down to enormous matrix multiplications that must be executed millions of times per second. On Nvidia GPUs those operations are performed by special routines called kernels. Today almost all of them come from hand-tuned libraries like Nvidia's cuBLAS, whose code is the product of decades of human expertise. A learned kernel is the same idea, written instead by the very kind of AI it helps to run — which is why the phrase keeps showing up in papers about making inference faster and cheaper.

Why it matters right now

For months, a steady stream of results has been making the same point: let an AI write its own math code, and it can beat the libraries humans spent years perfecting. The headline example is CUDA-L2, a system from the DeepReinforce team that combined a language model with reinforcement learning to automatically optimize the half-precision matrix-multiply kernels behind most inference. Published in December 2025, it claimed speedups between about 11% and 26% over Nvidia's own heavily optimized paths across 1,000 different problem sizes. Similar work — KernelFoundry, TritonRL and others — is attacking the same target.

Why care? Because on the frontier, model performance has become cheap to copy; every percentage point of inference speed compounds into dramatically lower operating costs and better latency. "Learned kernels" is the rare find that sits at the intersection of more speed, less electricity and the maturing ability of models to reason about raw hardware. It is also a sign of the field's mid-2020s mood: the bottleneck has shifted from "how do we think up ideas" to "how do we shave microseconds off the math," and the models themselves are increasingly the ones doing that engineering.

The mental model

Think of the whole stack as a kitchen. The model is the chef; a kernel is the recipe for one specific, endlessly repeated dish — say, the way the kitchen chops the vegetables. For years, the recipe was written by hand by a master cook (a human expert in GPU libraries), refined obsessively for every size of vegetable and every kitchen layout. A learned kernel is a recipe the chef — an AI trained to optimize — experiments with, trying thousands of variations and keeping only the ones that shave a second off the chopping. The chef doesn't fully understand why it works; it just measures which recipe finished faster and kept producing correct food, and keeps the winner.

The everyday analogy

Engine tuning offers a clearer everyday comparison. A factory setting for an engine maps each input to an output using carefully tuned curves. Hand-tuning is slow and imperfect; a racing team with a dynamometer makes tiny adjustments, measures the torque, repeats. A learned kernel is a dynamometer session run tens of thousands of times by a machine that can test configurations the human wouldn't have thought to try — and it only cares about one number: seconds around that section of track for that exact car. It doesn't require the tuner to know engine theory; it just needs the reward signal and lots of trials.

Common misconceptions

"Learned kernels are buggy, so they're unusable." Raw AI-generated GPU code is famously error-prone — one audit found nearly four in ten LLM-written kernels were broken. That is why the systems that work pair the generation with a verifier and use feedback from real runs, discarding any kernel that gives a wrong answer. The reliable learned kernels are the survivors of filtering, not the raw output.

"It's just compiler auto-tuning." Humans have auto-tuned this kind of code for twenty years. What's new is using a language model and reinforcement learning to search a far larger space of strategies, and letting the model generalize across shapes instead of hand-designing heuristics.

"This replaces cuBLAS tomorrow." Not yet. Learned kernels win on specific operations, shapes and hardware. Nvidia's libraries remain the safe default; learned kernels are winning specific battles, not the whole war.

Where to learn more

The CUDA-L2 paper is the clearest entry point — it explains both the search method and the speedup numbers. Nvidia's cuBLAS documentation describes the hand-built libraries that are the current baseline. For the broader wave, browse recent work on AI-driven CUDA optimization and the many benchmarks chasing it.

Related reading: Learned kernels get their intelligence from a training signal — the same idea at the heart of What is fine-tuning? — and reinforcement learning is a cousin of What is test-time training?. If you're wondering why small tuned models on your own hardware are so attractive, What are open-weight models? fills in the picture.

Will learned kernels put hand-tuned GPU libraries out of business — or will they stay a niche tool for the biggest labs? Tell us in the comments.

Sources: CUDA-L2 paper (arXiv) · Nvidia cuBLAS · KernelFoundry (arXiv) · RL-found kernels analysis (Groundy)