AI 101 — What is a TPU?

A TPU — Tensor Processing Unit — is a computer chip Google built to do one thing: the enormous matrix multiplications that neural networks run on, done faster and with less electricity than a general-purpose chip can manage. It is the silicon underneath Search, Translate and Gemini, it is what Google rents to other AI companies through its cloud, and in 2026 it became collateral in some of the largest loans the AI buildout has seen. If you have read about a "$22 billion loan" or "a million TPUs" this month, this is the chip in those sentences.

Why it matters right now

For most of its life the TPU was an inside story — Google's own hardware, serving Google's own products, with occasional cloud customers. Three things changed this year. Anthropic announced in October 2025 that it would expand to up to one million TPUs, a deal worth tens of billions of dollars and well over a gigawatt of capacity coming online in 2026. In April 2026, Google split its eighth generation into two purpose-built chips for the first time. And in September, a consortium of ten banks lent $22 billion to Crux AI — the compute venture Blackstone and Alphabet formed in May — to buy TPUs, with the loan secured by the chips themselves, as we covered in Ten banks lend $22B to fund Alphabet and Blackstone's TPU cloud. When lenders are writing nine-figure checks against a specific chip, the chip is worth ten minutes of anyone's time.

The mental model

Almost everything a neural network does reduces to one operation: multiplying huge grids of numbers together and adding up the results, millions of times over. A TPU is shaped around that operation. Its heart is a systolic array — a grid of small multipliers that hand numbers to their neighbors the way a bucket brigade passes water, so data gets worked on as it flows rather than being fetched and returned over and over. The chip deliberately uses low-precision numbers (fewer bits per calculation) because AI tolerates the rounding, and it sits next to its own high-bandwidth memory — HBM, the stacked memory the whole industry is short of — so the data pipeline stays fed. Individual chips are wired together into pods of thousands, and a compiler maps your model onto the array.

The lineage in one breath: announced publicly in May 2016 after running inside Google since 2015; the first version served neural-network inference and, by Google's account, helped power AlphaGo's matches against Lee Sedol. The 2017 engineering paper reported 15–30 times the performance and 30–80 times the performance-per-watt of contemporary CPUs and GPUs on those workloads. Version 2 added training; version 4 ran in pods of 4,096 chips; Ironwood, the seventh generation, reached customers in late 2025 with a 9,216-chip configuration Google put at 42.5 exaflops — by its own reckoning more than 20 times the throughput of the largest supercomputer. The eighth generation, announced in April 2026, split in two for the first time: TPU 8t for training, scaling to 9,600 chips in a superpod with two petabytes of shared memory, and TPU 8i for serving, built to keep reasoning models and agents responsive.

The kitchen analogy

Think of a GPU as a restaurant kitchen staffed by versatile chefs: they can cook anything — video game frames, video editing, physics simulations, neural networks — and that flexibility is why they dominated AI's early years. A TPU is a machine built to make one dish: a stir-fry that is nothing but repeated multiplying and adding, served all day. With no other orders to handle, it can be laid out so every ingredient arrives exactly when needed, and it produces far more servings per unit of electricity. The trade is the definition of a specialty chip: unbeatable on the one dish, useless if the menu changes. That is why the design philosophy is called an ASIC — application-specific integrated circuit.

Common misconceptions

"A TPU is just Google's GPU." They come from different lineages. A GPU descends from graphics hardware and kept its generality; a TPU has no graphics functions at all and was designed around low-precision matrix math. Google's own guidance says TPUs excel when a model is dominated by matrix computations and trains for weeks without custom operations — exactly the profile of large language models, and exactly where a generalist wastes effort.

"You can just buy one." Mostly, no. Unlike Nvidia's cards, TPUs are generally not sold as standalone parts — you rent capacity through Google Cloud (Google did sell a small edge version for devices). That is precisely what makes the Crux AI loan interesting: the collateral is silicon whose resale market outside Google's own software stack, as we noted at the time, is thin to nonexistent. The chip's value and Google's ecosystem are the same bet.

"TPUs are about to replace GPUs." Even Google's biggest TPU customer does not pick one horse. Anthropic describes its compute strategy as deliberately diversified across Google's TPUs, Amazon's Trainium and Nvidia's GPUs. Custom silicon wins on specific workloads; the GPU remains the flexible default. The realistic picture is a split market, not a changing of the guard.

Where to learn more

The 2017 paper by Norman Jouppi and the Google team is the founding document and still the clearest explanation of the systolic-array idea. Google's Ironwood and eighth-generation blog posts give the current-generation specifics, and the Cloud TPU documentation is the practical starting point if you want to use one. Two sibling explainers round out the hardware picture: What is an NPU (neural processing unit)? covers the same idea at the scale of your phone, and What is a learned kernel? covers the software that decides how efficiently any of these chips actually runs. And since the newest TPUs are built for serving mixture-of-experts models, What is a mixture of experts? explains why that architecture keeps the per-token cost low.

Related reading: What is an NPU (neural processing unit)? · What is a learned kernel? · What is a mixture of experts?

Ten banks just bet $22 billion on a chip that only runs one company's software. Smart engineering conviction, or the next thing to worry about? Tell us in the comments.

Sources: Google Cloud — Introduction to Cloud TPU · Jouppi et al. — In-Datacenter Performance Analysis of a Tensor Processing Unit · Google — Ironwood: the first TPU for the age of inference · Google Cloud — Our eighth generation TPUs: two chips for the agentic era · Anthropic — Expanding our use of Google Cloud TPUs and Services · AI Midday — Ten banks lend $22B to fund Alphabet and Blackstone's TPU cloud