Memory Is the Moat, Part 1: Teaching Machines to Reason About the Physical World

The first of three deep dives on physical AI. In a world where technology moats are rapidly shifting, we see one of the highest-value layers remaining in the embodied/application layers. 

Part 1: Why the next act of AI is physics, and what a machine needs to know before it can be trusted on a job site.

Tuesday on the Job Site

It's 6:40 AM on a construction site in Houston. A tower crane operator on a mixed-use development is running lifts while a concrete pour gets underway on level 14. The wind is gusting at 22 miles per hour. The crane's load moment indicator is green. Everything looks normal.

Three things are happening that no single system is tracking: The concrete pour schedule shifted 90 minutes earlier due to a supplier change, logged in the project management platform around 9 PM the night before. A safety inspector flagged incomplete rebar tying on the west elevation yesterday afternoon, buried in a Procore PDF nobody reads in real time. The 5 AM weather update shows sustained winds crossing the crane manufacturer's operational threshold within 45 minutes. The crane operator doesn't read inspection reports. The superintendent doesn't watch the weather minute by minute. The safety manager doesn't know the pour moved up. Each fact lives in a system. None of them lives together in a decision.

This hypothetical scenario plays out every day in almost every city in the modern world, and is emblematic of the gap physical AI is built to close: the unglamorous distance between data that exists and decisions that actually get made on a job site, a factory floor, or a grid control room.

For the past decade (and especially since we started investing in machine learning startups in 2017), AI has gotten exceptionally good at language and pixels, almost all of it in the digital world. The next act is physics, and physics demands a different architecture: systems that reason about forces, materials, and time; coordinate across heterogeneous data and control layers in real time; and get smarter the longer they operate.

That architecture has three layers, and this deep dive series takes them one at a time. This exploration covers the first: world models, the layer that gives a machine the capacity to reason about physics. Part 2 covers orchestration, the layer that gives the reasoning a job site to operate on. Part 3 covers compounding memory, the layer we believe becomes the most defensible asset in the entire stack.

The physical AI stack, and the map for this series: world models reason (Part 1), orchestration coordinates (Part 2), memory compounds (Part 3).

World Models: Teaching Machines to Reason About the Physical World

The LLM revolution rested on a simple idea: train a neural network to predict the next token, give it enough text, and it learns the structure of language. World models start from an analogous premise, but the sequence isn't text. It's the physical world. Sensor readings over time. Video of a robotic arm reaching for a part. Forces on a structural member under load. A world model learns that certain physical states follow other physical states, and that certain actions produce certain consequences.

The most important debate in the field is what a world model should actually try to predict. The first wave, descended from video generation, trained models to predict the next frame pixel by pixel. Visually impressive, practically limited. Pixel-space prediction burns most of the model's capacity on details irrelevant to physical reasoning. None of that helps an agent decide whether a load is about to slip.

The more promising direction, championed by Yann LeCun and embodied in joint-embedding predictive architectures (JEPA), is to predict in a learned latent space. The model learns a compact representation of the relevant state of the world and predicts how it evolves. This is closer to how humans reason about physics: we don't mentally render high-resolution video of a falling object; we track its trajectory in an abstracted space of position, velocity, and force. For industrial environments, where the question is almost never "what does this look like?" and almost always "what is about to happen?", latent-space prediction is the architecture that fits.

The difference is easiest to see side by side. A language model can describe what a tower crane does. A world model can predict what happens to a suspended load at a specific wind speed, boom angle, and radius, and flag when the physics stop being safe. A language model can summarize a maintenance report. A world model can simulate what happens to the turbine blade under those conditions and estimate the time to failure.

The deeper consequence is what world models let an agent do. A pure language-model agent in a physical environment is brittle: it pattern-matches over text, has no causal substrate, and confuses when it encounters something it hasn't seen. Fine for a chatbot. Dangerous on a factory floor. A world-model-equipped agent can run a proposed action through its internal model of physics, rehearsing thousands of rollouts and selecting the one whose simulated consequences look best. This is the same machinery AlphaZero used in Go and DreamerV3 used to learn Minecraft from scratch. The leap to industrial environments is real and largely unproven. But if the bet pays off, it changes what a physical agent can be trusted to do unattended.

The landscape is moving fast. NVIDIA's Cosmos is learning physical laws from curated and synthetic video. Fei-Fei Li's World Labs launched Marble for generating interactive 3D worlds. LeCun left Meta to found AMI Labs to build systems that "understand the physical world, have persistent memory, can reason, and can plan complex action sequences." And the pace is not exclusively Western: China's 2025 Government Work Report named embodied AI a core tool for next-generation industries, Unitree is shipping its G1 humanoid at roughly $16,000 per unit, and Chinese labs are publishing world-model results at a cadence that rivals their Western counterparts. The foundation layer is still in flux, and that instability is part of the opportunity.

World models also face a problem that language models sidestepped. The internet provided a near-infinite corpus of text. There is no equivalent for the physical world. Operational data is fragmented across thousands of operators, locked inside proprietary control systems, and mostly thrown away within days. The companies that turn industrial operations into a continuously generating training corpus are the ones that will own the foundation on which everything else gets built. We'll return to this in Part 3.

The Brain Is Not Enough

Go back to that Houston job site. A world model could tell you, precisely, what happens to a suspended load at a given wind speed, boom angle, and radius. It could flag the exact moment the physics stop being safe.

But the wind forecast lives in one system. The shifted pour schedule lives in another. The rebar flag is buried in a PDF. A brain that can reason about physics, with no way to see across those systems and no authority to act on what it sees, is a brilliant consultant locked in a windowless room.

What connects the reasoning to the job site is the orchestration layer. We call it the harness, and it is where the next decade of physical AI will be won or lost. That's Part 2.

Part 2, on agentic orchestration, will be published soon. Part 3 covers compounding memory: the asset we believe defines the category.

‍