What interests me about the energy question is how physical AI becomes once you look beyond the screen. Every interaction depends on machines, power and somewhere to put them. However capable the software becomes, it still needs that infrastructure.
On 25 August, OpenAI published the first measured performance results from Jalapeño, its custom inference chip. The company reported improvements in throughput per kilowatt and token latency against the commercial systems in its comparison. On the same day, OpenAI described its compute strategy as one integrated stack spanning data centres, chips, frontier models, developer infrastructure, products and AI-native devices.
That is a useful reminder. The race to make intelligence abundant is also a race to supply the physical resources intelligence consumes.
Software has become physical again
The cloud trained a generation of software companies to think of infrastructure as effectively infinite. Need more compute? Provision it. Need more storage? Buy it as a service. The abstraction was commercially transformative because developers could build without thinking very hard about the machines underneath.
Frontier AI is putting the machines back into the conversation. The workloads are large enough that chip architecture matters.
Power efficiency matters. Data-centre location matters. Grid capacity matters. Cooling matters. Supply chains matter. Capital matters.
The experience remains software. The constraint increasingly looks industrial.
Inference is where economics become real
Training frontier models attracts attention because of its scale. Inference is the work of running the model when people use it. That turns usage into an ongoing operating cost.
If billions of people and millions of businesses use AI continuously, small improvements in the efficiency of each interaction compound across enormous volumes. That is why tokens per kilowatt is not an obscure engineering metric. It connects model economics to energy economics.
A system that can deliver more useful intelligence from the same physical resources can serve more demand, lower cost or both. OpenAI’s Jalapeño announcement is therefore not just a chip story. It is an economics story.
Demand expands when cost falls
I am optimistic about what this investment could create. Building and operating infrastructure means work, and the capability it supports could make new businesses possible. But I would separate that hope from a claim that everyone automatically benefits. People can lose work during a transition even if the economy eventually becomes more productive. The timing and distribution matter.
There is a trap in thinking that efficiency automatically means lower total energy consumption. History often shows the opposite. When something becomes cheaper, we find more uses for it. Cheaper compute did not make the world consume less compute. Cheaper storage did not make us store less data.
If the cost of useful machine intelligence falls dramatically, demand may expand faster than efficiency improves. Businesses will analyse more. Generate more. Simulate more. Personalise more. Automate more. Build software that would previously have been uneconomic.
For me, that leaves two moving parts: how much energy each task needs and how many tasks become worth doing. I would be cautious about drawing a straight line from today’s usage to a future power requirement. Efficiency and demand can both change.
AI efficiency does not necessarily reduce the infrastructure race. It may be what makes the race bigger.
Energy becomes a technology strategy
This has consequences far outside the AI labs. Data-centre demand connects AI growth to utilities, grid operators, energy developers, governments and local planning. Nuclear, renewables, gas, storage and transmission all become part of a technology conversation that once belonged mostly to software.
That changes where strategic advantage can sit.
A company with better access to compute can train and serve more models. A region with abundant reliable power can become more attractive for data-centre investment. A chip that produces more output per watt can change operating economics at enormous scale. The AI stack therefore extends much further down than the application layer most businesses see.
Most companies should not buy chips
There is an obvious danger here. A CEO reading about custom silicon does not need to create a semiconductor strategy. For most organisations, the infrastructure will remain somebody else’s problem. But its economics will arrive in the products they buy.
Model prices. Latency. Availability. Capability. Data-residency choices. The viability of running models locally. The cost of embedding AI into high-volume customer experiences.
Infrastructure determines what becomes economical at the application layer. That is why business leaders should care without pretending to be chip designers.
Abundance changes behaviour
OpenAI describes the goal as abundant intelligence. Take that idea seriously for a moment. If high-quality reasoning, generation and software execution become cheap enough to use almost continuously, businesses will stop rationing them in the way they do today.
We currently decide whether a task is worth somebody’s time. In an abundant-intelligence environment, many more tasks can receive analysis by default.
- Every customer interaction can be examined.
- Every contract can be checked.
- Every operational anomaly can be investigated.
- Every employee can have access to specialist assistance.
- Every small process can potentially have software wrapped around it.
That is the upside of the infrastructure build-out. It is also why the physical demand could become so large.
Constraints move rather than disappear
Technology progress often removes one bottleneck and exposes another. More capable models expose the need for more inference. More inference exposes the need for more chips. More chips expose the need for more power and data-centre capacity.
More AI inside businesses exposes the need for better data, governance and implementation. The mistake is assuming that a breakthrough removes scarcity altogether. Usually it relocates it. For the AI economy, some of the most important scarcity may sit surprisingly far from the model.
Watch the watts as well as the benchmarks
Model benchmarks will continue to dominate technology headlines because capability is easy to compare and exciting to watch. I think we should pay more attention to the denominator.
- How much energy?
- How much compute?
- How much capital?
- How much useful output per unit of each?
Those numbers determine how widely intelligence can spread through the economy. Jalapeño is one company’s answer to that problem, and first-party benchmark results should be treated as exactly that: results published by the company that built the chip.
But the direction is unmistakable. The AI race is not taking place only inside neural networks. It is taking place in fabs, data centres, substations and power markets too.
I still expect the software to improve quickly. I am less confident about how quickly the physical constraints can move with it. That is why I want to follow the power and infrastructure alongside the model releases.
Sources: OpenAI — Jalapeño first results · The full stack behind abundant intelligence