This blog post is about enterprises consuming (artificial) intelligence as a commodity. I have a load of cool product stuff in the pipeline, but wanted to step up and do a “macro view” piece. This question matters:

What happens when everyone needs to spend bet-the-company money on AI?

Twenty years ago AWS launched the S3 cloud storage service and thus began the industry race to commoditise on-demand computing. Today, every company’s standard infrastructure line items are indeed commodities: compute, storage and network.

Three commodities: all bought from a handful of suppliers. Feature parity is expected. Instead CxOs ask: How much are we spending? What are we locked into? What happens when they put the price up? Can anyone constrain our use? For vendors it’s: how fast can we fund and build?

AI Inference is must-have infrastructure

There is a fourth line item now, and it is AI inference. In some companies it is already bigger than storage. The same CxOs are asking the same questions: How much? How fast? How soon? That is what makes it a commodity, rather than any claim that models are becoming alike. A commodity is something you purchase by the unit, from somebody, and have to keep buying.

So everything that happened to the first three commodities starts happening to it. When AI was a feature you bought to solve a problem using one vendor was a preference. Now, when it is a commodity you run on, using one vendor is a dependency, and enterprises do not accept single dependencies for anything they treat as infrastructure.

The exposures:

  • Prices can and do move — see eg. memory in the last year

  • Feature lock-in and deprecation, because your supplier can retire the model you depend on with a few weeks notice, and the replacement does not behave exactly the same way.

  • Capacity: every LOB wants the best GPUs at the same time.

  • Terms of use change in surprising ways. Looking at you, big tech.

  • Jurisdiction, which decides where your data may go and whose law reaches it.

So the interesting question is not whether large companies will use more than one source of inference. They will, the same way they ended up with more than one of everything else.

The question is what that actually looks like.

Five Ways to Diversify Intelligence

How will the C-team place their bets?

Build your own, on open weights. The drivers are cost and data. At low volume renting is obviously right. At high volume using foundation models is like using professional services: if it is not adding 4x or more to the top line then it is eroding margin and you’re in trouble. Unless you build your own model from one of increasingly many powerful ‘open’ options. This also keeps high value proprietary data inside your company, which can be a compliance red line.

Regional. The driver is sovereignty. A national model rather than one belonging to another country, and in a growing number of places that is a rule rather than a preference. The risk: what happens if country A stops exports of technology B to countries C but not D?

Vertical. The driver is “fitness”. The big one here is Security: which bleeding edge model will find and fix bugs the fastest? Remember we are in “Mythos time” and it is a race for survival. Or look at models trained on case law, or on markets, or fund management, or on insurance claims. The latter will be bought by LOBs and not by central IT or platform.

Small and large. The driver is unit economics. Most work does not need the biggest model. Routing the cheap work to something small is where the bill actually falls. Smaller models may also mean less hardware, and less corporate ‘ceremony’ in general. Good for: customer-facing.

More than one supplier. The oldest reason of all. Service continuity and negotiation flexibility. Nobody argues with it once the line item is big enough to notice.

Each of those is sensible on its own and so, frequently, each one of them is arranged separately and not coordinated with the others. In a global organization, then, enterprise IT does not end up choosing a strategy. It ends up running four or five arrangements at once.

How and where do I run the AI workloads?

Once you have a few diversified options, what’s next? Let’s do some work. But where?

The hipster answer is that agent workloads need something new. Fresh substrate, purpose built, unencumbered by the last decade. Greenfield GPU stacks, SaaS dev/test, VM private cloud… And for some companies that could be the right answer: eg guerilla teams moving super fast.

For most enterprises though: They spent the last fifteen years installing Kubernetes and virtual machines and cloud storage before that. It is paid for, it is staffed, it has contracts behind it and people who know how it behaves at three in the morning. They want AI to be the same.

Which means AI lands on the Kubernetes you already have.

AI in the software lifecycle. Making your existing applications and your existing operational tooling AI-enabled. This is the largest by count and the least discussed, because it looks like ordinary work: get rid of CVEs, speed up continuous delivery, automate fault resolution.

Agents at scale, single tenant. Deploy something agentic on the clusters you already run and you have thousands of agents in your own cloud, without worrying about where the compute is, or having to sign new contracts for it. It is worth noting that this is small compute rather than GPU, which is why it can go on what you have.

New GPU stacks. Let’s use Neocloud and Nvidia patterns for standard inference solutions. Guess what, a lot of these use Helm, Kubernetes, GitOps and can be part of the enterprise platform engineering systems — now rebranded “Agentic Platform 2.0” or something :-)

All of these are variant use cases for Kubernetes. That means we can package and manage them as configurations. Enter ConfigHub: a central control plane and storage system of record for configuration and any operational or agentic facts that are represented as configuration data.

Let us count the configurations

How many choices did you make?

  • AI Inference models

  • Suppliers and OSS

  • Hardware resources: GPUs, network and data

  • Use cases

  • Data access

  • Compliance

  • … more …

Every combination needs to be told which model, which version, in which region, through which endpoint, with what fallback, for which workload, in which environment, under which contract, at which price, with which data allowed to reach it. And the configuration surface is growing.

Then you have management and operations, such as:

  1. Which model version is production calling right now. Not which one is in the code. Which one is being called, this afternoon, and in which user team or company region.

  2. Who approved the switch to it, and when, and what happened to quality when it moved.

  3. Can you roll one region back and leave the others alone.

  4. And when a supplier gives ninety days notice on a model you depend on, how long does it take to find everything affected. Like a CVE patch day but worse…

What to do about it

Treat it as configuration, because that is what it is, and put it somewhere you can query.

That is what we built ConfigHub for. Your configuration lives as data with a history, so which model is being called where is a question with an answer. Changing it is a reviewed change, so somebody approved this exact version and you can see who. Environments keep what they need to keep, so a regional or vertical difference is a declared exception rather than a fork. And you can roll one back without touching the others.

None of that is completely specific to AI. It is what a system of record does. It just happens that the fourth commodity is arriving faster than the first three did, on infrastructure that is already full, and it is going to be configured by more people, in more places, than any of them were.

Next: email me.

Or: take the tutorial! It only needs fifteen minutes.

Or: read more blogs!

– alexis

so long!