The Long-term Reality of Hyperscalers: The Good, The Bad, and The Ugly Scenarios

In my “Lipstick on Frontier AI Pigs” piece, I wrote the following:

Apart from ignoring the cost of training, another deeply questionable assumption Amodei makes in his argument is the gross margin for inference. In his “stylized” example, he assumes gross margin to be 67% which is directly responsible for their ability to fund training the next model. How can we assume the gross margin will remain so high when its more well funded competitors know they can exert unduly amount of pressure on Anthropic if they take the gross margin on inference to…say 30%? If Anthropic cannot get enough gross profit from today’s model, its hands will be tied for training the next one.

Three days later after publishing my piece, The Information reported about OpenAI’s financials from which this particular excerpt stood out to me:

“OpenAI has told investors the costs of running its AI models, a process known as inference, quadrupled in 2025. As a result, the company’s adjusted gross margin—defined as revenue minus the costs of inference—fell to 33% from 40% the year prior. That’s lower than the gross margin expectations of 46% it had set for itself for 2025.”

The Information mentioned OpenAI suggested the falling margin was largely due to “the company having to buy more expensive compute at the last minute in response to higher than expected demand for its chatbots and models”. While this supports Dario Amodei’s point about “hellish demand prediction” between inference and training, it is hard not to think that finding the right balance can remain just as challenging which may not alleviate the gross margin challenge. Of course, OpenAI then promised with a straight face that they expect inference margin to increase to 52%-67% in the next 5 years. Let me emphasize this point. As of today, OpenAI (or anyone else) alone does not decide what their gross margins will be this year, or the next…let alone in 2030. Apart from the challenges related to “hellish demand prediction”, the model race itself remains quite competitive which is the primary driving force for gross margin today.

Ultimately, improving gross margin is primarily a function of pricing power and better cost structure. Even if you made perfect demand prediction for inference, your more well-funded competitors can keep their pricing low if they have numerous other sources of internally generated cashflows. This may of course force OpenAI and Anthropic to lower their API prices much faster than they would prefer. The problem exacerbates even further when such well-funded competitor e.g. Alphabet also has a better cost structure since they’re much more vertically integrated in their model development stack than OpenAI and Anthropic are.

I will also note that I’m discussing the gross margin question by accepting the labs’ lenient definition of gross margin which ignores the cost of training. The Information also mentioned OpenAI “plans to increase its training costs to $32 billion this year and $65 billion next year, or about $44 billion more than previously expected.” Again, the cost of training is also not quite under OpenAI or Anthropic’s control. If their competitors can invest $100 Billion on training and performance of future models largely follow scaling law, OpenAI and Anthropic must be willing to invest in training in the same vicinity to stay competitive. I think you can appreciate why I find the end state economics of AI labs rather a bit…hazy!

One of the strangest counters I have heard from my readers in the last few days is why I am assuming OpenAI and Anthropic cannot make the math work at scale. The reason it is perplexing to me is I am certainly not making the case that it is an impossible scenario. If either or both of these companies are able to find a technological breakthroughs that are time consuming or very hard to replicate by their more well-funded competitors, I can imagine how they may ace the economics questions in the future. However, it is very important to acknowledge that the investors of these labs have largely decided that is exactly going to be the case which explains their recent funding rounds at massive valuations. All I am pointing out is that there are compelling reasons to be skeptical that the economics question will be solved in the private AI labs favor. Again, if your argument is only a handful of companies will survive this race in 3-5 years and hence the oligopolistic industry will have attractive terminal margins, I have already addressed this point in my “lipstick in frontier AI Pigs” piece. I’m not saying that it’s impossible, but I’m trying to drive the point home that we have far more limited predictability of such “rational” margin structure in the long-term and hence, there are wider possible scenarios than are being appreciated in private AI labs valuation.

In most scenarios of AI labs future, one industry whose long-term economics is potentially more vulnerable than currently appreciated by investors is hyperscalers. While we can come up with endless scenarios in our head, I will outline three specific scenarios behind paywall to help you appreciate the range of potential outcomes here.

In addition to “Daily Dose” (yes, DAILY) like this, MBI Deep Dives publishes one Deep Dive on a publicly listed company every month. You can find all the 66 Deep Dives here.