The "Edge AI" Bear Case
On “Invest Like The Best” podcast yesterday, Gavin Baker mentioned the following:
“In three years, on a bigger phone, you’ll be able to run a pruned-down version of Gemini 5, Grok 4, or ChatGPT. And that’s free. This is clearly Apple’s strategy - we’re going to make it privacy-safe and run on the phone. Other than scaling laws breaking, edge AI is by far the most plausible and scariest bear case.”
One of my hypotheses in investing in general is most bear cases never truly die; they just keep re-appearing time and again and most investors start paying more attention to these perpetual bear cases only when stock prices force them to i.e. when stocks go down a lot. To Baker’s credit, he’s highlighting a risk when stocks are actually doing fine.
But indeed, edge AI has been touted as a plausible bear case for the “AI trade” for much of the last couple of years. Nilesh Jasani was one of the investors who actually took this bear case seriously and explained the appeal of edge AI:
“In late 2023 and early 2024, the promise of edge computing was not a niche technical idea; it was a palpable hum of excitement. The logic was clean and compelling. Moving artificial intelligence to the edge would solve the technology’s most pressing problems.
First, there was privacy. Processing data on your own device meant it never had to travel to a server owned by someone else. Your secrets would remain your own. Second was latency or annoying delays. With the thinking done locally, responses would be instantaneous, a crucial feature for everything from conversations with voice assistants and augmented reality to autonomous vehicles. Finally, there was cost and access. Why rent time on a remote supercomputer when your own device could do the work? This would democratize AI, making it reliable even without a perfect internet connection.”
However, Jasani admitted that it was “wrong”. The reality didn’t match edge AI’s promises at all. From his piece again:
“Top-tier smartphones like Apple’s iPhone 16 and Samsung’s latest Galaxy look and work pretty much like the models from a few years prior. Yes, they tout AI-powered camera modes and smarter assistants, but there’s little truly new that runs independently on the device. All the major phone launches emphasized AI features, yet those features often amounted to things like slightly better autocorrect, AI image editing, or voice dictation – useful, sure, but hardly the seismic shift envisioned by edge computing evangelists.
Notably, many of these “AI features” still lean on the cloud. Apple’s vaunted Apple Intelligence gave iPhones some on-device smarts, but even Apple admitted that for more “sophisticated commands” the system quietly reaches out for “help from ChatGPT” or other cloud models. In other words, your iPhone might rewrite a text message offline, but if you ask it to compose a complex email or summarize a document, it’s likely pinging a server somewhere. Apple’s commitment to privacy means those requests are anonymized and encrypted, but they’re not happening on your phone’s CPU alone.”
I recently bought an “AI PC” and the salesperson at “Best Buy” was passionately making the case that conversations on CoPilot stays on device and not on the cloud. That’s not true. So, there is clearly a gulf of difference what edge AI has promised so far and how much they could make it work.
But is it just a timing mismatch and much of the promises of edge AI will eventually bear fruit? If edge AI becomes reality in the medium to long term, the biggest losers are likely to be hyperscalers who are hoping to generate billions of inference revenue on their ongoing capex spree. Since they are Nvidia’s biggest customers today, you might imagine Jensen Huang to be highly incentivized to not believe in edge AI’s promises. Interestingly, Huang himself in his recent Joe Rogan podcast appearance made the case that edge AI is inevitable:
“the fact of the matter is your phone’s going to run AI just fine all by itself in a few years. Today, it already does it fairly decently. And so the fact that every country, every nation, every society will have the benefit of very good AI. It might not be tomorrow’s AI. It might be yesterday’s AI, but yesterday’s AI is freaking amazing. You know, in 10 years time, 9-year-old AI is going to be amazing. You don’t need 10-year old AI. You don’t need frontier AI…we need frontier AI because we want to be the world leader. But for every single country, everybody, I think the capability to elevate everybody’s knowledge and capability and intelligence, that day is coming.”
If edge AI starts to deliver its promises, I wonder if it’s an underappreciated headwind for OpenAI. Following GPT-5 backlash, Sam Altman once tweeted, “the percentage of users using reasoning models each day is significantly increasing; for example, for free users we went from <1% to 7%, and for plus users from 7% to 24%.”
It was a bit surprising to me that vast majority of ChatGPT users are not using reasoning models. If your queries can be responded without reasoning, those are exactly the type of queries that can be easily done by on device models in a few years. And if the free model on your device can have private, uninterrupted (personality won’t change due to any model changes) conversations, would ChatGPT remain compelling enough for incremental users to pay for subscription?
There is indeed a real possibility that “System 1” (fast/reflexive) queries are just commodities and it is the “System 2” (deep thinking) queries that prove to be high-value ones. It may be instructive to think why Anthropic is almost solely focusing on “System 2” queries whereas ChatGPT’s vast majority of queries are still likely to be “System 1” queries. System 1 queries can be quite profitable (very low inference costs now which may even be almost zero in edge AI scenario), but it is also going to be very hard to convince users that it is worth paying subscription for in a few years.
Of course, we cannot quite outline an impact of a certain risks playing out imagining ceteris paribus. Sam Altman has already declared “code red” in the face of Gemini’s recent growth spurt, and OpenAI will certainly not sit idly if edge AI ever starts proving to be a headwind for user growth. Let’s not forget that OpenAI still has the “Midas touch” when it comes to product itself. They may be behind on model quality, they may be less focused than Anthropic, and they clearly don’t have the infrastructure advantage as Google does, but AI still is synonymous with “ChatGPT” for most people. That itself will likely buy them time, but the clock is ticking.
In addition to “Daily Dose” (yes, DAILY) like this, MBI Deep Dives publishes one Deep Dive on a publicly listed company every month. You can find all the 65 Deep Dives here.
Current Portfolio:
Please note that these are NOT my recommendation to buy/sell these securities, but just disclosure from my end so that you can assess potential biases that I may have because of my own personal portfolio holdings. Always consider my write-up my personal investing journal and never forget my objectives, risk tolerance, and constraints may have no resemblance to yours.
My current portfolio is disclosed below: