Climbing the Scaling Wall

Last month, Gavin Baker in ILtB podcast mentioned the following:

“I think there's been a big misunderstanding of maybe in the public equity investing community, or the broader more generalist community, based on the scaling laws of pre-training, there really should have been no progress in '24 and '25.

…all the progress we've had, and we've had immense progress since October '24 through today, was based entirely on these two new scaling laws. And Gemini 3 was arguably the first test since Hopper came out of the scaling law for pre-training, and it held, and that's great, because all these scaling laws are multiplicative. So, now we're going to apply these two new reinforcement learnings, verifiable rewards and test-time compute, to much better base models.”

As one of the generalists, I was initially comforted by the idea of the multiplicative nature of these scaling laws. But it turns out there are indeed plenty of discomforting questions lurking around scaling laws. After publishing “Scaling Law Skepticism” a couple of days ago, I received several interesting emails. One smart reader stood out when he shared some of his thoughts on the “church” of scaling; his thoughts included a couple of reading materials which I went through yesterday and I came away with more nuanced concerns around Scaling Laws. I will discuss some of these nuances behind the paywall.

In addition to “Daily Dose” (yes, DAILY) like this, MBI Deep Dives publishes one Deep Dive on a publicly listed company every month. You can find all the 65 Deep Dives here