Less is more?
Just yesterday, I was wondering about the profound impact “the cat paper” had on so many companies and wrote “Perhaps someone is going to publish a paper today or in a couple of years that may fundamentally alter the tech landscape 5-10 years from now. This makes analyzing some of these tech companies and investing in them potentially harder than appreciated.”
Then I came across a tweet by Gavin Baker later yesterday: “get ready to hear a lot about TRMs”. When I asked Grok about the tweet, Grok thought Baker is talking about “Trust, Risk, and Security Management”.
Well, Grok is wrong. He is almost certainly talking about “Tiny Recursive Models” (TRM), an idea which just came out of Samsung SAIL (Samsung Advanced Institute of Technology AI Lab).
SAIL’s Alexia Jolicoeur-Martineau published a paper just last week titled: “Less is More: Recursive Reasoning with Tiny Networks” which can indeed cause some uproar in the coming days. The paper challenges the prevailing assumption that larger models are inherently better. It demonstrates that tiny, specialized models can outperform the largest language models on complex reasoning tasks by employing an iterative self-correction mechanism.
The central thesis of this paper is that for difficult logic problems, how an AI model thinks (its reasoning process) is more important than its sheer size.
Large Language Models (LLMs) like Gemini or GPT-4 typically generate answers auto-regressively i.e. one word or token at a time. When solving a complex problem, they try to produce the solution in a single forward pass. If they make an error early on, they struggle to recover, often rendering the entire answer invalid.
To compensate, LLMs use techniques like Chain-of-Thought (CoT), where they write out their reasoning steps, and Test-Time Compute (TTC), where they generate multiple answers and select the best one. However, these methods are computationally expensive and still fail on tasks requiring deep, multi-step logic, such as complex Sudoku puzzles, maze solving, and abstract reasoning benchmarks (i.e. ARC-AGI)
The paper introduces the Tiny Recursive Model (TRM), which builds upon a previous approach called the Hierarchical Reasoning Model (HRM). HRM showed that small networks using “recursive reasoning” i.e. repeatedly checking and refining their answers could beat LLMs on these hard tasks. However, HRM was overly complex and inefficient.
TRM significantly simplifies and improves this concept. It operates much like a human solving a difficult puzzle: iteratively making guesses, evaluating the consequences, and revising the guesses until a solution is found.
Instead of a huge model that tries to think once and be done, use a tiny model that thinks in loops. It keeps two things in memory: y: its current best guess for the answer, and z: an internal “scratchpad” of reasoning. At each step it updates z a few times, then updates y once, and repeats. That’s it. The loop lets a 2‑layer, ~7M‑parameter network iteratively fix its own mistakes (“less is more”).
Surprisingly, the authors found that smaller networks (only 2 layers deep) performed better. This is because when training data is scarce (e.g., only ~1,000 examples), large models tend to “overfit” (memorize the data), whereas tiny models are forced to learn the underlying logic.
TRM, with only 7 million parameters, outperforms models like Gemini 2.5 Pro on the hard reasoning ARC-AGI benchmarks, despite having less than 0.01% of the parameters. It also significantly advanced the state-of-the-art on Sudoku-Extreme (from 55% to 87% accuracy).
What are the implications of this paper? I will share some thoughts behind the paywall.
In addition to “Daily Dose” (yes, DAILY) like this, MBI Deep Dives publishes one Deep Dive on a publicly listed company every month. You can find all the 63 Deep Dives here.