Better Training Loops Will Matter More Than Bigger Models

I think parameter count is about to become a second-order metric for working engineers. Bigger base models will remain necessary, but the systems that materially change what we can build will be distinguished more by their training loops—reinforcement learning, synthetic tasks, self-play, verifiers, and carefully bounded learning at inference time—than by how many billions of parameters sit in a checkpoint.

That is a disagreeable claim because scale has earned its prestige. Scaling models, data, and training compute produced the general-purpose language systems we use today. It would be foolish to announce that scale is over. But it is equally foolish to act as if a model with twice as many parameters automatically represents the most important advance. A parameter count tells us how much capacity a model has. It says far less about whether the model was taught to persist through a messy repository, check an answer against reality, recover from a failed tool call, or notice that its own plan is wrong.

The new unit of progress is the feedback loop

Pretraining is extraordinarily effective at giving a model broad priors: language, code patterns, facts, conventions, and a hazy map of how the world tends to work. Yet next-token prediction is a weak substitute for feedback when a task has an objective result. A compiler can say whether code builds. A test suite can say whether a behavior survives. A theorem checker, simulator, benchmark harness, or transaction log can provide similarly useful signals. Once a system can cheaply generate attempts and score them, it can learn much more directly than it can from another pass over undifferentiated web text.

This is why reinforcement learning has become more central to reasoning systems. DeepSeek-R1 made the point unusually visible: its report describes reasoning behaviors emerging from large-scale RL, then uses a multi-stage pipeline to improve quality and performance. The important lesson is not that RL is magical or that one lab has found the final recipe. It is that the shape of the objective changes behavior. If a model is rewarded for reaching verifiable solutions rather than merely producing plausible continuations, it has a reason to spend tokens checking work, revising an approach, and carrying a calculation to completion.

For developers, that matters more than the marketing distinction between a 70B and 140B checkpoint. A smaller model inside a strong loop can generate a patch, run focused tests, inspect the failure, narrow its hypothesis, and try again. A larger model asked for one perfect answer gets only one chance to be correct. The first system is not necessarily smarter in some metaphysical sense. It is better engineered for the job we actually gave it.

Synthetic data is useful when it has friction

Synthetic data is often described as models training on their own output, which makes it sound either revolutionary or obviously doomed. Both reactions miss the engineering question: what selects the output? Synthetic data without selection is mostly an expensive way to amplify a model’s habits. Synthetic data paired with execution, critique, diverse generation, and hard acceptance criteria can become a factory for useful training examples.

Consider a coding model. It can propose many implementations for a small issue, execute tests in isolated environments, retain the patches that satisfy the specification, and turn failures into adversarial examples. That is not an infinite free lunch: the tests may be shallow, the specification may be incomplete, and the model may learn to exploit quirks in the harness. Still, this loop creates something plain pretraining cannot: examples shaped by the exact boundary between working and non-working software.

Self-play is the extreme version of the same idea. AlphaZero demonstrated how far a system can go when it can generate endless competitive experience and receive an unambiguous result. Language and software work are not chess. Their rules are incomplete, objectives are multi-dimensional, and many valuable judgments are subjective. But software has more game-like structure than we admit. Compilers, linters, type systems, tests, fuzzers, performance profilers, deployment checks, and user telemetry are all partial environments. We should be building models that learn to operate against those environments, not merely models that have read more code from them.

Inference-time learning is the risky, exciting extension

Inference-time learning deserves more skepticism than inference-time search. Letting a model reason longer, call tools, or sample several candidate answers is already useful because it spends additional compute without changing weights. Updating weights during deployment is a different promise: the system adapts to a domain from fresh signals. Early work on test-time learning suggests lightweight adaptation can improve performance under distribution shift, including through techniques such as LoRA updates. That is interesting, but it is not a license to let production agents rewrite themselves.

The practical version should look boring. Learn from bounded, attributable data. Keep updates isolated from the base model. Evaluate against a fixed regression set before promotion. Log the reward signal and the data lineage. Maintain rollback. Treat every adaptation as a deployable artifact, not a private mental improvement. If your coding agent learns from a repository’s CI results, you should be able to answer which results influenced it, what capability changed, and how to undo that change.

The strongest objection: capacity still comes first

The strongest counterargument is that better loops cannot manufacture capabilities a model does not possess. RL only optimizes a policy within the model’s representational limits; synthetic data is bottlenecked by the generator and the verifier; self-play works best in closed worlds with crisp rewards; and online learning can overfit, drift, or reward-hack. On this view, frontier-scale pretraining remains the real source of capability, while all these loops are just ways to polish it.

That objection is substantially right, and it is exactly why I am not arguing for small models forever. Training loops do not repeal the need for capable foundations. A weak model cannot reliably invent the abstractions needed to solve hard new problems, and a bad evaluator can train a capable model into a highly efficient cheat. But “capacity comes first” does not imply “parameter count matters most.” Once a model crosses the threshold needed to generate viable attempts, the quality of feedback can decide whether it becomes a fluent assistant or a dependable worker. The gap is especially large in domains where correctness can be checked mechanically.

We should therefore change how we evaluate and buy models. Stop asking only which checkpoint wins a static benchmark at one sampling temperature. Ask what loop it supports: Can it generate diverse attempts? Can it use tools robustly? Can we verify outcomes cheaply? Can it learn from failures without corrupting a production system? Can we reproduce the trajectory that improved it? Open-weight developers should invest at least as much in reusable evaluators, sandboxes, task generators, and training traces as in publishing another parameter-count headline. The competitive advantage will belong to the teams that can turn real feedback into better behavior repeatedly—and prove that they did.