Two Kinds of Test-Time Scaling
Citations
9th November 2026
Test-time scaling trades compute for performance at test time. Its adoption, chiefly via long RLVR-trained chains-of-thought, has likely been responsible for a speed-up in the rate of progress of general AI capabilities.
One can make a distinction between two different possible kinds of test-time scaling. One kind lets you trade more compute for more performance per task, but only up to the maximum quantity of per-task compute used during training. The other kind lets you trade more compute for more performance per task, and lets you do so up to quantities of compute (and performance) that never occurred during training. Scaling methods that permit this latter change display what I call “compute extrapolation.”
I currently believe that compute extrapolation:
- Likely accelerates the speed of advance of the most impressive AI capabilities, makes dangerous capability evaluations much harder, and changes AI race dynamics
- Is a more recent phenomenon than general test-time scaling, and one still being mastered by AI labs; it’s likely that the ceiling and slope of compute extrapolation will improve in the future through better LLM context management, multi-agent scaling, and other advances.
- Is a broadly important thing to notice and measure, and is one that is likely being actively tracked by labs because of the above.
It is possible that the impact of test-time scaling with compute extrapolation will be larger than the impact of test-time scaling considered simply. In this essay, I make this distinction, describe some ways it might impact the world, and describe some ways we could react to it.
1. Compute Extrapolation
1.1 What is it, and how does it relate to test-time scaling?
Test-time scaling methods trade more compute at test-time for better performance.
Many of the most effective such methods are learned rather than applied "externally" at test-time, like plurality voting. For example, some learned forms of test-time scaling include:
- length of chain-of-thought, for an LLM trained with RLVR
- number of loops, for a looped Transformer
- number of agents, in a multi-agent LLM swarm
Such learned methods will use a maximum amount of compute per task over the course of training. So, for instance, during training they will use a maximum length of chain-of-thought, or a maximum number of loops within a looped Transformer, or a maximum number of agents in an LLM swarm, and so on. This training-time maximum quantity of compute could then either (1) limit or (2) not limit the useful maximum test-time quantity of compute per task.1
Consider the case where a train-time maximum limits the effective test-time maximum. So – to use the easiest example – imagine that during RL some LLM was trained with a maximum of 100k chain-of-thought tokens. More tokens would then improve its performance at test-time only up to 100k tokens; one million or a billion tokens would not help. As I’ll discuss later, there’s pretty good evidence this is the state that LLMs such as o1 or R1 were in when trained with the earliest RLVR methods.
But consider the case where a train-time maximum does not limit the effective test-time maximum. So, again, imagine that during RL some LLM learned with a maximum of 100k chain-of-thought tokens; but a million or a billion tokens keep improving its performance at test-time. I call this kind of test-time scaling compute extrapolation, and, as I will discuss below, evidence suggests that labs only started to master this within the last twelve months, and perhaps only in some domains.
So far as I can tell, this distinction is not often made in discussions about test-time scaling.2 One reason people might not make this distinction is that it is mostly invisible to lab outsiders, who do not know the per-task training compute distribution. But, as I discuss later, the overall effect of compute extrapolation could be very large regardless of low initial external visibility.
Before I describe ways to measure compute extrapolation, let me amend the above definition of compute extrapolation slightly. Right now, it is too sensitive to per-task compute outliers during training.3
Imagine a training run where all tasks spend 10^17 FLOPs per task. If one can improve performance through some method by spending 10^21 FLOPs per task at test-time, then – given the above definition – this method displays a large amount of compute extrapolation. Now, imagine a training run where all tasks but one spend 10^17 FLOPs per task – but one spends 10^21 FLOPs. Now, by the above definition, the ability to spend 10^21 FLOPs per task at test-time is no longer an example of compute extrapolation. But this is undesirable, because an AI is unlikely to have learned much from a single task on which a great deal of compute is spent. And it’s also undesirable because this definition makes measurements of compute extrapolation very sensitive to the outliers during training.
To fix this, I define compute extrapolation so that it exists relative to the per-task compute at the 99th-percentile of total compute usage. So suppose that when training some neural network, 99% of the total training compute is spent on tasks that individually use at most N FLOPs per task; and so at most 1% of total compute is devoted to tasks spending more than N FLOPs per task. If you can then use 10x N or 1000x N FLOPs per task, with the same method, at test time, and continue to increase performance by doing so, this method has some degree of compute extrapolation. The important thing here is not the specific 99% number – for my purposes, 98% or 99.9% would work about as well. What matters is that (1) there is negligible compute use during training above this task size and (2) the overwhelming majority of learning done by the neural network happens below this task size.
In any event – given this notion of compute extrapolation, how can you characterize it, and “how good” it might be? Consider two characteristics:
- Ceiling: As you increase per-task compute past N at test time, at what multiple of N does performance cease rising? If performance keeps going up until you’ve used 20x N compute per task, and then improvements stop, then there is a 20x extrapolation ceiling. Obviously, the exact ceiling will be fuzzy, measuring it might depend on noise, and so on. In many cases, I believe there will be no absolute ceiling, or that it might be extremely (1,000-1,000,000) high.
- Slope: Consider some function f from FLOPs to performance, which has been fit to the empirical FLOP-to-performance curve beneath N. For every point past the 99th percentile compute-weighted per-task compute N, what is the closest real number p such that f(N) + p·[f(FLOP) − f(N)] matches performance at that point? Such an extrapolation slope would probably decrease as FLOPs increased far past N, but could be useful for characterizing how much worse scaling works past the training-time compute limit.
In general, these numbers might vary by domain. It’s likely easier to get high ceilings on tasks where it is easy to recognize incremental progress, like math or cyber, than on tasks where it’s hard to recognize incremental progress. And so it’s also likely that compute extrapolation itself will extend to new domains as AIs get the ability to discern fine-grained differences in quality and subtle progress in them.
Image
Let me disambiguate a few points, and connect compute extrapolation to some closely related but distinct concepts.
- Compute extrapolation is a broad enough category that it can apply to many ways of varying compute per-task, such as number of tokens in chain-of-thought, number of agents in multi-agent scaling, and so on.
- Compute extrapolation on non-general, non-LLM AIs is likely much easier to produce than on general, LLM AIs. AlphaGo Zero, for instance, was trained using 1,600 simulations per move during training time, but could use far more at test time while continuing to improve.
- Compute extrapolation could overlap with, but is conceptually distinct from, horizon-length generalization in Cotra’s bioanchors report. This report discusses horizon-length generalization as something that could hasten the arrival of transformative AI. That is, if transformative AI requires acting on certain long-horizon tasks, then transformative AI will arrive sooner if one can train on a mixture of short-horizon and long-horizon tasks, or entirely on short-horizon tasks, while learning to act well on long-horizon tasks. This would be a form of compute extrapolation, granted that long-horizon tasks take more compute than short-horizon tasks. But not all forms of compute extrapolation necessarily involve horizon-length generalization. You can imagine compute extrapolation that works via increasing the number of loops through the layers of a Transformer, which improves performance on some particular short-horizon tasks but might not increase the horizon of the tasks that some LLM can do.
1.2 What would its consequences be?
Consider a world that moves from test-time scaling without compute extrapolation to test-time scaling with compute extrapolation. And let’s suppose that the extrapolation has a relatively high ceiling and slope, such that it might make economic sense to spend 100x or 10,000x as much compute per-task at test-time as was spent per-task at training-time. What changes?
It seems likely that (1) the ceiling of impressive AI capabilities will advance much faster; that (2) it becomes significantly harder to run evaluations of AIs that rule out whether they have dangerous capabilities; and (3) we start seeing huge inference-time compute expenditures, as labs race to reach certain goals.
First, the most impressive LLM capabilities, at the upper percentiles of test-time compute intensiveness, likely start advancing much faster.
The line of reasoning here is simple. Without compute extrapolation, if some company wishes to be able to spend 10^24 FLOPs on a highly consequential task at test-time – such as solving Navier-Stokes or, most importantly for a software intelligence explosion, finding more efficient machine-learning algorithms – they would need to have spent 10^24 FLOPs individually on many such tasks at training time. But 10^24 FLOPs of compute, although relatively cheap as a price for an important single result, is still quite expensive in aggregate for tasks performed at training-time, whose only long-term result is the learning they enable. So releasing this constraint allows vastly more to be invested in large test-time scale-ups.4
By analogy, consider superhuman Go algorithms, like AlphaGo Zero. AlphaGo Zero was able to use more compute per move at test-time than it ever spent at training-time. During training, AlphaGo Zero was permitted about 0.4s of thought per move; during one kind of evaluation, it was given 5s; and during a tournament, it was given about a minute, as the original AlphaGo Lee was given during competition. So this algorithm exhibits a compute extrapolation ceiling of at least 100x. If it had required training with this amount of compute-per-move, then the per-move costs of training would have gone up by 100x, although total training costs would have gone up by slightly less because it might have learned more per move. My estimate is that it would have taken 10x-50x more to train AlphaGo Zero to the same level of performance without compute extrapolation as it in fact took to train it with compute extrapolation. I believe the same thing is happening with LLMs.
On the other hand, why might you not see the ceiling of the most impressive tasks LLMs can do increase in its speed of advance?
- It could be the case that, even without compute extrapolation, tasks of every size that some LLMs could in theory do fall easily within the training budget of an LLM. I think such a case is theoretically possible, but extremely unlikely. The presence of hard upper limits to per-task compute in those training runs that have been made public also points against it.5
- Compute extrapolation on some domains might be much easier to obtain than others, and the extrapolatable domains might not matter. It’s true that the domain-specificity of compute extrapolation is very relevant; extrapolation in domains where progress is easy to identify, or where relevant state is easy to track, will be easier than in domains where progress is hard to identify, or where relevant state is hard to track. But even if there is domain-dependent difficulty, compute extrapolation will still save compute on those domains; and when it does arrive in harder but highly consequential domains like AI algorithms it still might matter.
All in all, I still expect the increase in the size of the largest tasks that AIs can perform to be very consequential. All other consequences are downstream of this.
Second, compute extrapolation means that dangerous-capability evaluations, aimed at establishing the upper limits of some AI’s capabilities, by default become impossible or very expensive.
Suppose you wish to know whether an AI is able to do some dangerous task, for instance one relating to CBRN capabilities. Without compute extrapolation, then to check this empirically you only need to spend up to the maximum amount of compute spent per-task in training.6
But with compute extrapolation, then to check this empirically you might need to spend arbitrary quantities of test-time compute. It is not strictly necessary for you to spend arbitrary amounts, if you can find some other way: for instance, if you can look at the scaling curves of the AI on smaller tasks or sub-tasks, it might be possible to determine some AI’s upper limits without actually trying to spend up to some potentially unknown compute extrapolation ceiling. But such theory-based reasoning is mostly not, at the moment, how such capabilities evaluations proceed. So dangerous-capabilities evaluation could become prohibitively expensive, or prohibitively expensive outside of all but the most important areas.
As the UK AISI has already noted, as LLMs get smarter, good evaluations of dangerous capabilities require spending more money on inference compute. Ultimately compute extrapolation likely terminates in a world where evaluations can only determine the absence of dangerous capabilities beneath some particular spending budget, and evaluators can only afford to actually spend such budgets in a few high importance domains.
Third, compute extrapolation implies that races in compute spending move from training-time to test-time.
Some kinds of research offer outsized returns to the first entity who advances the frontier. This can happen in the case of patentable research, prestigious research, or – perhaps most importantly – areas of research that promise to speed the future speed of advance of research and technology generally. Improved AI R&D is a subset of the kind of research on which it might be rational to burn almost arbitrary quantities of money in order to be the first.
Without compute extrapolation, it makes sense to spend most money towards this purpose on training compute, or on research that improves training compute. There’s no reason to spend more per-task than was spent per-task in training; in order to accomplish more impressive test-time tasks, training-time compute has to be scaled up.
By contrast, given compute extrapolation, it might make sense to spend millions of dollars on some task during test-time. Even if some such task in AI R&D might be possible for 10% or 0.1% of the cost using subsequent generations of models, the value of being first could still make it rational to spend much more on inference compute using earlier generations of AI. So, granted compute extrapolation, we will likely see increasingly large sums of money being spent on inference for AI moonshots, and increasingly large sums of money dissipated in failed bets on the possibility of such findings.
Finally, although this is not a consequence of compute extrapolation, I think it’s also important to note that AIs displaying compute extrapolation is evidence of better generalization in general.
That is, imagine that the per-task compute use during RLVR involves tasks using from 10^15 to 10^19 FLOPs, i.e., spanning 4 OOMs. Good compute extrapolation might mean that this RLVR will then generalize to tasks using, for instance, 10^20 or 10^21 FLOPs per task. But by the same token, the ability to generalize in this way likely implies that there will be generalization even within the distribution on which training actually occurs, i.e., generalization from tasks at 10^16 to tasks at 10^18, for instance. If this is so, then good compute extrapolation is also a sign of just having a generically efficient learning mechanism, and the externally-visible existence of compute extrapolation is evidence for an underlying improvement in learning generally.
2. Recent History
It is likely that reinforcement learning with verifiable rewards, as first implemented at the end of 2024,7 did not display significant degrees of compute extrapolation.
But a year later, at around the end of 2025, progress in “LLM compaction” (and other forms of LLM context management) probably started producing significant compute extrapolation. Such extrapolation has probably been responsible in part for the explosion in compute use by OpenAI and Anthropic researchers.
It is difficult to show directly that compute extrapolation started around the end of 2025. Leading LLM companies such as Anthropic and OpenAI do not release information about the distribution of per-task compute during training runs, so we cannot make direct inferences about compute extrapolation merely from their models’ efficacy at using per-task compute during test time. Even comparatively open Chinese models tend to only release very coarse-grained information about training runs. So the evidence below is necessarily circumstantial.
Nevertheless, I think that the indirect evidence that LLM compaction introduced effective compute extrapolation, at least around the end of 2025, within some domains, is suggestive.
I will first discuss academic evidence from papers on open-weight models illustrating how standard RLVR does not produce compute extrapolation with a high ceiling, but that RLVR through layers of compaction does. Then I’ll discuss circumstantial evidence that locates the start of compute extrapolation for Anthropic and OpenAI’s models around the end of 2025.
Note that I currently think that compute extrapolation is much better in some domains than in others; it’s likely that many future forms of progress in AI will consist in extending it to new domains.
Academic Evidence for LLM Compaction As Increasing Compute Extrapolation
“The Markovian Thinker” describes two methods of training with RLVR, with results measured on competition math (e.g., AIME). One of these produces very little compute extrapolation; the other produces a great deal. I’ll spend some time describing the methods and results of this paper, because they are very clear. And then I’ll briefly mention some other work that supports the conclusions of this paper in broad strokes.
The first method of training with RLVR-over-CoT allows the LLM to generate up to 24k tokens of chain-of-thought during training without interruption; the generated “thinking” tokens get appended to the context until the LLM comes to the answer. This corresponds to the standard, classic way of training with policy gradient as first publicly described in DeepSeek’s R1.
The second method trains with “chunked thinking” or “Delethink”; the harness permits the LLM to generate 8k tokens, then deletes all of the chain-of-thought but the last 4k tokens, and then continues the CoT from this chunk. An LLM so trained might naturally learn to summarize or compact the state relevant to the problem at the end of these 4k tokens, although it’s not directly incentivized. During training time, the authors only permit this generation-pruning cycle up to 5 times, but during inference time, they permit it up to 30 times or more.
When so trained, the second method (in dark purple) continues to improve with more test-time compute, long past the maximum amount of per-task train-time compute; while the first method (in darker blue) does not.
Image
Source: Aghajohari et al. (2025), The Markovian Thinker, Fig. 2b, CC BY 4.0.
So, it looks like the “compaction” or “chunked thinking” allows the LLM to think past 4x its maximum per-task training-time compute and to keep improving. Notably, there is no clear extrapolation ceiling in this paper; the score is still visibly improving as the thinking budget is extended, so the paper gives only a lower bound on the ceiling.
Notably, Delethink also uses fewer total FLOPs per training step than the standard method, and also has better performance than the standard method at equal steps of reinforcement learning.
Other papers also broadly support the idea that chunked thinking helps extend compute extrapolation:
- “Reasoning Cache” alternates between explicit instructions to summarize prior CoT and continuations that think from these summaries. Using this technique, it extrapolates from training on up to 16k total generated tokens to a half-million generated tokens, and finds performance continuing to increase with this scaling at test-time, despite never using so many tokens at training-time.
- “Scaling LLM Multi-turn RL with End-to-end Summarization-based Context Management” shows similar chunked context management and summarization can improve performance in an agentic context.
- There is some contrary evidence. For instance “E3: Learning to Explore Enables Extrapolation of Test-Time Compute” shows that it’s possible to produce compute extrapolation without chunked thinking or explicitly requiring “hard” summarization. But it’s worth noting that it requires other changes to the training procedure, and also only enables “2.5x extrapolation,” per the abstract. They also do experiments showing that several open-weight LLMs do not benefit from allowing spending more tokens at test-time than were used at training time.
Overall, I don’t think chunking / context management is absolutely required for allowing test-time compute extrapolation. But I think the more Markovian nature of the loop provides a good prior that incentivizes strong generalization.
Historical Evidence for Compute Extrapolation
We don’t know how OpenAI / Anthropic allocate training-time compute between tasks. We also don’t know how many thinking and compaction cycles they train on. However, we can note the calendar time at which models started to seem to keep improving across an almost unlimited number of thinking and compaction cycles, at least in some domains. Such a time is a good candidate for when they reached compute extrapolation in these domains.
You can make a rough and approximate case that, for Anthropic, this time was around the release of Opus 4.6, February 2026.
- The preliminary blog post on MirrorCode (a benchmark in which AIs reimplement existing software programs from scratch, over very long runs) indicates that Opus 4.1 and 4.5 were prone to hallucinate time constraints and time limits, even when none existed. But Opus 4.6 was able to persevere without such hallucinations through multiple rounds of compaction; Opus 4.6 was still improving its reimplementation of pkl at the end of a billion-token budget, including about 2 million generated tokens and tens of compactions. Training on such tasks might introduce severe off-policy gradient update issues. So this is weak evidence that compute extrapolation, given this kind of a harness and domain, had started by this time.
- Similarly, Toby Ord analyzed data from the UK AISI showing how AI performance scales with longer chains-of-thought relative to how human performance scales with more time. With large error bars, the data indicates that Opus 4.5 needs 370x more tokens in chain-of-thought to gain the same increase in performance that a human gains by thinking for 10x longer. But Opus 4.6 only needs 46x more tokens in chain-of-thought to do the same, which is a huge proportional gain, and the largest leap between the Opuses in the (very small) dataset.
And you can make a similar case for OpenAI’s GPT-5.1-Codex-Max, released in November 2025, which they advertised as the “first model natively trained to operate across multiple context windows through a process called compaction.” Evidence is ambiguous here, though. METR notably reports that compaction does not increase the time horizon for this model at this time. But it also represents a large leap in efficiency at scaling with further chain-of-thought on the UK AISI chart.
Future Compute Extrapolation from Multi-Agent Scaling
I think it’s plausible that multi-agent scaling will permit further extension of the ceiling and slope of compute extrapolation. I don’t have strong evidence for this. Some weak evidence I have is as follows:
- We don’t know how many cooperating agents OpenAI trained on in their multi-agent training. But we do know that they were able to cooperate with on the order of 1,000 agents (the HF swarm, cooperating by accident) or 10,000 agents (the Navier-Stokes swarm, cooperating intentionally, albeit Noam Brown attributes little of their efficacy to having such a huge swarm). But it seems likely that during training they were using far fewer agents as a maximum (10-100), which would give a compute extrapolation horizon of at least 10-1,000.
- Hierarchies and networks where each member maintains a bounded number of connections to other members are likely to be self-similar across a huge variety of scales and forms of information distribution. But a single linear CoT that has to handle an arbitrarily increasing quantity of context and action seems less likely to have this kind of self-similar and scale-free structure. I think this suggests that compute extrapolation will work better for multi-agent swarms than for single-agent CoT.
- At a minimum, there’s a strong limit to the practical compute extrapolation of single-agent systems imposed by time. A single agent that thinks for several months becomes increasingly unwieldy to use; for many problems you want an answer faster than that, if only to get an answer before your AI is superseded by smarter agents. Multi-agent scaling allows cognition to be parallelized over many agents, which is much easier to scale up over hardware than speed of cognition. So regardless of how far compute extrapolation could be (hypothetically) extended for a single agent in a world without time pressure, multi-agent scaling probably allows it to be extended much further in a real world with time pressure.
3. Conclusion
Compute extrapolation extends the reach of inference-time compute, by letting LLMs spend more compute on test-time problems (that people actually care about) relative to training-time problems (whose existence has no point, apart from being the catalyst for AI capabilities).
The currently intensifying increase in compute extrapolation, and its spread to new domains, will speed AI capabilities, particularly those at the top percentile of test-time compute expenditure. In general, compute extrapolation in some domains means that it could become prohibitively or impossibly expensive to know the top level of performance in some domain.
Further questions relevant to this area include:
- Would it make sense to ask AI companies to release information about the most compute-intensive training-time tasks in various domains, which could serve as a baseline for measuring compute extrapolation? Would such information about generalization from low to high compute be as important for them to release as information about cross-domain generalization?
- High ceilings and slopes for compute extrapolation are probably easier to obtain in domains like math or offensive cyber, because in these domains it is easier to recognize incremental progress as incremental progress. What other domains have this property (or other properties relevant to compute extrapolation) and what does it look like for an LLM to acquire the skill of recognizing incremental progress? In particular, what does it look like to acquire the skills relevant for recognizing incremental progress in algorithmic progress and AI R&D?
- How would we know if high levels of compute extrapolation started to extend to algorithmic taste and machine learning research? What would be the first sign?
- Would it make sense for efforts to understand the likelihood of RSI to require very high-compute evaluations, spending $100,000 or $1 million per task on algorithmic research?