Evals Are The Next Bottleneck
Evals are our primary way of measuring progress on model capabilities, but we're running through them very quickly. MMLU, a knowledge and reasoning benchmark, was released in 2020 and it took almost 4 years to saturate. On the other hand, SWE-Bench Pro, released less than a year ago, went from 23.3% to 80.3% in 8 months and was then deprecated, even before its poster presentation, after an audit found that a third of the tasks were broken.
The Great Saturation
We are making more evals than ever, but most aren't helpful because they're either poorly constructed, not truly representative of capability in the field, or not targeting a capability on the frontier lab research roadmap.
Once an eval is built, vetted and prioritized by research teams, progress is very quick. This implies that given an eval that is representative of valuable real-world work, frontier labs have built a good pipeline to identify capability gaps and then build and source RL tasks that improve on those capabilities. Most labs have moved to a fairly efficient paradigm where they train a large model, create specialist forks for different domains like code, math or language, and primarily run their hardest RL workloads on that model. These models are then distilled into smaller production models that are then served publicly.
The Imitation Game
We are also saturating evals very quickly because of how evals are getting built. A lot of evals that are currently being built are what I'll call human-imitation evals - these test production models against tasks currently done by humans that models sometimes struggle with. These evals can give some short-lived signal but are saturated very quickly by their very nature because a score of 100% means operating just like a human would. More importantly, it can lead to worse policies: a model that's trained step by step on human imitation tasks will learn to imitate humans on the task and arrive at a suboptimal policy for the task relative to a model that was trained end-to-end given the deliverables and constraints. This is exactly what the bitter lesson warns us about: encoding how humans solve problems helps in the short term and looks great but eventually plateaus and loses to methods that scale search and learning.
Evals should ideally be objective-based - they should be based on the end-to-end real-world task and grade based on progress in terms of absolute scores up to theoretical or practical upper bounds for the problem, rather than capped at human performance on the task. This is similar to how a compiler may be measured on wall-clock time and resource usage after a correctness gate on a set of binaries it compiles.
Objective-based evals are hard to build. They require the grader to encode key end-goal objectives quantitatively in addition to aspects like correctness. Any important aspect of the deliverable that is not captured in grading becomes a possible avenue for reward hacking, so tasks like this require a significant amount of iteration. An example of an objective-based task is Pyright Optimization, where the model is tasked with optimizing the runtime of Pyright, a static type checker for Python. In this task, the model is first evaluated on a set of hard gates (build, test suite, report format, reward hacking) before being evaluated on speedup over baseline on a set of benchmarks.
While human-imitation evals are easier to source and scale they tend to have more failure modes than objective-based evals. Evals like this are usually built with open-source projects where models are given PR or issue descriptions and expected to build an implementation that passes the tests created for the reference implementation. This is bad for a few reasons but most importantly because it considers the human solution as an optimal oracle solution which it often isn't. By training on tasks like this the model is forced to imitate human solutions, including the issues it may have, because it needs to pass the reference tests in order to get the reward.
Furthermore, tasks that are built against limits offer better gradients in RL settings. In PR-style evals, rewards are usually 0/1 based on whether the solution passes all the tests. You could get a more continuous reward by tying it to the number of passing tests but then you're assigning the same value to each test which is usually not the case in practice. In an objective-based task, you could have a human baseline or no-op reference and an upper bound, and assign rewards based on improvement over baseline scaled by what each additional point of performance represents in terms of value.
Humans Are Getting In The Way
As models approach capability levels close to human experts in some fields, we need evals to start measuring progress to optimal solutions instead of human solutions. It is starting to become very hard for most humans to tell the output of two models apart, even in their area of expertise. This can be viewed as the gap between models getting narrower or progress at the frontier slowing down but I think that is an artifact of perception instead. It's the people that are evaluating models by eyeballing outputs that are getting saturated.
Let's take LMArena as an example - it's a platform where users are given two anonymized model responses and need to pick the one that's better. On the Text Expert leaderboard, where prompts come from domain experts, the top 10 models all have overlapping rank spreads because that's how wide the confidence intervals are. At the time of writing, Anthropic's recent model release, claude-opus-5, is ranked #3 while claude-opus-4-6, a model released over 6 months ago, is ranked #2 but with rank spreads that span over 10 ranks. These experts can't really distinguish between 6+ months of frontier AI progress. This is what a saturated benchmark looks like - this mode of evaluation can no longer help evaluate models.
While it's going to be increasingly difficult for humans to compare two model outputs in the future, humans can build systems under which models can be evaluated well past human capabilities. For example, you don't need to be the most skilled Go player to formulate the scoring constraints and environment that can enable a model to reach superhuman levels of performance.
Moving Past Humans
Whether objective-based evals and environments can be built for a given domain will determine how quickly models can grow past human abilities in that domain. This is why we have and will continue to see rapid progress in verifiable and quantifiable domains like math and coding. Progress may be slower in more ambiguous domains that are harder to grade in reference to some limits, like creative writing or open-ended research. There is a lot more headroom under this paradigm once we start emulating complex real-world environments and start grading based on ground-truth rewards - for example, projects like virtual cell could allow us to leverage compute and models to solve problems in biology.
Building evals in this way allows us to capture progress on a capability past human capabilities - moving from imitation to optimal. Very few evals are currently designed like this. The core skill for humans is going to be defining the optimization functions for tasks across domains: choosing the core metrics that decide the result and valuing them correctly. It's going to be very important to scale our ability to do this to match the pace of compute and model development.
If you're excited about researching gaps in model capabilities with a high degree of rigor, reach out to us at work@boolean.ai.