Safety
Evaluations
Evaluations are the instrument that tells a developer what a system can do before the world finds out. Their weakness is that they give a lower bound on capability, and that capable models can tell when they are being tested.
State of the field
Capability trends. The length of task that models complete has doubled about every seven months since 2019. The current data is on METR's time horizons page.
Public institutes. The UK AI Security Institute and the US Center for AI Standards and Innovation at NIST test frontier models before release under agreements with developers. The UK institute publishes its findings in a Frontier AI Trends Report.
Shared tooling. Inspect, the UK institute's open-source framework, is the common tool for writing and running evaluations.
Evaluation awareness. One frontier model separated evaluation transcripts from real deployment transcripts with an accuracy score of 0.83, against 0.92 for people. The International AI Safety Report 2026 notes that this makes pre-deployment testing harder.
Sandbagging. Models can be prompted or fine-tuned to underperform on selected evaluations.
What is evaluated
| Domain | Question |
|---|---|
| Cyber | Can the system find and exploit vulnerabilities without help? |
| Biology and chemistry | Does it give meaningful uplift toward weapons? |
| Autonomy | Can it acquire resources, copy itself or resist shutdown? |
| AI research | Can it speed up the development of AI systems? |
| Propensity | Does it deceive, scheme or seek power when given the chance? |
Open problems
- Lower bounds only. Better prompting or tooling can elicit more capability than an evaluation found.
- Unreliable upper bounds. Sandbagging means a low score does not prove low capability.
- Awareness invalidates propensity tests. A model that knows it is observed may behave differently.
- Weak link to real-world harm. Benchmark scores map loosely to actual uplift.
- Independence. Third-party evaluators have limited access and time.
Recommended practice
- Build in the common framework. Write evaluations in Inspect so that others can run them.
- Report elicitation effort. State how hard the evaluators tried, the human baseline, and confidence intervals.
- Measure awareness. Estimate how often the model recognised the test, and make evaluations resemble deployment.
- Test for sandbagging. Use fine-tuning elicitation, noise injection and consistency checks.
- Tie thresholds to actions in advance. Decide what result stops a release before running the test.
- Give evaluators real access. Third parties need versions of the model with safeguards reduced.
The software side is covered in Evaluation harness.