AI oversight with Apertus

What’s the idea?

As AI agents take on more work, how do we check what they actually did? Imagine a lab says its deployed model passed the required evaluations. Can an outside reviewer follow the records and verify that claim? What if some logs are missing, or the deployed model is a different checkpoint?

We want to build tools that help people investigate agent behaviour, find missing evidence and check whether AI developers are keeping their commitments. The longer-term goal is meaningful independent oversight, including when sensitive information needs to stay inside a lab.

This is already a practical problem. METR’s investigation of an agent incident describes how hard it was to check large volumes of AI-generated analysis. Anthropic’s September 2026 commitment to embedded evaluators makes the question especially timely: even with access, how does a small team keep up?

What records do we mean?

An agent trace is a record of an AI agent carrying out a task: the instructions and context it received, its responses, the tools it called and the results those tools returned. For example, a research agent might read an evaluation plan, run a training script, inspect the results and write a report.

To check what happened, we would connect these traces with supporting records:

• Model interactions: prompts, supplied documents and model responses, with model versions and timestamps where available.
• Tool activity: commands executed, files read or changed, searches and their results, including errors.
• Research and development records: code versions, job logs, experiment settings, model checkpoint IDs and evaluation outputs.
• Commitments and decisions: evaluation plans, approval and release records, and relevant messages or documents where access is agreed.

An auditor could then follow a claim such as “this model passed the required evaluation” back to the exact checkpoint, evaluation run and result. Part of the research is checking whether the available records support that conclusion, and identifying what is missing. A saved conversation alone does not show that every relevant action was recorded.

Connecting to Pacing the Frontier

Pacing the Frontier calls for an international effort to develop technical and governance tools for deliberately pacing automated AI development. Our project aims to contribute to that goal through practical, independent AI oversight: tools that help reviewers verify safety claims, investigate incidents and assess whether developers are following their commitments. Reliable evidence is essential if coordination across labs and countries is to be meaningful.

Why Apertus and Switzerland?

Apertus is the open language-model effort from ETH Zurich, EPFL and CSCS. It gives us a concrete setting for studying model evaluation and development workflows.

For the engineering work, I’m especially keen to work with students and collaborators based at ETH Zurich or EPFL who can connect this research with the Swiss AI community. We would start with public Apertus artifacts and experiments we can run ourselves, then explore work with Apertus contributors where suitable access can be arranged. Broader research collaborators can be based elsewhere.

What you could work on

We’ll pick one focused question together. For example:

• Find missing evidence: connect agent transcripts with job records to identify missing runs or incomplete logs.
• Investigate agent behaviour: search across traces to check a claim, such as whether an agent continued after a stop instruction, and show the evidence behind the answer.
• Check an evaluation workflow: connect checkpoint IDs, evaluation results and release records to verify that the right model was evaluated before a handoff.

You would build a working system and test it against sensible baselines. We care about whether it catches real problems, avoids false accusations and saves reviewers time. There is already research code and a literature collection to build on.

How you could contribute

Engineering: build tools, run LLM experiments and analyse agent behaviour. Depending on the project, this could involve working with model checkpoints in Python, PyTorch or Hugging Face, running GPU jobs with Slurm, or building evaluation pipelines and searching agent traces. We can shape the work around your experience and what you’d like to learn.

Research collaboration: bring experience in AI safety, security or governance to help shape what we audit, how we judge the evidence and how these tools could support independent oversight in practice. You don’t need hands-on LLM or GPU experience for this. We could develop research questions, design studies and work towards papers together.

Funding this international collaboration

We are actively seeking funding to support this international research collaboration on AI oversight and safety. We welcome conversations with funders who want to help develop and test tools for independent evaluation and verification, connecting research at MIT with collaborators in Switzerland and beyond.

Interested? Get in touch

Email joshyang@mit.edu and CC Prof. Michiel Bakker at bakker@mit.edu. Tell us a little about your background, how you’d like to contribute and when you could start. A CV or a link to relevant code, research or writing would help.

If you’re looking for a master’s thesis, let’s discuss the project first. We can then look for an ETH Zurich or EPFL professor willing to supervise locally and help set up the collaboration, with MIT co-supervision.

You don’t need to be looking for a thesis. If you want to contribute to the project and work towards papers together, I’d be happy to hear from you too.

Updated 22 September 2026.

← Back to Josh’s homepage