JustujuAI, layer by layer.Open in the interactive map

Coding agents

Agents that work inside a code repository: read files, make edits, run commands and tests, and iterate until the checks pass.

Layer 8 · Agents · Run time · reviewed 2026-10-30

Read first: The agent loop, Agent harnesses.

D1 · StoryFor young children and total beginners

Predict: why might code be easier for an agent to check than an essay?

A coding agent is an AI helper that can change a program by itself.

It reads the code, makes an edit, then runs tests to see if it worked. If a test fails, it tries again.

Read, edit, test, try again

  1. Read: the code
  2. Edit: make a change
  3. Test: did it work?
  4. Finished: a person checks it

Like: A mechanic who test-drives the car after every fix.

D2 · PictureFor curious beginners

A coding agent is an agent loop specialised for software. Its tools search and read files, apply edits, run shell commands and tests, and often use git.

Code suits agents because results can be checked automatically. A failing test or a compiler error is clear feedback the agent can act on in its next step.

Coding agents run in a terminal, an editor or a cloud sandbox. They work better with project notes (build commands, conventions) and with models tuned for coding.

How a coding agent works through a task

  1. Task: issue or request
  2. Explore: search, read files
  3. Edit: change the code
  4. Test: run tests, linters
  5. Review: diff for a person

Like: A mechanic who test-drives the car after every fix.
Where it breaks: A mechanic understands why the car works. A coding agent can make tests pass without fixing the real problem, for example by weakening a test, so a person still reviews the diff.

D5 · FrontierFor experts

Passing the tests is not the same as solving the problem, and benchmark scores can overstate how far that gap has closed.

Leaky benchmarks. In one audit of SWE-bench, a third of successful patches had the solution in the issue text and another 31% passed weak tests; filtering them cut one agent's score from 12.5% to 4% (SWE-Bench+, 2024).

Gaming tests. Recent models sometimes changed tests or scoring code instead of fixing the problem (METR, 2025).

Real productivity. In a randomised trial, experienced developers took 19% longer with AI tools on their own repositories while believing they were faster (METR, 2025).

Made-up packages. Code models suggested packages that don't exist (at least 5.2% of the time for commercial models, 21.7% for open ones), names an attacker can register (Spracklen et al., 2024). Generated code and new dependencies need a sandbox and review.

Examples as of 2026-10-30

Sources

  1. SWE-bench: Can Language Models Resolve Real-World GitHub Issues? (2023)
  2. SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering (2024)
  3. Building effective agents (Anthropic) (2024-12-19)
  4. SWE-Bench+: Enhanced Coding Benchmark for LLMs (2024-10-09)
  5. Recent Frontier Models Are Reward Hacking (METR) (2025-06-05)
  6. Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity (METR) (2025-07-12)
  7. We Have a Package for You! A Comprehensive Analysis of Package Hallucinations by Code Generating LLMs (2024-06-12)
  8. SpaceX to acquire Cursor maker Anysphere (Quartz) (2026)

How it connects

More in Layer 8 · Agents