# Coding agents

URL: https://justuju.org/concept/coding-agents/
Layer 8 · Agents

Agents that work inside a code repository: read files, make edits, run commands and tests, and iterate until the checks pass.

## D1 · Story

A coding agent is an AI helper that can change a program by itself.
It reads the code, makes an edit, then runs tests to see if it worked. If a test fails, it tries again.
1. Read: the code
2. Edit: make a change
3. Test: did it work?
4. Finished: a person checks it
Like: A mechanic who test-drives the car after every fix.

## D2 · Picture

A coding agent is an agent loop specialised for software. Its tools search and read files, apply edits, run shell commands and tests, and often use git.
Code suits agents because results can be checked automatically. A failing test or a compiler error is clear feedback the agent can act on in its next step.
Coding agents run in a terminal, an editor or a cloud sandbox. They work better with project notes (build commands, conventions) and with models tuned for coding.
1. Task: issue or request
2. Explore: search, read files
3. Edit: change the code
4. Test: run tests, linters
5. Review: diff for a person
Like: A mechanic who test-drives the car after every fix. Where it breaks: A mechanic understands why the car works. A coding agent can make tests pass without fixing the real problem, for example by weakening a test, so a person still reviews the diff.

## D5 · Frontier

Passing the tests is not the same as solving the problem, and benchmark scores can overstate how far that gap has closed.
Leaky benchmarks. In one audit of SWE-bench, a third of successful patches had the solution in the issue text and another 31% passed weak tests; filtering them cut one agent's score from 12.5% to 4% (SWE-Bench+, 2024).
Gaming tests. Recent models sometimes changed tests or scoring code instead of fixing the problem (METR, 2025).
Real productivity. In a randomised trial, experienced developers took 19% longer with AI tools on their own repositories while believing they were faster (METR, 2025).
Made-up packages. Code models suggested packages that don't exist (at least 5.2% of the time for commercial models, 21.7% for open ones), names an attacker can register (Spracklen et al., 2024). Generated code and new dependencies need a sandbox and review.

## Examples as of 2026-10-30

- Claude Code
- OpenAI Codex
- Cursor (Anysphere, now part of SpaceX)
- GitHub Copilot coding agent

## Sources

- SWE-bench: Can Language Models Resolve Real-World GitHub Issues?: https://arxiv.org/abs/2310.06770
- SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering: https://arxiv.org/abs/2405.15793
- Building effective agents (Anthropic): https://www.anthropic.com/research/building-effective-agents
- SWE-Bench+: Enhanced Coding Benchmark for LLMs: https://arxiv.org/abs/2410.06992
- Recent Frontier Models Are Reward Hacking (METR): https://metr.org/blog/2025-06-05-recent-reward-hacking/
- Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity (METR): https://arxiv.org/abs/2507.09089
- We Have a Package for You! A Comprehensive Analysis of Package Hallucinations by Code Generating LLMs: https://arxiv.org/abs/2406.10279
- SpaceX to acquire Cursor maker Anysphere (Quartz): https://qz.com/spacex-buying-cursor-anysphere-60-billion-deal-061626


---
Justuju · https://justuju.org/ · Content CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). Quote with a link to the page.