Docs › Agent › How the agent checks its work

How the agent checks its work

The checks that run before an agent run may say it is done: your project's own checks, the sabotage check, an independent evaluator, and the UI check.

On this page

An agent marking its own todo list complete is not evidence that the work is right. So before an Agent or Debug run that changed files can say it is finished, AstraCode runs four checks, in this order. Each failure goes back to the agent with what failed, and the run ends saying a check is failing only when it could not fix it.

Progress shows in the chat as lines labelled Check and Evaluator, and the run card ends with Checks passed or Checks failed.

Diagram: project checks, then the sabotage check, then an independent evaluator, then a UI check for web changes, before the run reports it is done.
The order of the checks. Each one can be turned off with the setting named under it.

Your project's checks

AstraCode runs the checks your project already declares, cheapest first, stopping at the first failure. Nothing is guessed: a check runs only if your project has it.

  1. Infrastructure checks, for files the run changed and tools you have installed: terraform fmt -check and validate (or OpenTofu's), tflint, trivy, checkov or tfsec, helm lint, kubeconform and hadolint.
  2. Type checks: tsc --noEmit for the TypeScript projects that cover the changed files, when TypeScript is installed and you have no typecheck or types script; pyright or mypy when configured and installed.
  3. Your scripts: npm run typecheck, types, lint and format:check, when package.json has them. A script that fixes or writes (--fix, --write) or that deploys, publishes or pushes is skipped.
  4. Your tests: npm test (not npm's placeholder script), cargo test, go test ./..., pytest when pyproject.toml has a [tool.pytest section, or make test when the Makefile has a test: target.

To run one command of your own instead of the whole ladder, set astracode.agent.verifyCommand, such as npm run test:unit when the full suite is too slow. It runs in the workspace folder with a ten-minute limit.

The sabotage check

Passing tests do not prove the new tests test anything. When a run changed both code and tests, and the tests pass, AstraCode briefly undoes the run's code changes, runs the tests with the new tests still in place, and puts the code back.

The evaluator

Then a second conversation reviews the change with none of the run's history: your request in your words, the diff, and the checks' results. It can read, search and run read-only commands, and must end with a verdict. The chat says Asking an independent evaluator to check the change...

The UI check

When the run changed web UI (files ending .tsx, .jsx, .vue, .svelte, .astro, .html, .css, .scss, .less or .sass, or files under folders such as components/, pages/, views/ and app/), the evaluator also opens the app in a browser, on your machine only:

  1. It uses a dev server the run already started. Otherwise it starts your project's dev, start, serve or preview script in the background and waits up to 25 seconds for its address. A plain index.html is opened directly.
  2. It opens the page, goes through the affected flow and reads the console. A console error or a broken flow counts as a blocking problem.
  3. If no server can be opened, the chat says so, the evaluator judges from the code, and the summary repeats that the UI was not tried.

This works with browser control off. Flutter projects are skipped, and phone screens are not covered.

Before the checks

A run that tries to stop with todo items still open is reminded of them, up to twice. A run that changed files is asked once to re-read your request against what it did, and a run that wrote code in its reply without changing any file is asked to make the change itself.

Settings

SettingDefaultWhat it does
astracode.agent.verifyonRun your project's checks before a run says it is done.
astracode.agent.verifyCommandemptyOne command to run instead of the detected checks.
astracode.agent.sabotageCheckonCheck that new tests fail without the change.
astracode.agent.evaluatoronHave an independent evaluator review each change.
astracode.agent.uiGateonHave the evaluator open the app when a run changes web UI.

Each evaluation is a short extra run and counts against your allowance like any other. Turning a check off makes runs quicker and cheaper and leaves you to check the work yourself.

If something goes wrong

← Browser tools Checkpoints and undo →

All documentation

Get started

Using the editor

Agent

Context

Workspaces

Connectors

Extending AstraCode

CLI

Account and usage

Troubleshooting

Reference