npm test | tail exits 0 when your tests fail
14 November 2026 | 4 min
One of the most common ways a CI job lies, and it fits in a single character.
In 2017 Equifax had a device whose job was to inspect encrypted traffic for signs of an attack. Its certificate had expired, so it quietly stopped inspecting, and the traffic passed through. According to the House Oversight Committee report, nobody had visibility into that traffic for nineteen months. When the certificate was renewed, staff noticed suspicious traffic straight away. About 147 million people were affected, according to the FTC.
A check that can stop checking without telling anyone is worse than no check, because people keep relying on it. CI has a small, common version of exactly this.
The line
npm test | tail -n 50
It looks harmless: run the tests, keep the log short. But in bash, as the manual puts it, the exit status of a pipeline is the exit status of the last command in it. The last command is tail, and tail succeeded. Your tests failed, the step exits 0, and the job is green. The same goes for | tee test.log, | grep -v noise and anything else you pipe a test run into.
The fix
set -o pipefail
npm test | tail -n 50
With pipefail, a pipeline returns the status of the last command that failed, so a failing npm test fails the line even though tail succeeded. Put it at the top of every CI script, next to set -e. If you need the test runner's own code specifically, bash keeps every stage's status in ${PIPESTATUS[@]}, and ${PIPESTATUS[0]} is the first command's.
One false alarm to know about: under pipefail, cmd | head can fail even when cmd was fine. head closes the pipe after the lines it wanted, cmd is killed by SIGPIPE, and the pipeline returns 141. Use tail for test logs, or write the log to a file and print the end of it.
GitHub Actions: it depends on one line
This is the part that catches people who know about pipefail. GitHub's workflow syntax docs list the exact command each run step uses on Linux and macOS:
| What the step says | What runs | pipefail? |
|---|---|---|
| no `shell:` at all | `bash -e {0}` | No |
| `shell: bash` | `bash --noprofile --norc -eo pipefail {0}` | Yes |
| `shell: sh` | `sh -e {0}` | No |
So a step with no shell: runs bash without pipefail, and the same step with shell: bash written out runs it with pipefail. The docs say so in as many words: the unspecified default "runs a different command to when bash is specified explicitly". Steps inside a container job default to sh. On Windows the default is PowerShell, which handles errors differently again.
The fix for a whole workflow is three lines at the top:
defaults:
run:
shell: bash
Prove it in your own pipeline
Do not take this post's word for it, or the docs'. Add a step that should fail and see whether it does:
- name: This must turn the job red
run: false | cat
If the job stays green, every piped test step in that workflow can pass with failing tests. Remove the step once you have fixed the shell. This is the same rule as for any check: break the thing it guards and confirm it goes red.
Other places it hides
- Makefiles. Each recipe line runs in
/bin/shby default. SetSHELL := bashand.SHELLFLAGS := -eo pipefail -c. - Dockerfiles.
RUN a | bruns under/bin/sh -c. UseSHELL ["/bin/bash", "-o", "pipefail", "-c"]before it. - package.json scripts. npm runs them with
shon Linux and macOS, so a pipe inside a script has the same problem. - Coverage and lint steps. Anything piped into a formatter or a reporter. These fail quietly for longest, because nobody expects them to fail.
Where AstraCode fits
When AstraCode's agent runs a command for you on macOS or Linux, it runs it under bash with pipefail, so npm test | tail fails when the tests fail and the agent sees a failure rather than a clean tail of a log. On Windows, where cmd.exe has no pipefail, a piped command that appears to succeed carries a note saying the pipe may have hidden a failure. It is a small thing, and it is the kind of small thing that decides whether "the tests pass" means anything.
An agent that does not trust a green pipe
It runs your commands with pipefail and shows you what came back. Free to start on macOS, Windows and Linux.
Also on the blog
- Make a coding agent fail loudly in CI
- Does your test test anything? The ten-second revert check
- Coding agents are part of your supply chain now: a checklist
- Developers trust AI they can check. Build agents for that.
- When an AI agent says it is done and it is not
- How to review code an agent wrote
- Making an AI agent follow your project's conventions
- We benchmarked eleven models in our own editor. The cheapest one won.
- Fable 5.1 cut cache reads to $0.25. Here is what that saves on a real agent run.
- What vibe coding is, and when it stops working
- What Google Antigravity is, and what it costs
- What GPT-6 Astra actually costs to run a coding agent
- GPT-6 Astra's pricing cliff at 272K tokens, and why agent runs fall off it
- How we're benchmarking GPT-6 Astra for coding (and why scores won't tell you)
- How to use GPT-6 Astra in your editor, and when not to