Flaky tests cost more than they look.
Every re-run is ten minutes of someone's focus, and every ignored failure teaches the team that red builds are normal.
Patchwork watches your CI, reruns a failing test in a sandbox until it repeats, finds the root cause and opens a small pull request with the fix. You review it like any other change.
Failed in 3 of the last 50 runs on main. Passes on re-run.
All workflows · main
Root cause. flush() reads the order balance while a ledger write for the same order is still open, so the refund lands on a stale balance.
test_refund_partial failed in 3 of the last 50 CI runs. In a sandbox it failed 6 of 50, every time flush() overlapped an open ledger write for the same order.
The balance is now read inside the per-order lock, and _drain() waits for that lock instead of skipping. No change for single refunds. 500 sandbox runs after the change: 0 failures (+0.4s).
ledger/write.py+10 -3Reading inside the lock fixes it. Does wait_for() need a timeout, or can it just block?
The 2 s timeout stops one stuck write from holding the whole queue. It raises LedgerTimeout, which your retry job already handles.
refunds/queue.py+4 -3
01:58 Re-run number four
Every re-run is ten minutes of someone's focus, and every ignored failure teaches the team that red builds are normal.
Install the GitHub app, pick repositories, and Patchwork starts with the next failing run. Setup takes about two minutes.
It reads the files involved in a failing test and their direct dependencies, inside a sandbox in your region. Nothing is kept after the job ends.
No. It only opens pull requests, which run your full checks and need your approval like any other change.
When the cause is shared infrastructure, such as a slow test database, it labels the failure, shows the evidence and tags the owner instead of writing a fix.
Python, TypeScript, Go, Java and Ruby today, with Rust and C# in beta.
GitHub Actions on the free plan. GitLab CI, CircleCI and Buildkite on Team.
02:09. A test went red. Reproduce it from CI.
Omar merged #918. test_refund_partial passed 500 of 500 runs.