Case studies
Would Warpway have caught it?
We take merged pull requests from popular open-source projects, replay them through Warpway as they stood before merge, and compare what it says with the bug reports, fixes and reverts that followed. Warpway was not installed on these projects at the time: every review here is a replay, run on 2026-10-07. Every page links to the public history, and says what Warpway missed.
- 01
Replay the pull request as it stood before merge
Its final code, title, description and commits, and the repository at the fork point. No review comments, no bug reports, nothing after merge.
- 02
Review with Warpway’s default lenses
The production role models, lenses and default policy, run through Codex subscription access. One run per pull request, with no repository configuration.
- 03
Compare with what actually happened
Later fixes, reverts and bug reports in the public history, linked from every page so you can check our reading.
- psf/requests #6667Python
requests 2.32 SSLContext regression: the wrong mTLS client certificate
Requests 2.32.0 shared one SSLContext across verified connections. In a replay of the PR before merge, Warpway flags the client-cert mix-up later reported as #6726.
- Warpway flags in a replay
- Client certificates leaking across requests through one shared SSLContext
- What happened
- Reported by users from 7 days after merge; reverted a year later in 2.32.5
- honojs/hono #5366TypeScript
Hono formData() after text() silently corrupts binary file uploads
- Warpway flags in a replay
- Binary file uploads silently altered when text() is read first
- What happened
- Not previously reported; we filed it as honojs/hono#5539
- honojs/hono #5133TypeScript
Hono cloneRawRequest: stale Content-Length after multipart re-encoding
- Warpway flags in a replay
- Forwarded multipart clones keep a Content-Length that no longer matches the body
- What happened
- Found independently 36 days after merge; fixed in 4.13.4
- honojs/hono #4707TypeScript
Hono 4.12.0: c.json(undefined) throws "Value is not JSON serializable"
- Warpway flags in a replay
- c.json(undefined) throws on the new fast path instead of returning a response
- What happened
- Reported 4 days after merge; fast path reverted in 4.12.2
- psf/requests #7431Python
requests 2.34.1 mypy error: headers dict[str, str] has incompatible type
- Warpway flags in a replay
- Header input type rejects ordinary dict[str, str] under type checkers
- What happened
- Partially reverted the same day; fixed in 2.34.2
The pilot
Every result, including the misses
On 2026-10-07 we replayed 12 merged pull requests from hono, axios and requests. Six were later fixed or reverted because they broke something. Six were controls that nothing later fixed.
We picked the bug cases knowing how they turned out, and each pull request was reviewed once, so this is a pilot, not a benchmark. Warpway saw each pull request's final code before merge, without review comments, and older pull requests may appear in the model's training data.
The noise is real too: low-value notes kept three clean pull requests from an all-clear. We show it because it matters as much as the catches.
| Pull request | What happened after merge | Warpway in the replay | Result |
|---|---|---|---|
| psf/requests#6667Later fixed or reverted | Client-certificate and CA-loading regressions; reverted | Named both regressions (high) | Caught |
| honojs/hono#5133Later fixed or reverted | Stale Content-Length, fixed in #5282 | Named it (medium) | Caught |
| honojs/hono#4707Later fixed or reverted | c.json(undefined) broke; reverted | Named it (medium) | Caught |
| psf/requests#7431Later fixed or reverted | Typing regression; partially reverted | Named it (medium) | Caught |
| axios/axios#11087Later fixed or reverted | Null handlers crashed every request | Flagged the same mechanism, not the crash | Partial |
| axios/axios#10866Later fixed or reverted | Reverted as a v1.x API change | Blocked it for other reasons | Partial |
| honojs/hono#5366Control | No later fix or revert | Found an unreported upload-corruption bug (high) | New bug found |
| honojs/hono#5250Control | No later fix or revert | All clear | Correct all-clear |
| psf/requests#7328Control | No later fix or revert | All clear | Correct all-clear |
| psf/requests#7427Control | No later fix or revert | Release-notes suggestion (low) | Noise |
| axios/axios#11040Control | No later fix or revert | Test may pass without the fix (medium) | Noise |
| axios/axios#11059Control | No later fix or revert | Contrived edge case (medium) | Noise |
See what it finds in your pull requests
Install the GitHub App on a repository. New repositories start in Assist, so Warpway reports what it finds but never blocks a merge.