How to Block Merges on AI Code Review Findings in GitHub

To block merges on AI code review findings in GitHub, the reviewer has to report a check run that ends in a non-passing conclusion such as failure when it finds a blocking problem. You then add that check as a required status check in a ruleset or branch protection rule, with the reviewer's GitHub App selected as the source. Two things commonly break this setup. GitHub counts neutral as passing, and some AI reviewers, Qodex among them, report neutral by default. And if a required check never reports, the pull request sits at "Waiting for status to be reported."

Three GitHub mechanisms an AI reviewer can drive

GitHub has three separate merge requirements an AI reviewer can feed. All three show up in the same merge box, which makes them easy to confuse, but they behave differently when the review goes wrong.

Status checks are the only one of the three that records the review's outcome as pass or fail. They are also the only one that fails closed when the review is missing. GitHub's docs say a required check must be success, skipped or neutral on the latest commit, and results from earlier commits don't count.

Approvals count people. GitHub's rule counts approving reviews from people with write permissions or from a designated code owner. Keep approvals for your engineers, and turn on the options that dismiss stale approvals when new commits change the diff and that require approval from someone other than the last person to push.

Conversation resolution is the mechanism Bito uses. Its "Request changes comments" toggle posts actionable feedback as formal Request changes review comments. Its docs tell GitHub users to enable Require a pull request before merging and Require conversation resolution before merging, and developers clear Bito's comments by commenting /resolve. This gate only blocks after the reviewer has posted something, so a review that times out or never starts doesn't hold the merge. An open Bitbucket feature request for a Rovo Dev merge check describes the same gap on that platform. The current workaround, "No changes are requested", blocks only if Rovo Dev actively flags an issue, and it doesn't confirm that a review was triggered or finished.

Why some AI reviewers always report neutral

neutral lets a tool report without blocking. A reviewer that ships with gating off will usually post neutral whatever it finds, so requiring that check only makes merges wait for the review to finish.

Qodex documents this directly. With pr_review.block_pr_merge: false, which is the default, "the conclusion is always neutral." Turning gating on takes a config change:

pr_review:
  block_pr_merge: true
  block_on_severity: critical

With this set, Qodex's docs say it reports failure when a verified finding meets block_on_severity or when the review can't finish (budget, step limit, wrap-up error), and success when no verified finding meets the threshold. A review that is skipped or fails with an LLM or GitHub error still concludes neutral. Unverified findings never cause a failure. You still have to add the Qodex review check to your required status checks, or GitHub won't block the merge button.

Warpway works the same way at its lower trust levels. At Observe and Assist every finished review concludes neutral, and it can only block at Gate and above.

Before you require any AI check, open a pull request that contains a known problem and look at the check's conclusion, not its summary text. If it says neutral while listing findings, the tool is in advisory mode, and no ruleset will change that.

If your AI review runs as a GitHub Actions job

Teams that call a model from their own workflow run into a different set of cases that pass or stall by accident. GitHub's troubleshooting docs list these:

• A job skipped by an if: conditional reports success. A guarded review step that doesn't run still passes.
• A dependent job that is skipped because an earlier job failed may not block the merge. GitHub suggests always() together with needs for required checks.
• A workflow skipped by path, branch or commit-message filters leaves its check pending and blocks the merge.
• A workflow_dispatch run on the head branch doesn't satisfy a required status check in a branch ruleset. This restriction doesn't apply to checks from external GitHub Apps.
• With a merge queue, required workflows need the merge_group trigger.

Make the job exit non-zero when it finds a blocking problem, and also when the model call or output parsing fails. Otherwise an outage looks like a clean review.

Require the check with a ruleset or branch protection

GitHub only lists checks that have reported in the repository recently, so let the reviewer run on one pull request first.

With a ruleset

1. In repository or organization settings, open Rules → Rulesets and create a branch ruleset.
2. Target your default branch and your release branches.
3. Enable Require status checks to pass and add the check by its exact name.
4. Select the reviewer's GitHub App as the expected source.
5. Decide whether branches must be up to date before merging (see the FAQ).
6. Review the bypass list. GitHub lets a ruleset exempt users with certain roles, specific teams or GitHub Apps, and anyone on that list can merge without the check.
7. Set enforcement to Active.

With classic branch protection

Open Settings → Branches, edit the rule for your branch, enable Require status checks to pass before merging, search for the check, select it and save.

Selecting a source matters more for AI checks than for most CI. Once an app is selected, a status with the same name set by another person or integration doesn't count, and GitHub reports Required status check "build" was not set by the expected GitHub App. GitHub also notes that if a check and a commit status share a required name, both have to pass.

Fixing "Waiting for status to be reported"

This message means GitHub expects a result for a required check name on the head commit and doesn't have one. It is the fail-closed behavior working as intended, and it is also a common way a new required check stalls a team.

The forum threads show how long the last row can last. In a 2018 CircleCI thread, jobs passed in CircleCI while GitHub stayed at "Waiting for status to be reported", and re-running didn't help. A commenter later pointed to CircleCI's status page, which reported that GitHub check deliveries were delayed on October 30–31. In a 2023 Sonar Community thread, the analysis passed after an upgrade from SonarQube 9.9 to 10.0, but GitHub stayed on "Expected — Waiting for status to be reported." The user fixed it by configuring the DevOps platform integration at the project level and removing two pull request decoration properties, sonar.pullrequest.provider and sonar.pullrequest.github.repository. If the tool says it finished and GitHub says it's waiting, look at delivery and integration settings, not your rule.

When you have to ship during a stall, a short bypass list is a smaller change than turning the rule off for everyone.

Making Warpway / Review a required check

Warpway reports one check, Warpway / Review, on the pull request's exact head commit. Warpway reports the result, and GitHub's rules enforce it. Gating needs the Team plan and a repository at trust level 2 (Gate) or higher. On the Free plan the trust level is capped at Assist, so every finished review concludes neutral. When the private-repository limit is reached, the check reports Usage limit reached (action_required) instead.

1. In the dashboard, set the repository's trust level to Gate. Admins can do this; higher levels need an Owner. If the base branch's .warpway.yml sets a lower trust_level, the lower level wins. See trust levels.
2. Open or update a pull request so the check reports once.
3. In a ruleset, require Warpway / Review spelled exactly that way, select the Warpway app as its source, and set enforcement to Active. Classic branch protection works too.
4. Optionally, tune what blocks in the base branch's .warpway.yml:

version: 1
merge_gate:
  require_all_required_lenses_resolved: true
  require_zero_open_human_tasks: true
  block_on_severities:
    - critical
    - high
  required_ci_checks:
    - unit-tests
  fail_on_incomplete: true

By default, critical, high and medium findings block. Warpway reads the configuration from the base branch, so a pull request can't loosen the policy it is reviewed under. The full reference is in configuration as code.

What each conclusion means

Warpway never reports success while blockers exist. A lens that couldn't finish is reported as Incomplete and is never treated as Cleared. The merge gating page lists every blocker type.

Warpway's merge gating and troubleshooting docs also call out these cases:

• Unreviewed pull requests get no check. That covers authors in review.ignore_authors, base branches outside review.base_branches, and changes that touch only review.ignore_paths. If the check is required on that branch, those pull requests wait.
• Merge queues. Warpway doesn't report on merge-queue groups, so don't require Warpway / Review in a ruleset that gates a merge queue.
• Broken configuration. The pull request that fixes an invalid .warpway.yml is reviewed under the broken file. Either have someone with bypass permission merge it, or lower the repository below Gate until it merges. See troubleshooting.
• Uninstalling. Remove the requirement first, or pull requests will wait for a check that never arrives.
• A passing check is not an approval. Warpway never merges, and required human approvals still apply.

A setup that holds up

1. Require the AI check, select its source app, and confirm on a test pull request that it can actually fail.
2. Require human approvals, dismiss stale approvals, and require approval of the most recent push.
3. Use conversation resolution if your team treats threads as a to-do list, but don't make it your only AI gate.
4. Keep the bypass list short.
5. Before enforcing the rule, make sure the reviewer's scope matches the ruleset's target branches, so nothing waits on a check that will never come.