I rage-built our email automation. Now we're open sourcing the engine.
Why we are sharing a real email system built with Charlie, and how repo-defined daemons helped keep the work bounded, reviewable, and testable.
A prototype comes together quickly, but things get rough quickly when building production systems.
The first version of an AI pull request reviewer is genuinely easy to build, and genuinely useful. A bot that can receive a PR event, fetch the diff, send it to a model, turn the useful output into inline comments, and publish a review can catch a lot of useful issues. That loop can catch an inverted condition, a missing authorization check, or an error path that silently drops data, especially if you’re talking about a PR that one person is working on.
When teams and fast-moving codebases are involved, things get hairy fast. If the PR changes while analysis is running, a webhook arrives twice, or a file is too large to fit, how are these edge cases handled? How about if the model repeats a concern somebody already resolved, GitHub rejects an inline anchor because the line moved, or a timeout happens after publication? The retry may create a second review, or nothing appears, and nobody knows whether the code was clean, the review was partial, or the system failed.
At that point, the hard question is no longer “can a model review a diff?” It is “what does a review mean when the evidence, code, and workflow keep moving?”
| Capability | GitHub Copilot | Claude Code Review | Graphite Agent | CodeRabbit | Charlie |
|---|---|---|---|---|---|
| Automatic review | ✓ | ✓ | ✓ | ✓ | ✓ |
| Uses context beyond the changed lines | ✓ | ✓ | ✓ | ✓ | ✓ |
| Uses repository-specific review instructions or rules | ✓ | ✓ | ✓ | ✓ | ✓ |
| Re-reviews after new pushes | ✓ | ✓ | ✓ | ✓ | ✓ |
Can be configured to submit a formal APPROVE or REQUEST_CHANGES review | ✕ | ✕ | N/A | ✓ | ✓ |
| Defines a versioned repo-local role with wake conditions, schedules, routines, and deny rules | N/A | N/A | N/A | N/A | ✓ |
This repository’s current pr-review daemon policy is COMMENT-only; Charlie’s publisher supports APPROVE and REQUEST_CHANGES when explicit repo policy authorizes them and GitHub’s identity, permission, and safety constraints are satisfied.
The first few rows are table stakes: these products are not simple diff bots. The differences below them expose harder choices about state, authority, and who owns the review operating contract. The sections that follow unpack the operating problems behind those differences.
Who gets to decide?
GitHub Copilot code review always submits a COMMENT review, not APPROVE or REQUEST_CHANGES. Its review therefore does not satisfy required approvals or block a merge. Claude Code Review makes a related boundary explicit: its check always concludes neutral, and teams that want to gate merges can interpret its severity output in their own CI.
These boundaries expose the problem of authority. Finding an issue and deciding whether code may merge are different jobs. The second needs an operating contract: which findings are advisory, which can affect branch protection, who owns false positives, and what happens when the reviewer is unavailable. “The model sounded confident” is not a safe authorization policy.
What was actually reviewed?
Production systems also need an honest definition of completeness. Graphite documents that its agent may not run when a PR exceeds 200,000 changed characters. Its customization documentation says large rule files are truncated for performance and too many files can reduce review quality.
That boundary exposes a universal constraint: review evidence is finite. Diffs, repository guidance, linked issues, prior discussion, generated files, and surrounding code all compete for time and context. A system has to select, retrieve, prioritize, and sometimes omit evidence. The dangerous outcome is not partial review by itself; it is partial review presented as complete. Teams need to know what was analyzed, what was skipped, what was truncated, and whether “no comments” means clean code or insufficient coverage.
What does the reviewer remember?
A PR is a stream of changing states, not a document. New commits arrive, old comments are resolved, lines move, humans downvote feedback, multiple runs may overlap. A lot can happen.
GitHub warns that a Copilot rereview may repeat comments even after they were resolved or downvoted. CodeRabbit’s automatic review controls take another visible approach: incremental review is enabled by default, but automatic re-reviews pause after five reviewed commits by default. Teams can change that threshold, disable the pause, or manually request and resume reviews.
These product boundaries expose the state and memory problem. A reviewer must decide which head SHA a run belongs to, whether a push invalidates in-flight work, whether an old finding still applies, and whether a new comment is meaningfully different from an existing one. De-duplication is not just string matching. The same bug may be described differently, while similar wording may refer to a newly introduced regression. Publishing safely requires fresh anchors, prior-feedback awareness, run identity, and rules for cancellation and re-review.
What happens when the outcome is ambiguous?
Claude Code Review documents that failed or timed-out reviews do not retry automatically; users can start a fresh review with a comment, or with a new push when push-triggered review is enabled. That boundary exposes the problem of recovery and publication safety.
Automatic retry sounds obviously good until the system times out after GitHub accepted the review but before the client recorded success. Now “retry” may mean “publish it twice.” Avoiding this situation requires idempotency keys, durable run state, reconciliation against the live PR, and terminal outcomes that distinguish clean, skipped, partial, failed, stale, and published. Immediately before posting, the system should verify the current head, anchors, existing feedback, permissions, and whether another attempt already succeeded.
The real build decision
None of this makes today’s products simplistic; their documented behavior and controls will keep evolving. The point is that every usable product draws boundaries around authority, completeness, memory, and recovery. Those boundaries are part of the product, not implementation trivia behind the model.
Building your own PR review agent can still be right, especially for a narrow internal check or a workflow central to your advantage. Just be clear about the ownership boundary. You are not only owning prompts and model choice. You are owning triggers, evidence provenance, repository policy, stale-work cancellation, coordination, de-duplication, permissions, safe publication, retries, telemetry, and evaluation as models and team preferences change.
Charlie’s bet is not that one smarter model makes these problems disappear. It is that the repository should own an explicit operating contract for what review means: when it runs, what evidence it uses, what authority it has, how work coordinates, when publication is safe, how failures recover, and how quality is evaluated. The model matters. The system around it is what earns permission to keep reviewing.
See the daemon use cases for recurring engineering work that benefits from an explicit operating contract.