How We Built Charlie, Part 15: What We Learned Building Charlie
What we learned about making AI engineering work owned, bounded, verifiable, and useful to a team.
How We Built Charlie continues from Chapter 11: recurring roles need durable instructions and fresh facts for each bounded activation.
Every week, the same pull request hygiene issues appear: a vague title, a missing Linear reference, a body that no longer describes the diff. The work is small, but it is easy to defer and expensive to rediscover during review.
We wanted a scoped maintainer role to own that routine. The role should wake when an open pull request changes, inspect the current pull request and repository conventions, repair only the allowed metadata, and leave code, tests, reviewers, labels, and comments alone. If the evidence is ambiguous, it should do nothing. The schedule, trigger conditions, routines, and deny rules should live beside the code so the team can review them like any other operational policy.
That is the product idea behind Charlie’s Daemons. A daemon is a persistent role, not a persistent model process. Its definition remains available in the repository, but each wake creates or joins ordinary bounded work with a Task, a run, tools, checkpoints, and a terminal result. The role persists; the compute does not.
This distinction makes recurring agent work manageable. Teams can see why a role exists, what wakes it, what it may change, and what it must refuse. The runtime can coordinate overlapping activations while keeping time, tools, and authority bounded.
Put the operating contract in the repository
A daemon begins as a file at .agents/daemons/<daemon-id>/DAEMON.md. YAML frontmatter supplies structured controls. The Markdown body supplies the operating procedure, evidence expectations, output rules, and judgment that do not fit cleanly into a few fields.
A simplified pull request metadata role might look like this:
---
id: pr-metadata
purpose: Keep open pull request metadata accurate and reviewable.
watch:
- Open pull requests when their title, body, or head changes
routines:
- Check title and body against current repository conventions
- Repair high-confidence Linear references
deny:
- Do not modify code, tests, labels, reviewers, or comments
- Do not infer an issue link when evidence is ambiguous
---
Read the current pull request before acting. Prefer linked metadata and
repository templates over guesses. If no correction is clearly supported,
finish without a visible write.The example is illustrative rather than a public API contract, but it captures the control split in Charlie’s current daemon model.
| Surface | What it owns |
|---|---|
id | Stable role identity within the repository |
purpose | The reason the role exists |
watch | Event conditions that can make the role eligible |
schedule | A cron-like cadence that can make the role eligible |
routines | The recurring work the role is expected to consider |
deny | Explicit capability and mutation boundaries |
| Markdown body | Evidence order, decision rules, coordination guidance, output format, escalation, and no-op policy |
The normalized runtime representation also derives an activation mode from the authored fields: watch-only, schedule-only, or hybrid. That value is metadata about the contract, not another knob an author has to keep synchronized.
Keeping this material in the default branch matters. A repository owner can review a daemon change in a pull request, compare it with code ownership and operational conventions, and roll it back with normal version control. Charlie does not need a hidden prompt edited in a separate control panel to understand the role.
The file is still only one layer of authority. Repository policy, available integrations, task-level instructions, and runtime safety constraints continue to apply. A sentence in Markdown cannot manufacture a permission that the system did not grant.
Waking is an eligibility decision
An event match or schedule tick answers one question: should the runtime consider activating this role now?
It does not answer whether a mutation is justified. That requires fresh evidence and the daemon’s full operating policy.
Consider three ways a role can wake:
| Wake type | Example | Eligibility established |
|---|---|---|
| Event-based | An open pull request is edited or updated | A relevant resource changed |
| Scheduled | A weekday review window arrives | It is time to inspect current state |
| Hybrid | A role watches urgent events and runs periodic checks | Either condition makes inspection eligible |
For event-based activation, routing first has to decide that the delivery belongs to the daemon. A provider event may be authentic and still be unsupported, duplicated, stale, authored by Charlie, or irrelevant to the role. Selection is a policy-aware routing result, not blind fanout from every matching word in a webhook.
For scheduled activation, the scheduler maintains durable schedule state and converts an eligible tick into an internal activation request. Temporary repository-read failures should not silently erase schedules, and a long outage should not trigger an unbounded replay of every missed tick. The goal is controlled reconciliation, not a backlog avalanche.
In both cases, the activation body can require a fresh read immediately before a write. A Linear status reconciler, for example, may wake each weekday and find that no issue has enough current evidence for an automatic transition. A successful activation can end with a no-op. That is healthy behavior when the role is written to prefer supported decisions over visible activity.
A wake becomes ordinary bounded work
Once routing approves a daemon activation, Charlie schedules it through the same Task-oriented system used for other engineering objectives. There is no separate long-lived model loop attached to the repository.
The lifecycle has familiar boundaries:
- A trusted event or durable schedule tick becomes eligible.
- Routing identifies a daemon and records why it was selected.
- Coordination determines whether the activation should create a Task or append to an active owner.
- A bounded run loads the role, sourceable context, and available tools.
- The role inspects fresh state, applies its routines and deny rules, and chooses an effect, escalation, or no-op.
- Tool results and durable references are checkpointed as execution proceeds.
- The Task reaches a terminal result.
This reuse is important because recurring work has the same hard problems as human-requested work: duplicate delivery, follow-up context, side-effect uncertainty, cancellation, tool failure, and evidence-backed completion. A separate continuous execution subsystem would need to reinvent those controls.
The bounded run also gives teams an understandable unit of cost and review. Each activation has a reason, a source, a transcript, a task identity, and an outcome. A daemon may be available every day, but it is not consuming model attention between activations.
Coordinate around the resource that owns the work
Recurring events frequently arrive faster than a useful investigation can finish. A pull request may receive a synchronize event, a review comment, and an edit in a short period. Slack alerts can be updated or followed by replies. A scheduled reconciliation may begin while a related event-based activation is already active.
Creating a new Task for each delivery would produce competing owners. They could repeat reads, post duplicate messages, or make decisions against different snapshots of the same resource.
Charlie derives daemon activation coordination from the role and a stable resource container. Fine-grained source events are mapped to the object that should own the work:
- GitHub review activity can coordinate at the pull request.
- Issue comments can coordinate at the issue or pull request they belong to.
- Slack replies can coordinate at the thread root.
- Linear comments can coordinate at their issue, project, or initiative.
- Scheduled activation can coordinate at the repository and daemon role.
The conceptual coordination identity combines the customer, daemon, and stable container. Mutable revisions, such as a pull request’s current head commit, can remain useful activation metadata without necessarily creating a new owner every time they change.
When no active owner exists, scheduling creates a Task. When a compatible daemon Task already owns that coordination scope, the new activation can be appended to its mailbox. The active run then receives the additional input at a safe checkpoint instead of racing a second run.
This is coalescing at a defined scheduler boundary. It does not create end-to-end exactly-once behavior. Some provider shapes cannot be mapped to a richer resource and may fall back to exact source identity. Unsupported cases may be skipped. External writes still need their own read-back, idempotency, or reconciliation strategy.
The role can collaborate through normal team surfaces
A daemon is most useful when it behaves like a scoped maintainer already operating in the team’s systems, where the relevant integrations are enabled.
A Sentry triage role can wake from an allowed Slack alert, read the current thread and Sentry issue, check for recent Charlie or human activity, and decide whether to add one concise thread reply or create or update a Linear issue. A pull request metadata role can repair the title or body without touching the code. A status reconciler can apply a small set of evidence-backed transitions and propose ambiguous ones for a person to review.
These experiences depend on native collaboration rather than a separate daemon dashboard:
- The pull request remains the source of truth for pull request state.
- The Slack thread remains the shared incident conversation.
- Linear remains the durable owner of planned work.
- The Task and transcript explain what Charlie observed and attempted.
The daemon file can require durable links in its handoff and can define how to avoid duplicate comments. The runtime can supply the tools and coordination context. Neither layer should pretend that posting into an external provider is transactional with Task completion.
This division also keeps deny rules concrete. “Do not modify code” is enforceable and reviewable. “Be safe” is too vague to guide a recurring maintainer. Strong daemon definitions list the surfaces that are out of scope, identify when fresh reads are required, and make no-op behavior explicit.
What failures the design is trying to prevent
Recurring automation tends to fail in recognizable ways. Encoding the role and running it through bounded Tasks lets us address each failure at the layer that owns it.
| Failure | Boundary that reduces it |
|---|---|
| One event starts several competing runs | Daemon-specific resource coordination can schedule once and append later activations |
| A schedule performs stale mutations | The role requires fresh provider reads and evidence immediately before the write |
| A useful trigger grants broad authority | watch and schedule establish eligibility; routines, deny rules, credentials, and runtime policy still govern action |
| The role expands beyond its purpose | Checked-in purpose, routines, and explicit deny rules make scope changes reviewable |
| A retry duplicates visible output | Stable source identities, provider read-back, dedupe markers, and operation-specific idempotency narrow the uncertainty |
| A temporary ingest problem erases work | Schedule reconciliation preserves prior state when repository inventory is degraded |
| Missed time creates an activation storm | Scheduled processing uses bounded catch-up rather than replaying every missed interval |
| The agent invents continuity | Current activation history and sourceable context are distinguished from dedicated persistent memory |
No row settles every failure mode. Together they move recurring work from “run this prompt sometimes” toward a system that has explicit owners, policy, recovery behavior, and evidence.
The Markdown body is where teams can encode domain judgment that would be dangerous to hide in a generic runtime. A metadata maintainer can define which repository template outranks free-form prose. A triage role can distinguish a new top-level alert from a reply or status update. A reconciler can list the transitions supported by objective provider evidence and reserve the rest for proposals. Those rules remain close to the work and can change as the repository changes.
That policy should also describe the handoff. A useful daemon result records which resource was inspected, whether an effect occurred, and the durable identifiers needed for review. If the activation declines to act, the terminal result can preserve the reason without posting another message into a busy team surface. Recurring roles earn trust partly by knowing when visible output would add no value.
A daemon re-reads the facts it needs
The word “daemon” can suggest a continuously running mind with durable personal memory. Charlie uses a smaller public contract.
The stable part is the authored role. Each activation can receive bounded, sourceable context: checked-in role instructions, the triggering Signal or schedule facts, current repository content, provider state read through tools, and relevant recent Task history. Past activations can help explain continuity.
That context is not universal personal memory. Provider configurations can vary by role and deployment, and current facts are re-read before action. A past Task can guide discovery, while the current issue, branch, dependency, or provider record remains authoritative.
The safe mental model is:
| Context source | Public contract |
|---|---|
| Checked-in role instructions | Durable, reviewable role and policy |
| Trigger and recent Task history | Bounded continuity about why activations ran and how they ended |
| Repository/provider read-back | Current source-of-truth context when tools and permissions allow |
| Dedicated personal memory | Not a universal property of daemon activation |
If a role needs a durable fact, the best home is usually the repository, an issue, a pull request, a Slack thread, or another system that people can inspect and correct. Those sources outrank hidden recollection.
Persistent ownership does not require persistent compute
The daemon abstraction gives a team a durable place to define recurring responsibility. The runtime retains the boundaries that make the responsibility safe enough to use: trusted activation, policy-aware routing, stable coordination, bounded execution, fresh verification, and terminal evidence.
That combination supports a useful kind of autonomy. A maintainer role can be available whenever its event or schedule policy makes it eligible. It can collaborate in the same tools as the team and absorb related activations without opening duplicate owners. It can also finish without a mutation when the current evidence does not support one.
The file persists. The role persists. Each run remains finite and accountable.
Continue the series
Previous: How We Built Charlie, Part 11: Context Without Magic Memory. Next: How We Built Charlie, Part 13: Proof, Not Vibes. Browse the full How We Built Charlie series.
