Ten Tickets or One: Cutting Setup Cost in Our Agent Loop

We run three products with one agent loop and one human. Claude Code writes, reviews and ships the code. Codex reviews it too. In the last few weeks on the old setup, the pipeline merged 650 pull requests.

Then we changed two things at once.

1. GitHub Projects to Milestones. An issue used to sit on a Projects board and move from backlog to ready to in progress to done. Now it belongs to a milestone and is open or closed.
2. Many small tickets to one large one. We used to file one issue per small change and open one PR per issue. Now we file one package per outcome, with a definition of done, and it takes as many PRs as it needs.

Both changes trace back to an idea from the Toyota production system that we had applied halfway. Toyota runs small batches, and it could afford them because it cut setup time, the time it takes to switch a line from one part to the next. Our pipeline had the small batches and kept the long setups.

What we found: in our data, large tickets cost far fewer tokens per line than small ones. The small ones were how we filed large work: one feature cut into many tickets, each run through the whole cycle on its own. After the switch, tokens per merged PR fell by about 15%.

How we read our sessions

Claude Code writes every session to a local transcript, one JSON line per message, with token usage on each response. Sessions a subagent runs are stored under the session that started them. We read those files directly.

First we tie each session to a ticket. The dispatch prompt names the issue or package, and the branch name, worktree path or reviewed PR fills the gaps. That ties 40 to 50% of all tokens to a ticket. The rest stays unassigned.

Then we sort each session’s tokens. The first response’s context is setup, and every later turn reads it again. Each shell command is sorted by what it calls. Board calls, package queue calls and issue reads count as coordination, and whatever remains is the work. That is how we found that a quarter of our board calls went around the helper scripts straight to the API.

Last, we look for what repeats: the same context in every session, the same call on every ticket, the same setup paid whatever the size of the change. Lines changed come from GitHub, so we can set each ticket’s cost against the size of its change.

Every figure is a token count, with cache reads counted at full size. That measures token volume, not dollars, since cached input is priced differently. Session totals are measured. The overhead split and the per-ticket fit are estimates. Codex review tokens are not included.

What one ticket costs

A ticket in our pipeline is not one agent call. It is a chain of agent sessions, built on Addy Osmani’s agent-skills:

1. A runner picks the next ticket and dispatches it.
2. A builder reads the issue and the repo’s instructions, writes the change and opens a PR.
3. Reviewers check the PR: code review, a simplify pass, a security review when the diff touches the server, a legal review when it touches user-facing claims.
4. Fix rounds address what the reviewers found.
5. The PR merges and the ticket closes.

Each session starts with its own context: the system prompt, the tool definitions, the repo’s agent instructions and the dispatch prompt. The median dispatch on the board started with 55,000 tokens and ran 24 turns, and every turn reads the prior context again. A merged PR took 5.6 sessions.

Part of that cost recurs whatever the size of the change. A 13-line copy fix on an about page ran three sessions and used 7.8M tokens. The median ticket under 100 lines used 4.8M. A straight-line fit of tokens against lines changed puts the size-independent part at about 13M tokens per ticket, plus about 9,300 per line. A few expensive tickets pull that intercept up, and it does not isolate the cost of coordination.

We call this setup time, after the Toyota example above. Shigeo Shingo’s single-minute exchange of die cut Toyota’s die changes from hours to minutes, so a press could run a small batch without losing the day to the changeover. Booting an agent session is our changeover.

Many small tickets or one large one

Picture one feature filed two ways. Ten separate issues each describe a slice of the work. One package describes the whole outcome: what the user can do when it is done, and why that matters. Ten is about how many tasks a human engineer would split a feature like this into. Both ways deliver the same behavior and must pass the same acceptance checks. The difference is where the full picture lives. With separate issues, it is spread across independently dispatched tickets. With a package, it sits in one issue the builder reads end to end.

Activity	Many separate issues	One coordinated package
Describe the work	A slice of the outcome per issue	The whole outcome and its value, in one issue
Classify, label and prioritize	Repeated per issue	Primarily per package
Select work and track status	Queue selection and status changes per issue	Package selection plus internal progress tracking
Load shared context	Repeated across independent sessions	Reused where the same session can continue
Implement, test and review	Required for the same feature	Required for the same feature
Accept the complete outcome	Required	Required

The package should remove repeated administration and context loading. An epic saves nothing if each child issue runs the same independent pipeline as before.

We have not filed one feature both ways to compare. What we have is the board era’s tickets sorted by size: ten times the median small ticket (150 to 500 lines changed) beside the median large ticket (more than 1,500 lines). These are different tickets, not one feature delivered two ways:

	10 × median small ticket	Median large ticket
Lines changed	2,780	4,338
Agent sessions	50	15
Estimated overhead tokens	37M	11M
Total tokens	78M	26M
Tokens per 1,000 lines changed	28M	6M

The median large ticket changed 56% more code with a third of the tokens. Task type and complexity may differ between the groups, so the table does not prove a saving for equivalent work. That gap is why we think the package bet is worth testing. Almost all of the estimated overhead is context loading: 7.2M of 7.5M tokens per merged PR on the board.

Keep commits small, let PRs grow

In our loop, agents do the routine PR review, so nothing forces a PR to fit one human sitting. A PR can cover a whole package while its commits stay small, each one describing a single change. Keep those commits intact when merging, so a change can be found and cherry-picked with its dependencies. We have not shown that agent review quality holds as diffs grow. The aim is less repeated setup, and larger PRs follow from grouping the work.

Why we had small tickets

GitHub Projects was great while it lasted

Projects gave us one view across many repos, with a status per card, and we used it for years. Milestones have been part of GitHub Issues for over a decade, but a milestone lives in one repo. We worked in polyrepos, with api, dashboard, infrastructure and marketing-website each in their own, so Projects was the only cross-repo view. We have since moved to one repo per product, so an agent can work across the API, dashboard and site in one branch. That is our repository choice. Teams with separate repos have their own reasons to keep them.

A board is a human interface

A Projects board answers one question: what is everyone working on right now? That matters when people work on tickets for days and others need to know where things stand.

Our pipeline does not work like that. By the time a card says “In progress”, the PR is often merged. Nobody, human or agent, made a decision based on that column.

We also rewarded small cards. A card that moves every day looks like progress, and one that sits in “In progress” for days looks stuck. We cut work into tickets a human could review in one sitting and watch move, and the pipeline inherited that shape. Card movement is a vanity metric: a number that looks good and drives no decision. Cards moved and PRs merged both went up as tickets got smaller. Neither told us what a feature cost.

What the board cost

GitHub Projects has no simple status field an agent can set. Setting one takes a GraphQL mutation with the project, item, field and option IDs. We ran two boards per product, and every status change carried a comment on the issue. We had Claude write 1,923 lines of helper scripts and board config so agents could use them. They took 29 commits in three weeks. The commit that removed them deleted 5,073 lines, including tests.

Over those last weeks, agents made 1,704 board calls, an estimated 44.9M tokens. Reading board state was the largest share at 13.6M. Seven sessions that built and fixed the board scripts used another 115M tokens. That figure counts whole sessions, so it is not directly comparable to the per-call estimate.

The helpers also kept unattended runs from stalling. Inline GraphQL trips Claude Code’s shell permission check, and a call that waits for approval stops a run nobody is watching. We did not measure how many tokens they saved.

The board itself was not the big cost. 44.9M tokens is small next to the context-loading totals. Our main suspect is the repeated coordination around small tickets, a habit of our workflow that Projects never required.

How we work now

A milestone is an outcome, with no due date. For example: “A new customer can sign up, pay and use the product without help.” A priority ranking across milestones sets the order of the work. Its issues are open or closed, with no status column, and it uses the REST API with plain issue numbers. A milestone alone is not a hypothesis. We still need a prediction and a measure of the outcome.

A package is one issue per outcome inside a milestone. It holds what a builder needs to finish without coming back to ask:

## Outcome
What is true for the user when this is done.

## Definition of done
- Each check a reviewer can verify against the running product.

## Evidence
The tests, screenshots or queries the final PR or the acceptance step must show.

## Out of scope
What a builder should not touch, so the package does not grow.

This is the shape Addy Osmani’s notes on getting the most out of Opus 5.5 ask for: give the whole task in one message, name the finish line and let it work.

A package closes through acceptance, not through a PR. When the definition of done is met, an acceptance step checks the evidence and closes the package. In the last board week, GitHub linked 208 closed issues to the PR that closed them. In the first Milestones week it linked 12, by design.

The backlog gets consolidated. Instead of loading context for a few hundred open issues, an agent loads a dozen packages. Every inbound ticket is absorbed into a package or parked as an idea. In the first week, the three products closed 334 issues. 116 were absorbed, 159 parked, 11 closed by a PR and 48 closed otherwise. We filed 68 packages, and 50 have merged PRs so far. Open issues fell from 261 to 109, and new issues per week from 383 to 160.

Packages are bigger than the issues they replaced. Measured on those 50 packages, against the 470 single issues closed through a PR in the board era:

	Single issue (Projects)	Package (Milestones)
Lines changed, median	305	512
Lines changed, 75th percentile	832	1,465
PRs per unit, median	1	2
Units with more than one PR	2%	54%

What the first two weeks show

Tokens per merged PR across the three products, for the last weeks on the board and the first weeks on Milestones. Overhead is coordination under either tracker: each session’s starting context, issue reads and tracker calls. Its rows are estimated allocations, and Work is what remains. The Milestones board calls come from the first day, before the board tooling was removed.

Tokens per merged PR	Projects	Milestones
Context loading	7.2M	7.3M
Issue reads	0.22M	0.50M
Board calls	0.07M	0.02M
Package queue and milestone calls	0	0.37M
Overhead	7.5M	8.2M
Work	13.3M	9.4M
Total	20.8M	17.6M

Tokens per merged PR fell by about 15%, and the fitted size-independent part dropped from 13M to 9.5M tokens per ticket. Sessions got shorter: a median of 14 turns instead of 24, starting from 46,000 tokens of context instead of 55,000. A before-and-after comparison cannot separate the effects of the tracker, the package size and the review policy.

The gain is smaller than the size comparison suggests, because the first version added costs of its own:

• The queue costs more than the board. It made 2,283 calls in those first weeks: 0.37M tokens per PR against the board’s 0.07M. Part of that was the tests and fixes of a tool still being built.
• The first week went into consolidation. 275 issues were absorbed or parked, and the loop spent part of the week working on itself.

If each PR inside a package starts fresh builder and reviewer sessions, most of the repeated context loading stays, whichever tracker holds the work. We keep review before merge, because it catches defects before they spread. Toyota’s jidoka calls this exposing errors early. Every PR gets one Claude and one Codex review session.

We also made the switch without writing down a prediction, so we had no number to check the first weeks against.

What we change next

Each change starts from a recorded baseline, makes one change and writes its prediction down before it ships. We check the same number a week later.

• Cut setup time. Load less context per role, and keep one builder through a whole package instead of booting one per PR. Prediction: the fitted size-independent part falls below 9.5M tokens.
• Count the loop every week. Report tokens, sessions and review sessions per accepted package next to escaped defects and reverts. PR and issue counts stay denominators. Tokens per line as a target would reward writing more code.
• Check the outcome. A week after a package closes, check whether the milestone’s user metric moved.

Lessons learned

Most of these hold with or without AI. Analyze the process, optimize it, then automate it. We automated first.

• Analyze before you change anything. Measure the whole cycle and write down what a change should move. We ran our analysis only after the switch.
• Optimize before you automate. We wrote 1,923 lines of helper scripts so agents could move cards on a board nobody made decisions from. We could have dropped the board instead.
• Automation repeats whatever the process does. Ours cut each feature into small tickets with long setups, and the agents repeated that setup on every ticket.
• Adjust the process to what the models can do. We sized tickets for a human reviewer in one sitting. Current models can take a whole outcome in one message, so the ticket can describe the whole outcome.
• Write one ticket per outcome, with a definition of done a reviewer can check. If ten tickets add up to one feature, file the feature.
• Keep a board where humans work. The case against it here is narrow: agent work that moves faster than anyone reads it.
• Analyze your own sessions. Tie each one to a ticket, sort its tokens and look for what repeats. Claude Code deletes old transcripts, so look before they are gone.