Orchestration Quality Patterns

What You’ll Learn

Session 27 covered the Workflow tool. This session covers the design question that matters more: how do you make a multi-agent result more trustworthy than a single plausible answer?

You will learn seven patterns:

1. orthogonal search;
2. explicit evidence contracts;
3. adversarial verification;
4. progressive synthesis;
5. loop-until-dry stopping;
6. visible coverage bounds; and
7. human or automated quality gates.

These are architectural patterns, not undocumented Workflow API guarantees. Express them in the task and let Claude write the current supported script.

1. Orthogonal Search, Not Cloned Fan-Out

Five identical prompts tend to reproduce the same blind spots. Split the search by method or boundary instead:

• inspect changed functions locally;
• trace callers and downstream contracts;
• examine error and recovery paths;
• test authorization or trust boundaries;
• compare documentation, tests, and implementation.

use a workflow to review this change through four independent lenses:
local logic, call-site contracts, failure recovery, and security boundaries.
Keep each lens separate until synthesis.

Independence matters. If every finder receives another finder’s answer first, the group can anchor on the same early mistake.

2. Define an Evidence Contract

Before spawning more workers, define what a finding must contain. A useful minimum is:

Field	Purpose
Claim	One falsifiable statement
Location	File, symbol, URL, or command
Evidence	The observation supporting the claim
Reproduction	Inputs and steps, when applicable
Impact	What actually breaks and for whom
Confidence	Confirmed, plausible, or unverified

Ask finders to return compact records and to say NONE when they found no qualifying result. Do not let the final writer infer evidence that a finder did not supply.

3. Adversarial Verification

Discovery and verification should have opposite incentives. The finder tries to surface a candidate; the verifier tries to disprove it.

For every candidate, launch an independent verifier with this instruction:
assume the claim is wrong; reproduce it, identify the strongest counterexample,
and mark it confirmed, refuted, or unverified with evidence.
Do not count a rate limit or tool failure as refutation.

That last distinction mirrors the current /deep-research behavior: a claim that could not be checked is reported as unverified, not silently treated as false.

For high-risk work, use different verification lenses such as correctness, security, and operational impact. Multiple opinions are useful only when their evidence is independent.

4. Synthesize After Verification

The final synthesizer should not merely concatenate agent output. Give it a narrower job:

1. discard refuted candidates;
2. keep unverified claims visibly separate;
3. deduplicate by root cause, not wording;
4. rank confirmed findings by observable impact; and
5. cite the evidence record for every conclusion.

This stage is a justified barrier: it needs the verified set as a whole to compare, merge, and rank.

5. Loop Until the Search Is Dry

A one-pass audit proves only that the first prompts returned something. For open-ended discovery, use an explicit stop condition:

use a workflow to find flaky tests in this repository.
Run independent rounds, deduplicate against every candidate already seen,
and stop after two consecutive rounds produce no new candidates.
Also stop if the same failures repeat without new evidence.

Two important details:

• Deduplicate against all seen candidates, including rejected ones, or rejected claims may be rediscovered forever.
• “No progress” must be observable: no new files, no new evidence, or no change in the failing check.

A dry-round rule is a practical stop heuristic, not proof that the search space is mathematically exhausted.

6. Make Coverage Bounds Visible

Every finite review has a boundary. Record it instead of hiding it:

• directories and file types included;
• files skipped and why;
• sampled versus exhaustive paths;
• commands that failed or timed out;
• claims that remained unverified;
• workflow stopped by the user, a runtime cap, or a stated no-progress rule.

Run a small slice before a costly workflow. /workflows exposes per-agent token usage and lets you stop a run. The size guideline and Large workflow warning are advisory; neither is a documented token-budget API.

7. Put Gates Where Failure Is Expensive

Use a gate before an irreversible or high-risk phase:

• require a passing test before synthesis;
• require every confirmed finding to have a reproduction;
• separate implementation and deployment into different runs;
• ask for human approval before schema changes or release actions.

Workflows cannot ask ordinary user questions mid-run, so place human sign-off between workflows.

When workers need to debate or coordinate continuously, an agent team may fit better. Agent teams share tasks and can message peers directly. They are experimental and disabled by default; they also cost more and work poorly for sequential or same-file work. TaskCompleted and TeammateIdle hooks can reject premature completion and feed corrective instructions back to teammates.

A Reference Review Shape

Discover independently
  ├─ local logic
  ├─ cross-module contracts
  ├─ failure paths
  └─ trust boundaries
          │
          ▼
Normalize evidence records
          │
          ▼
Adversarially verify each candidate
  ├─ confirmed
  ├─ refuted
  └─ unverified
          │
          ▼
Deduplicate, rank, and report coverage bounds

This design scales down too. Two finders and one verifier can outperform twenty cloned agents when the roles and evidence contract are clear.

Anti-Patterns

• Clone swarm: many copies of one vague prompt
• Consensus as proof: voting without independent evidence
• Verifier contamination: showing every reviewer the same proposed conclusion first
• Invisible sampling: presenting a partial sweep as exhaustive
• Failure collapse: treating timeout, denial, or unavailable data as “false”
• Endless search: no dry-round, passing-check, or no-progress condition
• Invented runtime contract: relying on undocumented cost, exception, or null-return semantics

Key Insight

Multi-agent quality comes from separation of incentives and visible evidence, not agent count. Search broadly, verify skeptically, stop explicitly, and report the boundary. The workflow is valuable because it can encode that structure repeatedly—not because fan-out alone guarantees correctness.

Next Session

Session 29 covers Persistent Memory: keeping durable knowledge available without turning old conclusions into unquestioned truth.

Official Sources

• Dynamic workflows and quality examples
• Agent teams, shared tasks, and quality hooks
• Subagent design and isolation