The Presentation
Use arrow keys or Space to advance. Press F for fullscreen.
What This Talk Covers
A 35-minute presentation + 25-minute live workshop on how we built a 15-agent system that takes a raw ADO user story and delivers a reviewed, tested pull request — with every handoff structured and every quality gate enforced.
The problem we were solving
Three things go wrong with generalist AI assistants:
Context loss — every tool switch is a cold start. Requirements discussed in one session vanish when you start the next. The AI doesn’t know what you just read in ADO.
Manual handoffs — “dev finished, telling QE it’s ready” is not a handoff. It’s context being reconstructed, not transferred. Stuff gets lost every time, and it wastes tokens.
Inconsistent quality — a generalist AI is a jack of all trades. Without structural quality gates, review is optional. Sometimes thorough, sometimes skipped under pressure. No enforced bar.
The solution: agents as specialists. Each owns one phase. Each hands off with context preserved. Nothing gets written to ADO or Git without your explicit approval.
The multi-agent system: 15 agents in 3 tiers
| Tier | Agents | Model | Role |
|---|---|---|---|
| Orchestrators | @feature, @story, @requirements, @plan |
Claude Opus | Deep reasoning, planning, routing |
| Implementation | @code-backend, @code-frontend, @test-api, @test-e2e |
Claude Sonnet | Code generation, speed |
| Quality Gates | @author-req, @author-code, @author-tests, @code-review, @pr-create |
Sonnet / Opus / GPT-5.4 mini | Review, gate, format |
Two orchestrators, different scopes
@feature — for work that spans multiple stories. Creates a PRD, design document with API contracts, and breaks into user stories. Every phase is gated by /approve. Delivers ADO stories, not code.
@story — for a single well-scoped story ready to implement. One command (/start <id>) creates the branch, detects scope (backend / frontend / full-stack), routes to the right agents. Use --dev for code + draft PR, --qe to pick up tests. Use --auto to add author quality gates between every phase.
Auto mode & quality gates
When you add --auto, three author agents gate each phase:
- Rating ≥ 4 → advances automatically
- Rating 2–3 → refines, up to two retries
- Rating 1 → escalates to you immediately
Context management
Without optimization, every chat loads ~4,600 lines of context. The four-tier architecture brings this down 85–89%:
- Instructions — scoped by file type glob, auto-loaded, zero cost otherwise
- Skills — on-demand SKILL.md files, loaded only when the task domain matches
- Scripts — PowerShell for deterministic operations, no tokens wasted on prose
- Memory Bank — persistent context across sessions and users (ADO info, story plans, feature designs)
Drift detection
Conventions evolve through PRs. Without validation, agents follow yesterday’s rules — or teach patterns from code that no longer exists. On the first scan run: 10 phantom references caught across 6 skills, plus a wrong security rule that would have told every agent to map 401 → 404 incorrectly. Skills amplify mistakes. Review skill content like production code.
Event Details
Date: July 22, 2026 · 11:00 AM – 12:00 PM PST
Audience: Internal engineering team
Format: 35 min presentation + 25 min live workshop
Agent framework: sthirion/agent-framework (currently private)