The right context is everything. All context is not.
Before we optimised our agent framework, every chat session loaded approximately 4,600 lines of context. Frontend naming rules when writing a C# endpoint. Test patterns when asking an architecture question. Markdown formatting rules in the middle of writing a SQL migration. Most of it was wrong for the task in front of the agent.
There’s a debate in AI engineering circles about whether that’s a problem. One camp argues: give the agent everything it could ever need — all domain knowledge, all languages, all team conventions, all processes, all repositories. That way it’s never missing context. I understand the appeal. But I think it’s wrong — and not just slightly wrong. Wrong in a way that compounds with every team, repo, and rule you add.
An agent loaded with enterprise-wide knowledge doesn’t become more capable. It becomes confused. Backend naming conventions conflict with frontend ones. Test patterns from Team A contradict patterns from Team B. Security rules for one stack become noise when the agent is scaffolding in another. The model can’t tell which rules apply — so it averages across them, makes assumptions, or hallucinates a synthesis that looks plausible but reflects nothing you actually intended. We saw this: agents producing code that blended C# conventions with TypeScript patterns because both were present in context and neither was explicitly scoped.
There’s also the cost argument: every token you load into context is a token you pay for. Every session. Whether it’s relevant or not.
After building a 15-agent system, we landed on a four-tier architecture that brings context down to 550–700 lines per session — an 85–89% reduction — without removing a single rule that matters. The reduction isn’t from knowing less. It’s from loading exactly what’s needed, exactly when it’s needed.
The four tiers
Tier 1: Instructions (always-on, scoped by file)
Instructions are rules that should apply automatically whenever you’re in a specific context. The key insight: scope them to file types using applyTo globs.
C# naming conventions load only when you’re editing .cs files. TypeScript rules load only for .ts and .tsx. React component patterns load only in the frontend directory.
You’d be frustrated if the agent forgot these mid-edit. But loading them for every session — including sessions where they don’t apply — is how you get an agent that applies C# conventions to a Python script because both were present in context and it couldn’t tell which to weight. With instructions, the failure mode is wrong output, not just slow or expensive output: conflicting rules produce a plausible-looking result that violates the conventions you actually want.
When to use instructions: If you’d be annoyed that the agent violated this rule during work on that file type — it’s an instruction.
Tier 2: Skills (on-demand, loaded by task domain)
Skills are SKILL.md files that an agent loads when the task domain matches. Test patterns load when you’re writing tests. The PR review checklist loads when you’re reviewing code. The new endpoint scaffold loads when you’re creating an API endpoint.
You’d never think about test patterns until you sit down to write tests. Loading them into every session is pure waste. Note that the failure mode here is different from instructions: irrelevant skills don’t produce wrong output so much as unfocused output — the model’s attention is diluted across rules that don’t apply, and you pay for every token of that distraction.
When to use skills: If you’d only ever think about this rule when doing a specific, named task — it’s a skill.
Tier 3: Scripts (deterministic operations, zero prose)
Some operations don’t need language model reasoning. Validating that a branch name matches a pattern. Scanning for deprecated imports. Counting test file coverage. Checking if a type name exists in the codebase.
These are deterministic: given inputs, one correct answer. Writing prose instructions for something that should be a function call wastes tokens and introduces variability where you want none. A script gives you a guaranteed result at zero inference cost.
This tier is different in kind from the other three — it’s not about what the agent reads, but about what the agent doesn’t need to read because a function answered the question first. In practice, scripts run as pre-flight checks, tool calls, or CI steps that gate agent execution: the agent sees the result, not the reasoning.
When to use scripts: If the answer is deterministic given known inputs — it’s a script, not an instruction.
Tier 4: Memory Bank (persistent context across sessions)
Some context needs to survive between sessions. Your project management configuration (ADO org and project IDs, GitHub project settings, Jira board details). Story plans mid-flight. Feature design documents. Architectural decisions.
A backend agent doesn’t need to load the frontend feature roadmap any more than a backend developer needs to read the design team’s Figma files before writing a controller. The memory bank holds persistent context at rest — agents read only the files relevant to their current task. Story agents read story plans. Feature agents read feature designs. Cross-session domain knowledge lives here, available to the agents that need it and invisible to those that don’t.
When to use memory bank: If the context needs to persist across sessions and be read selectively by multiple agents — it belongs in the memory bank.
What this costs you — and what it saves
At current model pricing, context tokens are not free. A session loading 4,600 lines vs. 550 lines is roughly an 8x difference in context cost, per session, per agent. Across a team running dozens of agent sessions per day, that compounds fast.
| Scenario | Before | After | Reduction | Cost impact |
|---|---|---|---|---|
| Generic chat | ~4,600 lines | ~550 lines | 88% | ~8x cheaper per session |
| Backend (C# endpoint) | ~4,600 lines | ~700 lines | 85% | ~6.5x cheaper per session |
| Frontend (React component) | ~4,600 lines | ~700 lines | 85% | ~6.5x cheaper per session |
At current Claude Sonnet pricing (~$3/M input tokens), a 10-person team running 50 agent sessions per day moves from roughly $0.07 to $0.009 per session in context cost alone — around $1,800/year saved on context tokens, before counting the output quality improvement that makes retries less frequent. The exact numbers shift with model and pricing tier, but the ratio is structural: loading 8x fewer tokens costs 8x less.
The model right-sizing benefit
Scoped context also unlocks something the omniscient-agent approach forecloses: using smaller, faster, cheaper models for focused tasks.
An agent loaded with enterprise-wide knowledge needs a large reasoning model to make sense of it — you need the capacity to weigh conflicting rules, infer which context applies, and reason across a sprawling information space. That’s expensive capacity, and you pay for it on every token.
An agent with 550 lines of precisely relevant context can run on a significantly smaller model. It doesn’t need to reason across competing conventions — it only has the conventions that apply. Focused inputs enable focused models. And focused models are faster, cheaper, and more predictable.
The omniscient approach doesn’t just cost more per session. It forces you to use more capable (and more expensive) models to compensate for the noise you created.
The decision rule
When you’re deciding where a piece of context belongs, ask:
- Would you be frustrated if the agent violated this rule during work it wasn’t asked to do? → Instruction
- Would you never think about this rule until you sit down for that specific task? → Skill
- Is the answer deterministic given known inputs? → Script
- Does it need to persist across sessions and be read selectively by multiple agents? → Memory Bank
Most teams put everything in instructions. That’s the equivalent of reading the entire company handbook before answering every email. You’d be right more often than if you read nothing — but you’d be slow, expensive, and you’d get confused by rules that don’t apply to the question in front of you. You’d also need a smarter person to hold it all together.
That last point matters. Scoped context doesn’t just reduce cost — it means you can add new teams, new repos, and new conventions without polluting every existing agent’s context space. The architecture scales horizontally. Each new domain gets its own instructions, skills, and memory entries. Existing agents never see them unless the task demands it.
The right context, at the right time, for the right agent. That’s the architecture.
The framework is the easy part. The hard part is the placement decisions — what counts as an instruction versus a skill, when a pattern becomes a convention that deserves its own scope, which persistent context actually needs to survive between sessions and which is just clutter you’re afraid to delete. Get those wrong and you rebuild the noise problem with better labels.
We’ve made most of those mistakes across a 15-agent system. If you’re in that stage — or trying to figure out whether context architecture is even your real problem — I’m happy to think through it. That’s usually where the useful conversation starts.