1---
2name: context-optimization
3description: This skill should be used when the user asks to "optimize context", "reduce token costs", "improve context efficiency", "implement KV-cache optimization", "partition context", or mentions context limits, observation masking, context budgeting, or extending effective context capacity.
4---
5 
6# Context Optimization Techniques
7 
8Context optimization extends the effective capacity of limited context windows through strategic compression, masking, caching, and partitioning. Effective optimization can double or triple effective context capacity without requiring larger models or longer windows — but only when applied with discipline. The techniques below are ordered by impact and risk.
9 
10## When to Activate
11 
12Activate this skill when:
13- Context limits constrain task complexity
14- Optimizing for cost reduction (fewer tokens = lower costs)
15- Reducing latency for long conversations
16- Implementing long-running agent systems
17- Needing to handle larger documents or conversations
18- Building production systems at scale
19 
20## Core Concepts
21 
22Apply four primary strategies in this priority order:
23 
241. **KV-cache optimization** — Reorder and stabilize prompt structure so the inference engine reuses cached Key/Value tensors. This is the cheapest optimization: zero quality risk, immediate cost and latency savings. Apply it first and unconditionally.
25 
262. **Observation masking** — Replace verbose tool outputs with compact references once their purpose has been served. Tool outputs consume 80%+ of tokens in typical agent trajectories, so masking them yields the largest capacity gains. The original content remains retrievable if needed downstream.
27 
283. **Compaction** — Summarize accumulated context when utilization exceeds 70%, then reinitialize with the summary. This distills the window's contents while preserving task-critical state. Compaction is lossy — apply it after masking has already removed the low-value bulk.
29 
304. **Context partitioning** — Split work across sub-agents with isolated contexts when a single window cannot hold the full problem. Each sub-agent operates in a clean context focused on its subtask. Reserve this for tasks where estimated context exceeds 60% of the window limit, because coordination overhead is real.
31 
32The governing principle: context quality matters more than quantity. Every optimization preserves signal while reducing noise. Measure before optimizing, then measure the optimization's effect.
33 
34## Detailed Topics
35 
36### Compaction Strategies
37 
38Trigger compaction when context utilization exceeds 70%: summarize the current context, then reinitialize with the summary. This distills the window's contents in a high-fidelity manner, enabling continuation with minimal performance degradation. Prioritize compressing tool outputs first (they consume 80%+ of tokens), then old conversation turns, then retrieved documents. Never compress the system prompt — it anchors model behavior and its removal causes unpredictable degradation.
39 
40Preserve different elements by message type:
41 
42- **Tool outputs**: Extract key findings, metrics, error codes, and conclusions. Strip verbose raw output, stack traces (unless debugging is ongoing), and boilerplate headers.
43- **Conversational turns**: Retain decisions, commitments, user preferences, and context shifts. Remove filler, pleasantries, and exploratory back-and-forth that led to a conclusion already captured.
44- **Retrieved documents**: Keep claims, facts, and data points relevant to the active task. Remove supporting evidence and elaboration that served a one-time reasoning purpose.
45 
46Target 50-70% token reduction with less than 5% quality degradation. If compaction exceeds 70% reduction, audit the summary for critical information loss — over-aggressive compaction is the most common failure mode.
47 
48### Observation Masking
49 
50Mask observations selectively based on recency and ongoing relevance — not uniformly. Apply these rules:
51 
52- **Never mask**: Observations critical to the current task, observations from the most recent turn, observations used in active reasoning chains, and error outputs when debugging is in progress.
53- **Mask after 3+ turns**: Verbose outputs whose key points have already been extracted into the conversation flow. Replace with a compact reference: `[Obs:{ref_id} elided. Key: {summary}. Full content retrievable.]`
54- **Always mask immediately**: Repeated/duplicate outputs, boilerplate headers and footers, outputs already summarized earlier in the conversation.
55 
56Masking should achieve 60-80% reduction in masked observations with less than 2% quality impact. The key is maintaining retrievability — store the full content externally and keep the reference ID in context so the agent can request the original if needed.
57 
58### KV-Cache Optimization
59 
60Maximize prefix cache hits by structuring prompts so that stable content occupies the prefix and dynamic content appears at the end. KV-cache stores Key and Value tensors computed during inference; when consecutive requests share an identical prefix, the cached tensors are reused, saving both cost and latency.
61 
62Apply this ordering in every prompt:
631. System prompt (most stable — never changes within a session)
642. Tool definitions (stable across requests)
653. Frequently reused templates and few-shot examples
664. Conversation history (grows but shares prefix with prior turns)
675. Current query and dynamic content (least stable — always last)
68 
69Design prompts for cache stability: remove timestamps, session counters, and request IDs from the system prompt. Move dynamic metadata into a separate user message or tool result where it does not break the prefix. Even a single whitespace change in the prefix invalidates the entire cached block downstream of that change.
70 
71Target 70%+ cache hit rate for stable workloads. At scale, this translates to 50%+ cost reduction and 40%+ latency reduction on cached tokens.
72 
73### Context Partitioning
74 
75Partition work across sub-agents when a single context cannot hold the full problem without triggering aggressive compaction. Each sub-agent operates in a clean, focused context for its subtask, then returns a structured result to a coordinator agent.
76 
77Plan partitioning when estimated task context exceeds 60% of the window limit. Decompose the task into independent subtasks, assign each to a sub-agent, and aggregate results. Validate that all partitions completed before merging, merge compatible results, and apply summarization if the aggregated output still exceeds budget.
78 
79This approach achieves separation of concerns — detailed search context stays isolated within sub-agents while the coordinator focuses on synthesis. However, coordination has real token cost: the coordinator prompt, result aggregation, and error handling all consume tokens. Only partition when the savings exceed this overhead.
80 
81### Budget Management
82 
83Allocate explicit token budgets across context categories before the session begins: system prompt, tool definitions, retrieved documents, message history, tool outputs, and a reserved buffer (5-10% of total). Monitor usage against budget continuously and trigger optimization when any category exceeds its allocation or total utilization crosses 70%.
84 
85Use trigger-based optimization rather than periodic optimization. Monitor these signals:
86- Token utilization above 80% — trigger compaction
87- Attention degradation indicators (repetition, missed instructions) — trigger masking + compaction
88- Quality score drops below baseline — audit context composition before optimizing
89 
90## Practical Guidance
91 
92### Optimization Decision Framework
93 
94Select the optimization technique based on what dominates the context:
95 
96| Context Composition | First Action | Second Action |
97|---|---|---|
98| Tool outputs dominate (>50%) | Observation masking | Compaction of remaining turns |
99| Retrieved documents dominate | Summarization | Partitioning if docs are independent |
100| Message history dominates | Compaction with selective preservation | Partitioning for new subtasks |
101| Multiple components contribute | KV-cache optimization first, then layer masking + compaction |
102| Near-limit with active debugging | Mask resolved tool outputs only — preserve error details |
103 
104### Performance Targets
105 
106Track these metrics to validate optimization effectiveness:
107 
108- **Compaction**: 50-70% token reduction, <5% quality degradation, <10% latency overhead from the compaction step itself
109- **Masking**: 60-80% reduction in masked observations, <2% quality impact, near-zero latency overhead
110- **Cache optimization**: 70%+ hit rate for stable workloads, 50%+ cost reduction, 40%+ latency reduction
111- **Partitioning**: Net token savings after accounting for coordinator overhead; break-even typically requires 3+ subtasks
112 
113Iterate on strategies based on measured results. If an optimization technique does not measurably improve the target metric, remove it — optimization machinery itself consumes tokens and adds latency.
114 
115## Examples
116 
117**Example 1: Compaction Trigger**
118```python
119if context_tokens / context_limit > 0.8:
120    context = compact_context(context)
121```
122 
123**Example 2: Observation Masking**
124```python
125if len(observation) > max_length:
126    ref_id = store_observation(observation)
127    return f"[Obs:{ref_id} elided. Key: {extract_key(observation)}]"
128```
129 
130**Example 3: Cache-Friendly Ordering**
131```python
132# Stable content first
133context = [system_prompt, tool_definitions]  # Cacheable
134context += [reused_templates]  # Reusable
135context += [unique_content]  # Unique
136```
137 
138## Guidelines
139 
1401. Measure before optimizing—know your current state
1412. Apply masking before compaction — remove low-value bulk first, then summarize what remains
1423. Design for cache stability with consistent prompts
1434. Partition before context becomes problematic
1445. Monitor optimization effectiveness over time
1456. Balance token savings against quality preservation
1467. Test optimization at production scale
1478. Implement graceful degradation for edge cases
148 
149## Gotchas
150 
1511. **Whitespace breaks KV-cache**: Even a single whitespace or newline change in the prompt prefix invalidates the entire KV-cache block downstream of that point. Pin system prompts as immutable strings — do not interpolate timestamps, version numbers, or session IDs into them. Diff prompt templates byte-for-byte between deployments.
152 
1532. **Timestamps in system prompts destroy cache hit rates**: Including `Current date: {today}` or similar dynamic content in the system prompt forces a full cache miss on every new day (or every request, if using time-of-day). Move dynamic metadata into a user message or a separate tool result appended after the stable prefix.
154 
1553. **Compaction under pressure loses critical state**: When the model performing compaction is itself under context pressure (>85% utilization), its summarization quality degrades — it omits task goals, drops user constraints, and flattens nuanced state. Trigger compaction at 70-80%, not 90%+. If compaction must happen late, use a separate model call with a clean context containing only the material to summarize.
156 
1574. **Masking error outputs breaks debugging loops**: Over-aggressive masking hides error messages, stack traces, and failure details that the agent needs in subsequent turns to diagnose and fix issues. During active debugging (error in the last 3 turns), suspend masking for all error-related observations until the issue is resolved.
158 
1595. **Partitioning overhead can exceed savings**: Each sub-agent requires its own system prompt, tool definitions, and coordination messages. For tasks with fewer than 3 independent subtasks, the coordination overhead often exceeds the context savings. Estimate total tokens (coordinator + all sub-agents) before committing to partitioning.
160 
1616. **Cache miss cost spikes after deployment changes**: Reordering tools, rewording the system prompt, or changing few-shot examples between deployments invalidates the entire prefix cache, causing a temporary cost spike of 2-5x until the new cache warms up. Roll out prompt changes gradually and monitor cache hit rate during deployment windows.
162 
1637. **Compaction creates false confidence in stale summaries**: Once context is compacted, the summary looks authoritative but may reflect outdated state. If the task has evolved since compaction (new user requirements, corrected assumptions), the summary silently carries forward stale information. After compaction, re-validate the summary against the current task goal before proceeding.
164 
165## Integration
166 
167This skill builds on context-fundamentals and context-degradation. It connects to:
168 
169- multi-agent-patterns - Partitioning as isolation
170- latent-briefing - Selective KV retention across orchestrator–worker boundaries (compatible models)
171- evaluation - Measuring optimization effectiveness
172- memory-systems - Offloading context to memory
173 
174## References
175 
176Internal reference:
177- [Optimization Techniques Reference](./references/optimization_techniques.md) - Read when: implementing a specific optimization technique and needing detailed code patterns, threshold tables, or integration examples beyond what the skill body provides
178 
179Related skills in this collection:
180- context-fundamentals - Read when: unfamiliar with context window mechanics, token counting, or attention distribution basics
181- context-degradation - Read when: diagnosing why agent performance has dropped and needing to identify which degradation pattern is occurring before selecting an optimization
182- evaluation - Read when: setting up metrics and benchmarks to measure whether an optimization technique actually improved outcomes
183 
184External resources:
185- Research on context window limitations - Read when: evaluating model-specific context behavior (e.g., lost-in-the-middle effects, attention decay curves)
186- KV-cache optimization techniques - Read when: implementing prefix caching at the inference infrastructure level (vLLM, TGI, or cloud provider APIs)
187- Production engineering guides - Read when: deploying context optimization in a production pipeline and needing operability patterns (monitoring, alerting, rollback)
188 
189---
190 
191## Skill Metadata
192 
193**Created**: 2025-12-20
194**Last Updated**: 2026-03-17
195**Author**: Agent Skills for Context Engineering Contributors
196**Version**: 2.0.0
197