1# Architectural Reduction: Production Evidence
2 
3This document provides detailed evidence and implementation patterns for the architectural reduction approach to agent tool design.
4 
5## Case Study: Text-to-SQL Agent
6 
7A production text-to-SQL agent was rebuilt using architectural reduction principles. The original architecture used specialized tools with heavy prompt engineering and careful context management. The reduced architecture used a single bash command execution tool.
8 
9### Original Architecture (Many Specialized Tools)
10 
11The original system included:
12- GetEntityJoins: Find relationships between entities
13- LoadCatalog: Load data catalog information
14- RecallContext: Retrieve previous context
15- LoadEntityDetails: Get entity specifications
16- SearchCatalog: Search data catalog
17- ClarifyIntent: Clarify user intent
18- SearchSchema: Search database schema
19- GenerateAnalysisPlan: Create query plan
20- FinalizeQueryPlan: Complete query plan
21- FinalizeNoData: Handle no-data cases
22- JoinPathFinder: Find join paths
23- SyntaxValidator: Validate SQL syntax
24- FinalizeBuild: Complete query build
25- ExecuteSQL: Run SQL queries
26- FormatResults: Format query results
27- VisualizeData: Create visualizations
28- ExplainResults: Explain query results
29 
30Each tool solved a specific problem the team anticipated the model would face. The assumption was that the model would get lost in complex schemas, make bad joins, or hallucinate table names.
31 
32### Reduced Architecture (Two Primitive Tools)
33 
34The reduced system included:
35- ExecuteCommand: Run arbitrary bash commands in a sandbox
36- ExecuteSQL: Run SQL queries against the database
37 
38The agent explores the semantic layer using standard Unix tools:
39 
40```python
41from vercel_sandbox import Sandbox
42 
43sandbox = Sandbox.create()
44await sandbox.write_files(semantic_layer_files)
45 
46def execute_command(command: str):
47    """Execute arbitrary bash command in sandbox."""
48    result = sandbox.exec(command)
49    return {
50        "stdout": result.stdout,
51        "stderr": result.stderr,
52        "exit_code": result.exit_code
53    }
54```
55 
56The agent now uses `grep`, `cat`, `find`, and `ls` to navigate YAML, Markdown, and JSON files containing dimension definitions, measure calculations, and join relationships.
57 
58### Comparative Results
59 
60| Metric | Original (17 tools) | Reduced (2 tools) | Change |
61|--------|---------------------|-------------------|--------|
62| Average execution time | 274.8s | 77.4s | 3.5x faster |
63| Success rate | 80% (4/5) | 100% (5/5) | +20% |
64| Average token usage | ~102k tokens | ~61k tokens | 37% fewer |
65| Average steps | ~12 steps | ~7 steps | 42% fewer |
66 
67The worst case in the original architecture: 724 seconds, 100 steps, 145,463 tokens, and a failure. The reduced architecture completed the same query in 141 seconds with 19 steps and 67,483 tokens, successfully.
68 
69## Why Reduction Works
70 
71### File Systems Are Powerful Abstractions
72 
73File systems have 50+ years of refinement. Standard Unix tools like `grep` are well-documented, predictable, and understood by models. Building custom tools for what Unix already solves adds complexity without value.
74 
75### Tools Were Constraining Reasoning
76 
77The specialized tools were solving problems the model could handle on its own:
78- Pre-filtering context the model could navigate
79- Constraining options the model could evaluate
80- Wrapping interactions in validation logic the model didn't need
81 
82Each guardrail became a maintenance burden. Each model update required recalibrating constraints. The team spent more time maintaining scaffolding than improving the agent.
83 
84### Good Documentation Replaces Tool Sophistication
85 
86The semantic layer was already well-documented:
87- Dimension definitions in structured YAML
88- Measure calculations with clear naming
89- Join relationships in navigable files
90 
91The custom tools were summarizing what was already legible. The model needed access to read the documentation directly, not abstractions on top of it.
92 
93## Implementation Pattern
94 
95### The File System Agent
96 
97```python
98from ai import ToolLoopAgent, tool
99from sandbox import Sandbox
100 
101# Create sandboxed environment with your data layer
102sandbox = Sandbox.create()
103await sandbox.write_files(data_layer_files)
104 
105# Single primitive tool
106def create_execute_tool(sandbox):
107    return tool(
108        name="execute_command",
109        description="""
110        Execute a bash command in the sandbox environment.
111        
112        Use standard Unix tools to explore and understand the data layer:
113        - ls: List directory contents
114        - cat: Read file contents
115        - grep: Search for patterns
116        - find: Locate files
117        
118        The sandbox contains the semantic layer documentation:
119        - /data/entities/*.yaml: Entity definitions
120        - /data/measures/*.yaml: Measure calculations  
121        - /data/joins/*.yaml: Join relationships
122        - /docs/*.md: Additional documentation
123        """,
124        execute=lambda command: sandbox.exec(command)
125    )
126 
127# Minimal agent
128agent = ToolLoopAgent(
129    model="claude-opus-4.5",
130    tools={
131        "execute_command": create_execute_tool(sandbox),
132        "execute_sql": sql_tool,
133    }
134)
135```
136 
137### Prerequisites for Success
138 
139This pattern works when:
140 
1411. **Documentation quality is high**: Files are well-structured, consistently named, and contain clear definitions.
142 
1432. **Model capability is sufficient**: The model can reason through complexity without hand-holding.
144 
1453. **Safety constraints permit**: The sandbox limits what the agent can access and modify.
146 
1474. **Domain is navigable**: The problem space can be explored through file inspection.
148 
149### When Not to Use
150 
151Reduction fails when:
152 
1531. **Data layer is messy**: Legacy naming conventions, undocumented joins, inconsistent structure. The model will produce faster bad queries.
154 
1552. **Specialized knowledge is required**: Domain expertise that can't be documented in files.
156 
1573. **Safety requires restrictions**: Operations that must be constrained for security or compliance.
158 
1594. **Workflows are genuinely complex**: Multi-step processes that benefit from structured orchestration.
160 
161## Design Principles
162 
163### Addition by Subtraction
164 
165The best agents may be the ones with the fewest tools. Every tool is a choice made for the model. Sometimes the model makes better choices when given primitive capabilities rather than constrained workflows.
166 
167### Trust Model Reasoning
168 
169Modern models can handle complexity. Constraining reasoning because you don't trust the model to reason is often counterproductive. Test what the model can actually do before building guardrails.
170 
171### Invest in Context, Not Tooling
172 
173The foundation matters more than clever tooling:
174- Clear file naming conventions
175- Well-structured documentation
176- Consistent data organization
177- Legible relationship definitions
178 
179### Build for Future Models
180 
181Models improve faster than tooling can keep up. An architecture optimized for today's model limitations may be over-constrained for tomorrow's model capabilities. Build minimal architectures that benefit from model improvements.
182 
183## Evaluation Framework
184 
185When considering architectural reduction, evaluate:
186 
1871. **Maintenance overhead**: How much time is spent maintaining tools vs. improving outcomes?
188 
1892. **Failure analysis**: Are failures caused by model limitations or tool constraints?
190 
1913. **Documentation quality**: Could the model navigate your data layer directly if given access?
192 
1934. **Constraint necessity**: Are guardrails protecting against real risks or hypothetical concerns?
194 
1955. **Model capability**: Has the model improved since tools were designed?
196 
197## Conclusion
198 
199Architectural reduction is not universally applicable, but the principle challenges a common assumption: that more sophisticated tooling leads to better outcomes. Sometimes the opposite is true. Start with the simplest possible architecture, add complexity only when proven necessary, and continuously question whether tools are enabling or constraining model capabilities.
200 
201## References
202 
203- Vercel Engineering: "We removed 80% of our agent's tools" (December 2025)
204- AI SDK ToolLoopAgent documentation
205- Vercel Sandbox documentation
206 
207 
208 
209 
210 
211