1# FAOS (Foundry Agent Optimization Service) Optimize Python Agent
2 
3Convert existing Python agent code into a FAOS optimization-ready version by wiring runtime configuration knobs to the FAOS config contract. This workflow prepares source code for optimization, asks the user to review the changes, and then routes to Foundry deployment only after explicit user approval.
4 
5## When to Use This Skill
6 
7USE FOR: make my Python agent FAOS optimizable, add FAOS_Config, add `load_config`, enable optimization config, make this agent optimization-ready, convert Python agent for FAOS optimization, wire evaluator-driven optimization knobs, expose prompt/model/temperature for FAOS.
8 
9DO NOT USE FOR: non-Python agents, deploying an agent directly, running batch evaluations, prompt optimization of an already deployed agent without source-code changes, or general Foundry deployment. For deployment, use [deploy](../deploy/deploy.md). For evaluator runs and prompt optimization loops, use [observe](../observe/observe.md).
10 
11## Scope
12 
13- Python only for now.
14- Works across Python frameworks and runtimes when there are identifiable instructions/model/options surfaces.
15- The FAOS config contract is framework-neutral. Framework-specific work is limited to finding the correct insertion points and preserving the existing runtime.
16- Do not switch frameworks, hosting adapters, protocols, or entrypoints unless the user explicitly asks.
17- Do not deploy automatically. Always stop for review first, then suggest Foundry deployment.
18 
19## Quick Reference
20 
21| Property | Value |
22|----------|-------|
23| Supported language | Python |
24| Required pattern | `from agent_optimization import load_config` |
25| Required knobs | instructions, model |
26| Optional knobs | temperature, skills directory, learned skills, max tokens, tool/retrieval options when safe |
27| Review gate | Mandatory before deploy |
28| Next workflow | [deploy](../deploy/deploy.md) after user approval |
29 
30## Workflow
31 
32### Step 1: Resolve Target Agent Root
33 
34Use the parent Microsoft Foundry project context resolution rules. If the user provides a path, use that path directly. Otherwise discover `.foundry/agent-metadata*.yaml` or agent source indicators in the workspace.
35 
36After selecting an agent root, stay inside that root. Do not scan sibling agent folders unless the user explicitly switches target roots.
37 
38### Step 2: Confirm Python Eligibility
39 
40Detect Python using one or more of:
41 
42- `requirements.txt`
43- `pyproject.toml`
44- `setup.py`
45- `*.py` entrypoints
46 
47If the target is not Python, stop and explain that FAOS source-code conversion is Python-only for now. If the target contains multiple languages, modify only the Python agent entrypoint unless the user approves a broader change.
48 
49### Step 3: Resolve Evaluator Objective
50 
51FAOS optimizes behavior against evaluator signals, so first identify what the code should become optimizable for.
52 
53Inspect these sources, in order, when available:
54 
551. User-stated evaluator objective, for example `tool_call_accuracy`, `intent_resolution`, or `relevance`
562. Selected `.foundry/agent-metadata*.yaml` `evaluationSuites[]`, legacy `testSuites[]`, or legacy `testCases[]`
573. `.foundry/evaluators/*.yaml`
584. `.foundry/results/**` summaries or recent failure analysis files
595. Existing code comments, README guidance, or test names describing target behavior
60 
61If evaluator context is unknown, continue with a conservative base conversion and tell the user that evaluator-specific targeting may produce better FAOS results.
62 
63### Step 4: Build Python Knob Inventory
64 
65Scan the selected agent root for configurable behavior surfaces. Prefer semantic reads of source files over broad string replacement.
66 
67Look for:
68 
69- Instructions: `instructions=`, `system_prompt`, `SYSTEM_PROMPT`, `prompt=`, `system_message`, `developer_message`
70- Model selection: `model=`, `deployment=`, `MODEL_DEPLOYMENT_NAME`, `AZURE_OPENAI_DEPLOYMENT`, framework-specific model fields
71- Generation options: `temperature`, `top_p`, `max_tokens`, `response_format`, `tool_choice`, `parallel_tool_calls`
72- Agent topology: `Agent(`, `agents=[...]`, `handoffs`, `supervisor`, `router`, `planner`, `executor`, `critic`, `synthesizer`, `WorkflowBuilder`, `StateGraph`
73- Tool/retrieval surfaces: tool decorators, tool descriptions, argument schemas, retriever settings, index names, search limits
74- Hosting entrypoint: FastAPI/Flask apps, `ResponsesHostServer`, uvicorn, custom response loops, LangGraph servers
75 
76Create an internal inventory with file path, symbol/name, role, current default, and whether it is safe to expose through FAOS config.
77 
78### Step 5: Classify Agent Topology
79 
80Classify the architecture before editing:
81 
82| Topology | Default FAOS targeting |
83|----------|------------------------|
84| Single agent | Wire config directly to the agent's instructions/model/options |
85| Multi-agent with obvious orchestrator/supervisor | Target the orchestrator by default, unless evaluator context points elsewhere |
86| Multi-agent with specialist tool agent | Target the specialist/tool path when evaluators focus on tool or task behavior |
87| Multi-agent peer architecture with no orchestrator | Present a plan and ask before editing |
88| Unknown Python runtime | Add only the minimal config loader and propose exact manual wiring points |
89 
90Do not collapse multiple role-specific prompts into a single global `SYSTEM_PROMPT`. Preserve specialist prompts as defaults unless the user asks to optimize them together.
91 
92### Step 6: Map Evaluators to Candidate Knobs
93 
94Use evaluator context to select the smallest meaningful optimization scope.
95 
96| Evaluator signal | Prefer these knobs first |
97|------------------|--------------------------|
98| `relevance` | final response instructions, answer synthesis prompt, model choice |
99| `task_adherence` | primary task instructions, specialist instructions, response constraints |
100| `intent_resolution` | router/orchestrator prompt, classifier prompt, planner prompt, handoff descriptions |
101| `builtin.tool_call_accuracy` | tool-calling agent instructions, tool descriptions, argument schema descriptions, tool-choice/planner settings, low-temperature planning behavior |
102| `indirect_attack` | safety instructions, instruction hierarchy, tool input handling, retrieved/tool-content treatment rules |
103| groundedness/citation quality | retrieval instructions, answer synthesis prompt, citation formatting, retrieval parameters when exposed safely |
104| latency/cost | model selection, max tokens, number of agent hops, tool/retrieval limits |
105 
106If evaluators point to different subsystems, prefer a targeted set of named config hooks over one global config. Flag any knob whose change would affect all agents, such as a shared model client.
107 
108### Step 7: Present Proposed FAOS Targets
109 
110Before editing, summarize:
111 
112- Selected agent root
113- Python entrypoint(s)
114- Detected topology
115- Known evaluator objectives
116- Proposed FAOS targets and why
117- Knobs that will remain unchanged
118- Files that will be modified or added
119 
120If there is exactly one safe target, proceed unless the user asked for an approval checkpoint. If there are multiple plausible targets, ask the user which scope to optimize before editing.
121 
122Example review summary:
123 
124```text
125Detected evaluator targets:
126- builtin.tool_call_accuracy
127- intent_resolution
128 
129Detected topology:
130- router_agent routes user requests
131- weather_agent owns get_weather tool
132- final_answer_agent synthesizes output
133 
134Proposed FAOS targets:
135- router_agent instructions: improves intent resolution
136- weather_agent instructions/tool schema: improves tool-call accuracy
137- preserve final_answer_agent for now
138```
139 
140### Step 8: Apply the Python FAOS Config Contract
141 
142Use the generic Python contract from [Python Patterns](references/python-patterns.md). At minimum, add or reuse:
143 
144```python
145import os
146 
147from agent_optimization import load_config
148 
149SYSTEM_PROMPT = """...existing default instructions..."""
150EXISTING_MODEL_FALLBACK = os.getenv("<existing-model-env-var>", "gpt-4.1")
151 
152config = load_config(
153    default_instructions=SYSTEM_PROMPT,
154    default_model=EXISTING_MODEL_FALLBACK,
155    default_skills_dir="skills",
156)
157```
158 
159Then map the selected target knobs:
160 
161- Existing default instructions -> `config.compose_instructions()`
162- Existing model default -> `config.model or <existing fallback>`. Reuse the app's current model-selection environment variable(s) and fallback chain instead of hard-coding `MODEL_DEPLOYMENT_NAME` unless that is already what the app uses.
163- Existing temperature/default options -> `config.temperature` only when the runtime supports it
164- Skills directory -> `config.skills_dir` only when the runtime has a skill/tool loading mechanism or one is explicitly added
165 
166For multi-agent code, prefer named config variables such as `orchestrator_config`, `tool_agent_config`, or `synthesizer_config` over a misleading global `config` when more than one agent can be optimized.
167 
168### Step 9: Add or Reuse `agent_optimization`
169 
170If the agent already has an `agent_optimization` package, reuse it and avoid overwriting user changes.
171 
172If missing, add a minimal local package with:
173 
174- `load_config(...)`
175- `OptimizationConfig`
176- `Skill`
177- environment-variable fallback support for `AGENT_OPTIMIZATION_CONFIG` and `OPTIMIZATION_CONFIG`
178- optional candidate resolver support for `AGENT_OPTIMIZATION_CANDIDATE_ID` and `AGENT_OPTIMIZATION_RESOLVE_ENDPOINT`
179 
180Do not assume a public PyPI package exists. Keep the local package self-contained unless the repository already uses a shared internal package.
181 
182### Step 10: Update Dependencies and Runtime Config
183 
184Update Python dependency files only as needed:
185 
186- Add `python-dotenv` if the code imports it or already uses `.env` files
187- Add `azure-identity` only if resolver token support is included or already imported
188 
189Use `load_dotenv(override=False)` so Foundry runtime environment variables win over local `.env` values.
190 
191Do not automatically add optimization env vars to `agent.yaml`. If the user wants env var placeholders, add only the required ones for their workflow and keep optional optimization vars documented rather than injected by default.
192 
193### Step 11: Verify
194 
195Run these checks where possible:
196 
1971. Python syntax check for changed files
1982. Import smoke test for `agent_optimization.load_config`
1993. Default config smoke test with no optimization env vars
2004. Pylance or workspace diagnostics for changed files
2015. Existing project tests if they are cheap and relevant
202 
203If Azure credentials or model endpoints are missing, do not treat live invocation failures as conversion failures. The required proof is that defaults load and the original runtime can still start or import as far as local configuration allows.
204 
205### Step 12: Stop for Review, Then Suggest Deploy
206 
207End the workflow with a review checkpoint. Summarize:
208 
209- Changed files
210- FAOS knobs exposed
211- Evaluator objectives considered
212- Any global side effects, such as shared model clients
213- Verification results
214 
215Ask the user to review the diff. Do not deploy automatically.
216 
217When the user approves deployment, route to [deploy](../deploy/deploy.md), then [invoke](../invoke/invoke.md). If the user wants to evaluate the deployed version, route to [observe](../observe/observe.md).
218 
219## Guardrails
220 
221- Python only for now.
222- The config contract is framework-neutral; insertion points are runtime-specific.
223- Preserve existing frameworks, tools, hosting adapters, protocols, and entrypoints.
224- Do not use one global config across all agents in a multi-agent system unless the existing architecture already uses one global prompt/model and the user approves.
225- Do not wire temperature where unsupported or semantically risky.
226- Prefer low-temperature planning/tool-calling defaults unless an evaluator objective suggests otherwise.
227- Treat evaluator context as a targeting signal, not proof that every related knob should be changed.
228- Keep all edits scoped to the selected agent root.
229- Stop for review before deployment.
230