1# Foundry Agent Deploy
2 
3Create and manage agent deployments in Azure AI Foundry. For hosted agents, this includes the full workflow from containerizing the project to verifying the deployed agent.
4 
5## Quick Reference
6 
7| Property | Value |
8|----------|-------|
9| Agent types | Prompt (LLM-based), Hosted |
10| MCP server | `azure` |
11| Key Foundry MCP tools | `agent_definition_schema_get`, `agent_update`, `agent_get` |
12| CLI tools | `docker`, `az acr` (hosted agents only) |
13| Container protocols | `a2a`, `responses`, `invocations`, `invocations_ws`, `mcp` |
14| Supported languages | .NET, Node.js, Python, Go, Java |
15 
16## When to Use This Skill
17 
18USE FOR: deploy agent to foundry, push agent to foundry, ship my agent, build and deploy container agent, deploy hosted agent, direct code deployment, upload code deployment, create hosted agent, deploy prompt agent, ACR build, container image for agent, docker build for foundry, redeploy agent, update agent deployment, clone agent, delete agent, azd deploy hosted agent, azd ai agent, azd up for agent, deploy agent with azd.
19 
20> ⚠️ **DO NOT manually run** `azd up`, `azd deploy`, `az acr build`, `docker build`, `agent_update`, or direct-code REST upload commands **without reading this skill first.** This skill orchestrates the full deployment pipeline: project scan → env var collection → deployment method selection → Dockerfile/image build or direct-code metadata upload → agent creation/version update → verification. Running CLI commands or calling MCP tools individually skips critical steps (env var confirmation, schema or REST metadata validation, RBAC setup, invocation verification).
21 
22## MCP Tools
23 
24| Tool | Description | Parameters |
25|------|-------------|------------|
26| `agent_definition_schema_get` | Get JSON schema for agent definitions | `projectEndpoint` (required), `schemaType` (`prompt`, `hosted`, `tools`, `all`) |
27| `agent_update` | Create, update, or clone an agent | `projectEndpoint`, `agentName` (required); `agentDefinition` (JSON), `isCloneRequest`, `cloneTargetAgentName`, `modelName` |
28| `agent_get` | List all agents or get a specific agent | `projectEndpoint` (required), `agentName` (optional) |
29| `agent_delete` | Delete an agent and clean up hosted-agent runtime resources | `projectEndpoint`, `agentName` (required) |
30 
31## Deployment Method Selection
32 
33Direct code deployment is opt-in only.
34 
35- Prompt agents use [Workflow: Prompt Agent Deployment](#workflow-prompt-agent-deployment).
36- Hosted agents use [Workflow: Hosted Agent Deployment](#workflow-hosted-agent-deployment); select the hosted deployment method in Step 3.
37- Do not infer direct code deployment just because Docker is unavailable or a Dockerfile is missing. Ask or use the default Docker/ACR workflow guidance.
38 
39If the user explicitly says `using direct code deployment`, `direct-code deployment`, `upload code deployment`, or otherwise clearly asks to deploy by uploading source code, Step 3 reads [Direct Code Deployment Reference](references/direct-code-deployment.md), deploys the agent directly, then proceeds directly to [Step 7: Test the Agent](#step-7-test-the-agent).
40 
41## Workflow: Hosted Agent Deployment
42 
43> ⚠️ **Warning: hosted agent deployment has 8 steps, not 7.**
44>
45> The single most common failure of this skill is stopping after Step 7 (invocation smoke test) and emitting a "deployment complete" summary. **Step 8 (auto-generate evaluation suite) is mandatory and runs automatically after every deploy — including redeploys, version bumps, and `azd deploy` re-runs.**
46>
47> Before you write any final summary, Playground link, version table, or deployment success message, you MUST self-verify:
48>
49> 1. Did Step 8 run to completion (suite generated **or** documented fallback persisted)?
50> 2. Is deployment context resolvable from azd or metadata, and was `.foundry` updated only with non-derivable overlay/cache state?
51> 3. Did you prompt the user to run an evaluation?
52>
53> If the answer to any of these is **no**, do not summarize — go run Step 8 now.
54 
55> ⚠️ **`azd deploy` ≠ deployment complete.** `azd deploy` (or any `azd up`/`az acr build`/`agent_update` shortcut) only covers Steps 1–6. You **MUST** still execute Step 7 (invocation test) and Step 8 (auto-generate evaluation suite) before reporting success to the user. A successful `azd deploy` exit code is **not** a stopping condition. A successful invocation in Step 7 is **not** a stopping condition either.
56 
57### Definition of Done — Hosted Agent Deployment
58 
59A hosted-agent deployment is complete only when **every** box below is checked. Do **not** produce a final "deployment successful" summary, table, or Playground link until all items are done. If you skip any item, your response is incomplete.
60 
61For direct-code deployments, Step 3 runs the direct-code reference and deploys the agent directly, then proceeds directly to Step 7.
62 
63- [ ] Step 1 — Project scanned, type detected
64- [ ] Step 2 — Environment variables confirmed with user
65- [ ] Step 3 — Deployment method selected and prepared
66- [ ] Step 4 — Agent configuration collected
67- [ ] Step 5 — Agent definition schema retrieved
68- [ ] Step 6 — `agent_update` called successfully
69- [ ] Step 7 — RBAC checked **and** invocation smoke test passed (via the invoke skill)
70- [ ] Step 8 — Auto-generated evaluation suite job reached `succeeded` (or documented fallback)
71- [ ] Step 8 — Cache files written: `.foundry/suites/<suite>-v<ver>.json`, `.foundry/evaluators/<eval>-v<ver>.json` (FULL definition, not stub), `.foundry/datasets/<agent>-<dataset>-v<ver>.ref.json`, AND `.foundry/datasets/<dataset>-v<ver>/<blob>` (actual dataset rows via SAS-url download)
72- [ ] Deployment context is resolvable from azd or metadata; `.foundry/agent-metadata*.yaml` contains only non-derivable overlay/cache state for the selected environment
73- [ ] User prompted to run an evaluation
74 
75### Step 1: Detect and Scan Project
76 
77Get the project path from the selected agent root in the project context (see [Common Project Context Resolution](../../SKILL.md#agent-common-project-context-resolution)). Detect the project type by checking for these files. Do **not** scan sibling agent folders.
78 
79| Project Type | Detection Files |
80|--------------|-----------------|
81| .NET | `*.csproj`, `*.fsproj` |
82| Node.js | `package.json` |
83| Python | `requirements.txt`, `pyproject.toml`, `setup.py` |
84| Go | `go.mod` |
85| Java (Maven) | `pom.xml` |
86| Java (Gradle) | `build.gradle` |
87 
88Delegate an environment variable scan to a sub-agent. Provide the selected agent root path and project type. Search source files inside that folder only for these patterns:
89 
90| Project Type | Patterns to Search |
91|--------------|--------------------|
92| .NET (`*.cs`) | `Environment.GetEnvironmentVariable("...")`, `configuration["..."]`, `configuration.GetValue<T>("...")` |
93| Node.js (`*.js`, `*.ts`, `*.mjs`) | `process.env.VAR_NAME`, `process.env["..."]` |
94| Python (`*.py`) | `os.environ["..."]`, `os.environ.get("...")`, `os.getenv("...")` |
95| Go (`*.go`) | `os.Getenv("...")`, `os.LookupEnv("...")` |
96| Java (`*.java`) | `System.getenv("...")`, `@Value("${...}")` |
97 
98Classification: if followed by a throw/error → required; if followed by a fallback value → optional with default; otherwise → assume required, ask user.
99 
100### Step 2: Collect and Confirm Environment Variables
101 
102> ⚠️ **Warning:** Environment variables are included in the agent payload and are difficult to change after deployment.
103 
104Use azd environment values from the project context to pre-fill discovered variables. Merge with any user-provided values. Present all variables to the user for confirmation with variable name, value, and source (`azd`, `project default`, or `user`). Mask sensitive values.
105 
106Loop until the user confirms or cancels:
107- `yes` → Proceed
108- `VAR_NAME=new_value` → Update the value, show updated table, ask again
109- `cancel` → Abort deployment
110 
111### Step 3: Select Deployment Method and Prepare
112 
113If the user explicitly requested direct code deployment or upload code deployment, do not generate a Dockerfile or build an image. Read and follow [Direct Code Deployment Reference](references/direct-code-deployment.md), deploy the agent directly, then proceed directly to [Step 7: Test the Agent](#step-7-test-the-agent).
114 
115For all other hosted-agent deployments, continue with the Docker/ACR preparation below.
116 
117#### Image built and pushed to ACR
118 
119Delegate Dockerfile creation to a sub-agent. Guidelines:
120- Use official base image for the detected language and runtime version
121- Use multi-stage builds for compiled languages
122- Use Alpine or slim variants for smaller images
123- Always target `linux/amd64` platform
124- Expose the correct port (usually 8088)
125 
126> 💡 **Tip:** Reference [Hosted Agents Foundry Samples](https://github.com/microsoft-foundry/foundry-samples/tree/main/samples/python/hosted-agents) for containerized agent examples.
127 
128Also generate `docker-compose.yml` and `.env` files for local development.
129 
130**IMPORTANT**: You MUST always generate image tag as current timestamp (e.g., `myagent:202401011230`) to ensure uniqueness and avoid conflicts with existing images in ACR. DO NOT use static tags like `latest` or `v1`.
131 
132Collect ACR details from project context.
133 
134- If an ACR already exists, use it, then verify that the Foundry project managed identity has pull permissions (for example, `Container Registry Repository Reader` or equivalent) on the target repository/registry. If the role assignment is missing, add it.
135- If no ACR exists, create a new one with ABAC repository permissions mode, and assign `Container Registry Repository Reader` to the Foundry project managed identity. Foundry hosted agents use ABAC mode that requires repository-scoped roles, not the registry-level `AcrPull` role.
136 
137Let the user choose the build method:
138 
139**Cloud Build (ACR Tasks) (Recommended)** — no local Docker required:
140```bash
141az acr build --registry <acr-name> --image <repository>:<tag> --platform linux/amd64 --source-acr-auth-id "[caller]" --file Dockerfile .
142```
143 
144> ⚠️ **Mandatory:** The `--source-acr-auth-id "[caller]"` parameter is required. Do NOT omit it — without this flag the build will fail due to missing authentication context.
145 
146**Local Docker Build:**
147```bash
148docker build --platform linux/amd64 -t <image>:<tag> -f Dockerfile .
149az acr login --name <acr-name>
150docker tag <image>:<tag> <acr-name>.azurecr.io/<repository>:<tag>
151docker push <acr-name>.azurecr.io/<repository>:<tag>
152```
153 
154> 💡 **Tip:** Prefer Cloud Build if Docker is not available locally. On Windows with WSL, prefix Docker commands with `wsl -e` if `docker info` fails but `wsl -e docker info` succeeds.
155 
156### Step 4: Collect Agent Configuration
157 
158Use the project endpoint and ACR name from the project context. Ask the user only for values not already resolved:
159- **Agent name** — Unique name for the agent
160- **Model deployment** — Model deployment name (e.g., `gpt-4o`)
161 
162### Step 5: Get Agent Definition Schema
163 
164Use `agent_definition_schema_get` with `schemaType: hosted` to retrieve the current schema and validate required fields.
165 
166### Step 6: Create the Agent
167 
168Use `agent_update` with the agent definition:
169 
170> ⚠️ **Protocol version source of truth:** Do NOT copy the protocol version from `agent_definition_schema_get` examples. Use the protocol version declared by the agent source itself (for example, `agent.yaml` or `agent.manifest.yaml`).
171 
172```json
173{
174  "command": "agent_update",
175  "intent": "Update a hosted agent with a new docker image",
176  "parameters": {
177    "projectEndpoint": "<project-endpoint>",
178    "agentName": "<agent-name>",
179    "agentDefinition": {
180      "kind": "hosted",
181      "image": "<acr-name>.azurecr.io/<repository>:<tag>",
182      "cpu": "<cpu-cores>",
183      "memory": "<memory>",
184      "container_protocol_versions": [
185        { "protocol": "<protocol>", "version": "<version>" }
186      ],
187      "environment_variables": { "<var>": "<value>" }
188    }
189  }
190}
191```
192 
193Capture the per-agent identity from the agent creation response, then retrieve the project-level agent identity from the project resource after creation. You will need both identities to assign the minimum RBAC required for invocation before running invoke tests.
194 
195### Step 7: Test the Agent
196 
197For a newly deployed hosted agent, before invocation testing, first check whether the per-agent identity and project-level agent identity already have the minimum RBAC required for invocation.
198 
199Required role assignment:
200- `Foundry User`
201 
202Required scope: the Cognitive Services account, not the project.
203 
204Check existing assignments before creating any new assignment. If the required role assignment is missing for either identity, assign it before invocation testing.
205 
206If the current user account does not have permission to create a missing role assignment, stop the deployment workflow here. Explain to the user that hosted-agent invocation requires `Foundry User` on the per-agent identity and project-level agent identity at the Cognitive Services account scope, and the deployment cannot be treated as complete until someone with RBAC assignment permission grants the missing role.
207 
208After this RBAC check is complete, read and follow the [invoke skill](../invoke/invoke.md) to send a test message and verify the agent responds correctly. DO NOT SKIP reading the invoke skill — it contains important information about required hosted-agent session handling.
209 
210If invocation testing still fails after this RBAC check, immediately read and follow the [troubleshoot skill](../troubleshoot/troubleshoot.md). Do not treat the deployment as fully successful until invocation succeeds.
211 
212> ⚠️ **Not done yet: invocation success is the midpoint, not the finish line.** The next action after a passing smoke test is **Step 8**, not a deployment summary. Do not write a summary, version table, or Playground link yet.
213 
214### Step 8: Auto-Generate Evaluation Suite (MANDATORY — RUNS AUTOMATICALLY)
215 
216> ⚠️ **Pre-summary gate.** If you are about to write a deployment summary, Playground link, or "deployment complete" message and Step 8 has not run, you are violating this skill. Run Step 8 first.
217>
218> This step **runs automatically** without waiting for the user to ask. The only user input required is the one-question prompt below in 8a.
219 
220This step is mandatory — not optional — for every hosted-agent deployment, including redeploys, version bumps, and `azd deploy` re-runs against an already-existing agent. In azd projects, resolve deployment context from `azd env get-values` and treat `.foundry/agent-metadata*.yaml` as an overlay/cache instead of copying azd-owned values into it.
221 
222**8a. Ask the user (one question, required).** Before generating, inspect the selected agent root for `eval.yaml`, then ask the user to pick a setup source. Recommend local `eval.yaml` when it exists and matches the selected agent; otherwise recommend traces when the agent has recent traces, or current agent code/definition:
223 
224> *"Your agent is deployed. I'll now auto-generate an evaluation suite. Which source should I use?*
225> *(a) **Current agent code/definition** — synthetic Q&A from `agent.yaml` / instructions. Best when there's little or no trace history.*
226> *(b) **Historical traces** — last 3 days, ~50 traces. Best if the agent has recent invocations.*
227> *(c) **Existing eval.yaml** — local dataset/evaluator intent from the selected agent folder. Best when azd eval config already exists."*
228 
229**8b. Follow the full procedure.** Read and follow [After Deployment — Auto-Generate Evaluation Suite](#after-deployment--auto-generate-evaluation-suite) below for the generation, polling, persistence, and metadata-update steps. Required parameters and poll-to-terminal rules are non-negotiable.
230 
231**8c. Cache artifacts locally (MANDATORY after `succeeded`).** Once the suite-generation job is `succeeded`, perform the required cache calls described in [Evaluation Suite Generation → Cache Artifacts Locally](../observe/references/evaluation-suite-generation.md#cache-artifacts-locally):
232 
233- `evaluation_suite_get` → `.foundry/suites/<suite>-v<ver>.json` (full object)
234- `evaluator_catalog_get` → `.foundry/evaluators/<eval>-v<ver>.json` (full definition, NOT a stub)
235- `evaluation_dataset_get` + `evaluation_dataset_sas_url_get` → `.foundry/datasets/<agent>-<dataset>-v<ver>.ref.json` (metadata stub) AND `.foundry/datasets/<dataset>-v<ver>/<blob>` (actual JSONL rows). The SAS-url tool returns a container-scope SAS — list the container then `curl.exe` each blob. See the reference for the exact list+download steps. Set `contentDownloaded: true` in the stub once files are on disk.
236 
237Do not write the deployment summary until all cache files exist.
238 
239**8d. Skip-only-on-explicit-request.** If — and only if — the user explicitly says "skip eval suite generation," record that decision in your summary and still ensure deployment context remains resolvable from azd or metadata. "The user didn't ask for it" is **not** a valid reason to skip; this step is opt-out, not opt-in.
240 
241## Workflow: Prompt Agent Deployment
242 
243### Definition of Done — Prompt Agent Deployment
244 
245A prompt-agent deployment is complete only when **every** box below is checked. Do **not** produce a final "deployment successful" summary, table, or Playground link until all items are done.
246 
247- [ ] Step 1 — Agent configuration collected
248- [ ] Step 2 — Agent definition schema retrieved
249- [ ] Step 3 — `agent_update` called successfully
250- [ ] Step 4 — Invocation smoke test passed (via the invoke skill)
251- [ ] Step 5 — Auto-generated evaluation suite job reached `succeeded` (or documented fallback)
252- [ ] Step 5 — Cache files written: `.foundry/suites/<suite>-v<ver>.json`, `.foundry/evaluators/<eval>-v<ver>.json` (FULL definition, not stub), `.foundry/datasets/<agent>-<dataset>-v<ver>.ref.json`, AND `.foundry/datasets/<dataset>-v<ver>/<blob>` (actual dataset rows via SAS-url download)
253- [ ] Deployment context is resolvable from azd or metadata; `.foundry/agent-metadata*.yaml` contains only non-derivable overlay/cache state for the selected environment
254- [ ] User prompted to run an evaluation
255 
256### Step 1: Collect Agent Configuration
257 
258Use the project endpoint from the project context (see [Common Project Context Resolution](../../SKILL.md#agent-common-project-context-resolution)). Ask the user only for values not already resolved:
259- **Agent name** — Unique name for the agent
260- **Model deployment** — Model deployment name (e.g., `gpt-4o`)
261- **Instructions** — System prompt (optional)
262- **Temperature** — Response randomness 0-2 (optional, default varies by model)
263- **Tools** — Tool configurations (optional)
264 
265### Step 2: Get Agent Definition Schema
266 
267Use `agent_definition_schema_get` with `schemaType: prompt` to retrieve the current schema.
268 
269### Step 3: Create the Agent
270 
271Use `agent_update` with the agent definition:
272 
273```json
274{
275  "kind": "prompt",
276  "model": "<model-deployment>",
277  "instructions": "<system-prompt>",
278  "temperature": 0.7
279}
280```
281 
282### Step 4: Test the Agent
283 
284Read and follow the [invoke skill](../invoke/invoke.md) to send a test message and verify the agent responds correctly.
285 
286> ⚠️ **Not done yet: invocation success is the midpoint, not the finish line.** The next action is **Step 5**, not a deployment summary. Do not write a summary or Playground link yet.
287 
288### Step 5: Auto-Generate Evaluation Suite (MANDATORY — RUNS AUTOMATICALLY)
289 
290> ⚠️ **Pre-summary gate.** If you are about to write a deployment summary or Playground link and Step 5 has not run, you are violating this skill. Run Step 5 first.
291>
292> This step **runs automatically** without waiting for the user to ask. The only user input required is the one-question prompt below.
293 
294**5a. Ask the user (one question, required).** Before generating, inspect the selected agent root for `eval.yaml`, then ask which setup source to use. Recommend local `eval.yaml` when it exists and matches the selected agent; otherwise recommend recent traces when present, or current agent code/definition:
295 
296> *"Your agent is deployed. I'll now auto-generate an evaluation suite. Which source should I use? (a) Current agent code/definition (synthetic Q&A), (b) Historical traces (last 3 days, ~50 traces), or (c) Existing eval.yaml from the selected agent folder?"*
297 
298**5b. Follow the full procedure.** Read and follow [After Deployment — Auto-Generate Evaluation Suite](#after-deployment--auto-generate-evaluation-suite) below.
299 
300**5c. Cache artifacts locally (MANDATORY after `succeeded`).** Once the suite-generation job is `succeeded`, perform the required cache calls described in [Evaluation Suite Generation → Cache Artifacts Locally](../observe/references/evaluation-suite-generation.md#cache-artifacts-locally): suite JSON, evaluator full definition, dataset `.ref.json` PLUS the actual dataset blobs downloaded via `evaluation_dataset_sas_url_get` (container SAS → list → curl each blob). Do not write the deployment summary until those files exist.
301 
302**5d. Skip-only-on-explicit-request.** Skip only if the user explicitly says "skip eval suite generation." Keep deployment context resolvable from azd or metadata. "The user didn't ask for it" is **not** a valid reason to skip.
303 
304## Display Agent Information
305 
306> ⚠️ **Gate:** Do not render the table or Playground link until the Definition of Done checklist for the selected workflow (Hosted or Prompt) is fully satisfied, including the invocation smoke test, the auto-generated evaluation suite (or documented skip), and resolvable deployment context plus `.foundry` overlay/cache updates. The Playground link is the final artifact, not a mid-workflow checkpoint.
307 
308Once deployment is done for either hosted or prompt agent, display the agent's details in a nicely formatted table.
309 
310Below the table you MUST also display a Playground link for direct access to the agent in Azure AI Foundry:
311 
312[Open in Playground](https://ai.azure.com/nextgen/r/{encodedSubId},{resourceGroup},,{accountName},{projectName}/build/agents/{agentName}/build?version={agentVersion})
313 
314To calculate the encodedSubId, you need to take subscription id and convert it into its 16-byte GUID, then encode it as URL-safe base64 without padding (= characters trimmed). You can use the following Python code to do this conversion:
315 
316```
317python -c "import base64,uuid;print(base64.urlsafe_b64encode(uuid.UUID('<SUBSCRIPTION_ID>').bytes).rstrip(b'=').decode())"
318```
319 
320## Document Deployment Context
321 
322After a successful deployment, make the deployment context reusable without duplicating azd-owned values. If `azure.yaml` and `azd env get-values` provide the project endpoint, agent name/version, ACR, App Insights, subscription, resource group, and project identifiers, treat azd as the source of truth. Persist only non-derivable overlay/cache state to the selected metadata file under `<agent-root>/.foundry/`.
323 
324| State | Preferred Source | Metadata Behavior |
325|-------|------------------|-------------------|
326| Project endpoint, agent name/version, ACR, observability | azd env values | Do not copy when azd provides them |
327| azd binding | selected azd env/service | Store `azd.environmentName` and `azd.service` when useful |
328| Evaluation suites | Foundry lookup/generation or verified `eval.yaml` sync | Persist in `evaluationSuites[]` |
329| Local cache paths and results | `.foundry/` cache/results | Persist suite, dataset, evaluator, and result refs |
330 
331If metadata and azd disagree for the same deployment value, stop and ask which source is authoritative. If the selected metadata file is a preferred single-environment file, update only that one environment block and leave sibling metadata files untouched. If the selected metadata file is a legacy multi-environment file, merge the selected environment instead of overwriting other environments or cached evaluation suites without confirmation. If the selected environment still uses older `testSuites[]` or legacy `testCases[]`, rewrite that environment to `evaluationSuites[]` when you persist synced suite metadata.
332 
333## After Deployment — Auto-Generate Evaluation Suite
334 
335> ⚠️ **This step is automatic.** After a successful deployment, immediately prepare the selected `.foundry` environment for evaluation without waiting for the user to request it. This matches the eval-driven optimization loop.
336 
337### 1. Resolve Effective Context
338 
339Use the [Common Project Context Resolution](../../SKILL.md#agent-common-project-context-resolution) flow. In azd projects, resolve project endpoint, agent name/version, ACR, and observability from `azd env get-values`; use `.foundry/agent-metadata*.yaml` only for overlays and synced suite/cache refs. If `eval.yaml` exists in the selected agent root, parse it as local evaluation intent before generating anything new.
340 
341### 2. Read Agent Instructions
342 
343Use **`agent_get`** (or local `agent.yaml`) to understand the agent's purpose and capabilities.
344 
345### 3. Reuse or Refresh Suite Cache
346 
347Inspect the selected agent root before generating anything new:
348 
349- Reuse a selected environment `evaluationSuites[]` entry when it has `suiteName`, `suiteVersion`, matching `.foundry/datasets/`, and matching `.foundry/evaluators/` cache files.
350- When `eval.yaml` exists and matches the selected agent, prefer verifying/registering its dataset and evaluator references before creating a brand-new generated suite.
351- Call `evaluation_suite_get` to confirm the remote suite still exists before reusing it.
352- Ask before refreshing cached files, replacing thresholds, or writing a new suite version.
353- If cache or the remote suite is missing/stale, generate a new suite and update metadata for the active environment only.
354 
355### 4. Identify Generation Deployment
356 
357Use **`model_deployment_get`** to list the selected project's actual model deployments, then choose one that supports chat completions for quality evaluators. Do **not** assume `gpt-4o` exists in the project. If no deployment supports chat completions, stop the auto-setup flow and tell the user quality evaluators cannot run until a compatible judge deployment is available.
358 
359### 5. Generate Evaluation Suite
360 
361Read and follow [Evaluation Suite Generation](../observe/references/evaluation-suite-generation.md) for source selection, required parameters, polling, and cache writes. In the deploy flow, keep these guardrails:
362 
363- Ask the user which setup source to use before calling `evaluation_suite_generation_job_create`; recommend matching `eval.yaml` when present, then recent traces when available, otherwise the current agent code/definition.
364- Use the chat-capable generation deployment selected above and honor the reference's service constraints, especially `maxSamples` (15-1000) and `agentSourceNames: [<agentName>]` for agent-sourced suites.
365- Do not report deployment complete while the generation job is `in_progress`; poll with `evaluation_suite_generation_job_get` until `succeeded`, `failed`, or `canceled`, then inspect the suite with `evaluation_suite_get` and cache artifacts as described in the reference.
366 
367### 6. Fallback to Manual Suggestions
368 
369If `evaluation_suite_generation_job_create`, `evaluation_suite_generation_job_get`, or `evaluation_suite_get` fails, is unavailable, or returns incomplete artifacts, fall back to the previous manual flow:
370 
3711. Call `evaluator_catalog_get` and suggest relevant built-in/custom evaluators.
3722. Read [Generate Seed Evaluation Dataset](../eval-datasets/references/generate-seed-dataset.md), generate valid local JSONL with `query` and `expected_behavior`, and register it with `evaluation_dataset_create`.
3733. Persist the suite with `generationSource: manual-fallback` and include the fallback reason in the workflow summary.
374 
375Do **not** silently ignore generation failures; the user should know whether setup used the generated-suite path or the fallback path.
376 
377The local filename must start with the effective selected Foundry agent name before adding stage, environment, or version suffixes.
378 
379### 7. Persist Artifacts and Evaluation Suites
380 
381Save generated or fallback evaluator definitions, local datasets, and evaluation outputs under `.foundry/` using the cache paths defined in [Evaluation Suite Generation](../observe/references/evaluation-suite-generation.md), then register or update evaluation suites in the selected metadata file for the selected environment:
382 
383```text
384.foundry/
385  agent-metadata.yaml
386  agent-metadata.prod.yaml
387  suites/
388    <suite-name>-v<version>.json
389  evaluators/
390    <evaluator-name>-v<version>.json
391  datasets/
392    <agent-name>-<dataset-name>-v<version>.ref.json
393    <dataset-name>-v<version>/<blob>
394  results/
395```
396 
397Each evaluation suite should bundle the remote suite reference, local cache paths, thresholds, and a `tags` map (for example, `tier: smoke`, `purpose: baseline`, `stage: generated`). Persist `suiteName`, `suiteVersion`, `generationJobId`, `generationSource`, `datasetFile`, and `datasetUri` together. Do not persist azd-owned deployment fields when azd resolves them. If the selected environment still uses older `testSuites[]` or legacy `testCases[]`, replace that list with `evaluationSuites[]` in the rewritten metadata and map legacy `priority` to `tags.tier` only when `tags.tier` is missing.
398 
399### 8. Prompt User
400 
401*"Your agent is deployed and running in the selected environment. The `.foundry` cache now contains generated evaluation-suite metadata, local dataset/evaluator references, and remote Foundry suite references. Would you like to run an evaluation to identify optimization opportunities?"*
402 
403- **Yes** → follow the [observe skill](../observe/observe.md) starting at **Step 2 (Evaluate)** — cache and metadata are already prepared.
404- **No** → stop. The user can return later.
405- **Production trace analysis** → follow the [trace skill](../trace/trace.md) to search conversations, diagnose failures, and analyze latency using App Insights.
406 
407## Agent Definition Schemas
408 
409### Prompt Agent
410 
411| Property | Type | Required | Description |
412|----------|------|----------|-------------|
413| `kind` | string | ✅ | Must be `"prompt"` |
414| `model` | string | ✅ | Model deployment name (e.g., `gpt-4o`) |
415| `instructions` | string | | System message for the model |
416| `temperature` | number | | Response randomness (0-2) |
417| `top_p` | number | | Nucleus sampling (0-1) |
418| `tools` | array | | Tools the model may call |
419| `tool_choice` | string/object | | Tool selection strategy |
420| `rai_config` | object | | Responsible AI configuration |
421 
422### Hosted Agent
423 
424| Property | Type | Required | Description |
425|----------|------|----------|-------------|
426| `kind` | string | ✅ | Must be `"hosted"` |
427| `image` | string | ✅ | Container image URL |
428| `cpu` | string | ✅ | CPU allocation (e.g., `"0.5"`, `"1"`, `"2"`) |
429| `memory` | string | ✅ | Memory allocation (e.g., `"1Gi"`, `"2Gi"`) |
430| `container_protocol_versions` | array | ✅ | Protocol and version pairs |
431| `environment_variables` | object | | Key-value pairs for container env vars |
432| `tools` | array | | Tool configurations |
433| `rai_config` | object | | Responsible AI configuration |
434 
435### Container Protocols
436 
437| Protocol | Description |
438|----------|-------------|
439| `a2a` | Agent-to-Agent protocol |
440| `responses` | OpenAI Responses API |
441| `invocations` | Invocation payload protocol for arbitrary request bodies and custom SSE behavior |
442| `invocations_ws` | Duplex WebSocket protocol for real-time / voice / signaling workloads (`WS /invocations_ws` on port 8088). Connect through `wss://...endpoint/protocols/invocations_ws?...&agent_session_id=...`. See the dedicated [invocations-ws skill](../invocations-ws/invocations-ws.md) for the full client/server contract. |
443| `mcp` | Model Context Protocol |
444 
445## Agent Management Operations
446 
447### Clone an Agent
448 
449Use `agent_update` with `isCloneRequest: true` and `cloneTargetAgentName` to create a copy. For prompt agents, optionally override the model with `modelName`.
450 
451### Delete an Agent
452 
453Use `agent_delete` — automatically cleans up hosted-agent runtime resources.
454 
455### List Agents
456 
457Use `agent_get` without `agentName` to list all agents, or with `agentName` to get a specific agent's details.
458 
459## Error Handling
460 
461| Error | Cause | Resolution |
462|-------|-------|------------|
463| Project type not detected | No known project files found | Ask user to specify project type manually |
464| Docker not running | Docker Desktop not started or not installed | Start Docker Desktop, or use Cloud Build (ACR Tasks) instead |
465| ACR login failed | Not authenticated to Azure | Run `az login` first, then `az acr login --name <acr-name>` |
466| Build/push failed | Dockerfile errors or insufficient ACR permissions | Check Dockerfile syntax, verify Contributor or AcrPush role on registry |
467| ACR build log crash | `UnicodeEncodeError` when `az acr build` streams remote logs | The remote build continues independently — do not assume failure. Get the `<run-id>` from the earlier `az acr build` output and check status with `az acr task show-run -r <acr-name> --run-id <run-id> --query status`. |
468| Agent creation failed | Invalid definition or missing required fields | Use `agent_definition_schema_get` to verify schema, check all required fields |
469| Hosted agent not running after creation | Provisioning failed or the image is not usable | Verify ACR image path, check cpu/memory values, confirm ACR permissions, then inspect hosted-agent logs with the troubleshoot skill |
470| Role assignment failed | The required invocation RBAC was not granted | Stop the deployment workflow and explain that hosted-agent invocation requires `Foundry User` on the per-agent identity and project-level agent identity at the Cognitive Services account scope |
471| Invocation test failed after deployment | Missing or incorrect invocation RBAC for the per-agent identity or project-level agent identity | Check whether `Foundry User` is assigned to the per-agent identity and project-level agent identity at the Cognitive Services account scope; assign missing role assignments, then retry invocation |
472| Permission denied | Insufficient Foundry project permissions | Verify Foundry Owner or Contributor role on the project |
473| Schema fetch failed | Invalid project endpoint | Verify project endpoint URL format: `https://<resource>.services.ai.azure.com/api/projects/<project>` |
474 
475## Non-Interactive / YOLO Mode
476 
477When running in non-interactive mode (e.g., `nonInteractive: true` or YOLO mode), the skill skips user confirmation prompts and uses sensible defaults:
478 
479- **Environment variables** — Uses values resolved from `azd env get-values` and project defaults without prompting for confirmation
480- **Agent name** — Must be provided in the initial user message or derived sensibly from the project context (`agent.yaml`, `agent.manifest.yaml`, folder name); if missing, the skill fails with an error instead of prompting
481- **Docker/ACR hosted-agent verification** — Automatically continues into RBAC and invocation verification without additional prompts once deployment succeeds
482- **Direct code deployment** — If explicitly requested, Step 3 reads the direct-code reference, deploys the agent directly, then proceeds directly to Step 7
483 
484> ⚠️ **Warning:** In non-interactive mode, ensure all required values (project endpoint, agent name, model deployment name, and ACR image for Docker/ACR deployments) are provided upfront in the user message, local `.env`, manifests, or available via `azd env get-values`. Missing values will cause the deployment to fail rather than prompt.
485 
486## Additional Resources
487 
488- [Foundry Hosted Agents](https://learn.microsoft.com/azure/ai-foundry/agents/concepts/hosted-agents?view=foundry)
489- [Foundry Agent Runtime Components](https://learn.microsoft.com/azure/ai-foundry/agents/concepts/runtime-components?view=foundry)
490- [Foundry Samples](https://github.com/microsoft-foundry/foundry-samples/)
491