1# Deploy a Foundry Agent
2 
3Provision Azure resources when needed, deploy the agent, and smoke-test it.
4 
5For **hosted agents** (custom container or code), use `azd deploy`. Prefer **direct code deployment through azd** (no Docker/ACR required): `agent.yaml` must contain `code_configuration:`, so `azd deploy` will use direct code deployment and zip the source and let Foundry build it. Use container/ACR deployment only when the agent truly needs a Dockerfile, custom system packages, or a pre-built image.
6 
7For **prompt agents** (LLM + instructions, no custom code), use the Foundry MCP `agent_update` tool.
8 
9## Quick Reference
10 
11| Property | Value |
12|----------|-------|
13| Hosted (recommended) | `azd provision` when needed, direct code deployment via `azd deploy` (`code_configuration` present), `azd ai agent invoke` |
14| Hosted (container) | `azd provision` when needed, container/ACR deployment via `azd deploy` (requires Docker/Podman + ACR, no `code_configuration:` in agent.yaml) |
15| Prompt MCP | `agent_definition_schema_get`, `agent_update`, `agent_get`, `agent_delete` |
16| Versioning | Each successful `azd deploy` creates an immutable agent version |
17| Endpoint-only patch | `azd ai agent endpoint update` (no new version) |
18| Local dev | [create-hosted](../create/create-hosted.md), [local-run](../create/references/local-run.md) |
19 
20## Hosted vs Prompt
21 
22- Shipping Python / .NET / Node code -> **Hosted** (azd workflow below).
23- Updating only model / instructions / tools -> **Prompt** (MCP workflow below).
24 
25## Deployment Method Selection -- Hosted agents
26 
27Before running `azd deploy`, inspect `<service-dir>/agent.yaml`.
28 
29| Agent YAML state | Deployment path |
30|------------------|-----------------|
31| `code_configuration:` present | **Direct code deploy** through `azd deploy`; no Docker/ACR build. |
32| No `code_configuration:` | **Container/ACR deploy** through `azd deploy`; builds/pushes an image or uses a pre-built `image:`. |
33 
34`code_configuration:` example in agent.yaml:
35 
36```yaml
37code_configuration:
38  runtime: python_3_13
39  entry_point: main.py
40  dependency_resolution: remote_build
41```
42 
43Default to direct code for standard hosted-agent code. If `azd deploy` prints `Packaging container` for an agent that does not need container-specific behavior, add or fix `code_configuration` and retry. Use the container path when the agent depends on Dockerfile behavior, system packages, or a pre-built image.
44 
45## Workflow -- Hosted agent (azd)
46 
47> Prerequisite: project scaffolded with `azd ai agent init`. If not, start at [create-hosted](../create/create-hosted.md).
48 
49### Step 1 -- Resolve azd environment
50 
51If the user provided an existing project endpoint, project ARM ID, or model deployment, set those values before deploy. Then verify the azd environment with `azd env get-values`.
52 
53```bash
54azd env set AZURE_AI_PROJECT_ENDPOINT "<project-endpoint>"
55azd env set AZURE_AI_PROJECT_ID "<project-arm-id>"
56azd env set AZURE_AI_MODEL_DEPLOYMENT_NAME "<model-deployment-name>"
57azd env get-values
58```
59 
60Run:
61 
62```bash
63azd ai project show --output json
64azd ai agent show --output json
65```
66 
67Branch on output: `not_deployed` -> Step 2. `active` / `deployed` -> redeploy (skip Step 2, go to Step 3). If `azd ai project show` fails with `missing_project_endpoint`, do Step 2 first -- `azd provision` will create the project.
68 
69> **Important:** Before deploy, also make sure `agent.yaml` and the azd environment are aligned with the user's provided configuration values.
70 
71### Step 2 -- Provision Azure resources (one-time per env)
72 
73Skip `azd provision` when the user gave you an existing `AZURE_AI_PROJECT_ENDPOINT` or `FOUNDRY_PROJECT_ENDPOINT` and the workflow only needs to deploy the agent into that project.
74 
75Run provision only for new projects or real infrastructure changes:
76 
77```bash
78azd provision --no-prompt
79```
80 
81> Optional: run `azd provision --preview --no-prompt` first to preview the resource changes (a what-if) before applying them.
82 
83What this does:
84 
85- Creates the Foundry project (if not present) and supporting resources under `infra/`.
86- Creates project connections declared in `azure.yaml services.<name>.config.connections[]`. `${PARAM_*}` placeholders resolve from the active azd env.
87- Wires model deployments, AI Search, ACR, etc. `infra/layers/` provision in parallel when present.
88 
89This is a core `azd` command. Skip provision when the user gave you an existing `AZURE_AI_PROJECT_ENDPOINT` via `azd env set` -- the extension uses the existing project as-is.
90 
91After provision completes for a new project, run `azd env get-values` and set missing required azd env values, especially `AZURE_AI_PROJECT_ID` and `AZURE_TENANT_ID`, before local run or the first `azd deploy`.
92 
93### Step 3 -- Deploy the agent
94 
95```bash
96azd deploy --no-prompt
97# Multi-service:
98azd deploy <service-name> --no-prompt
99```
100 
101What deploy does:
102 
103- Reads `<service-dir>/agent.yaml`, packages the agent, uploads it, and registers a new immutable version.
104- **Direct code deploy** (`code_configuration` present): zips source, excludes `.agentignore`, and lets Foundry build the runtime image.
105- **Container deploy** (no code configuration): builds the `Dockerfile`, pushes to the project's ACR, registers the version. When `agent.yaml` has `image:` set, `azd` reuses the pre-built image.
106 
107After deploy, azd writes `AGENT_<SVC>_NAME`, `AGENT_<SVC>_VERSION`, and `AGENT_<SVC>_<PROTO>_ENDPOINT` (one per protocol) into the active env.
108 
109Re-deploying an identical build still creates a new version; `azd` prints `Agent version <n> is already active.` and skips the poll.
110 
111If deploy reports `Done` for the service and then fails only in `postdeploy` with `Agent <service-name> with version <n> not found`, the service key and `agent.yaml name` were mismatched. Rename the `azure.yaml services` key to the deployed agent name and rerun `azd deploy --no-prompt`; do not switch deployment method.
112 
113### Step 4 -- Verify and invoke
114 
115```bash
116azd ai agent show --output json
117```
118 
119Expect `"status": "active"` (or `"deployed"`) and an `agent_endpoints` map. Smoke-test:
120 
121```bash
122azd ai agent invoke "hello, are you up?"
123```
124 
125> `azd ai agent invoke` is billed, so it prints a confirmation envelope on `--no-prompt`. Summarize `changes[]`, then run `confirmCommand` once consented.
126 
127Run one remote invocation only unless the user explicitly asked to test multi-turn/session behavior. A single successful response is enough for the deployment smoke test. Anything other than a completed/successful response -> run `azd ai agent doctor --output json`, then follow [troubleshoot](../troubleshoot/troubleshoot.md).
128 
129### Step 5: Auto-Generate Evaluation Suite (MANDATORY — RUNS AUTOMATICALLY)
130 
131> ⚠️ **Pre-summary gate.** If you are about to write a deployment summary or Playground link and Step 5 has not run, you are violating this skill. Run Step 5 first.
132 
133This step runs automatically after deploy. Ask the user which source to use and start it right after deploy succeeds — with `--no-wait`, `generate` returns in seconds and generation runs server-side, so it overlaps with invoke/test steps and finishes faster overall.
134 
135> *"Your agent is deployed. Want me to set up an evaluation suite now? (a) Yes — current agent instructions (synthetic Q&A), (b) Yes — historical traces (last 3 days), (c) Yes — use existing `eval.yaml`, (d) No / later."*
136 
137| Choice | Command | What's next |
138|---|---|---|
139| (a) Agent instructions | `azd ai agent eval generate --gen-instruction "<agent purpose>" --no-wait --no-prompt` — `--gen-instruction` is required (hosted agents don't auto-derive it); use `agent.yaml` `description:`. | Generation runs server-side. Tell the user: *"Suite submitted. Run `azd ai agent eval run` whenever you're ready — it'll finalize `eval.yaml` and execute the eval in one step."* |
140| (b) Historical traces | `azd ai agent eval generate --trace-days 3 --max-samples 50 --no-wait --no-prompt` | Same as (a). |
141| (c) Existing `eval.yaml` | Skip `generate`. | Tell the user: *"Using existing `eval.yaml`. Run `azd ai agent eval run` when ready."* |
142| (d) No / later | Skip. | Tell the user: *"You can run `azd ai agent eval generate` (and then `eval run`) anytime."* |
143 
144Other useful flags on `generate`: `--dataset <path-or-name>` to reuse an existing dataset instead of generating one, `--evaluator <name>` (repeatable) to pin built-in or custom evaluators, `--eval-model <name>` to choose the model used for generation and evaluation, `--reset-defaults` to overwrite an existing eval config, `--name <suite-name>` and `--out-file <path>` (default `eval.yaml`).
145 
146Then proceed to Step 6. See [After Deployment — Auto-Generate Evaluation Suite](#after-deployment--auto-generate-evaluation-suite) for run/refresh details.
147 
148### Step 6 -- Hand off
149 
150- Send more messages -> [invoke](../invoke/invoke.md)
151- Evaluate / optimize -> [observe](../observe/observe.md)
152- Diagnose failures -> [troubleshoot](../troubleshoot/troubleshoot.md)
153- Search traces / latency -> [trace](../trace/trace.md)
154 
155## `.agentignore`
156 
157`azd ai agent init` writes a default `<service-dir>/.agentignore` for code-deploy projects (gitignore syntax) that excludes tooling files, secrets, language artifacts, and Docker files from the deploy ZIP. Only the root file is read; use `!path` to force-include.
158 
159## Endpoint or card edits -- no new version
160 
161When only `agentEndpoint:` or `agentCard:` changed in `agent.yaml`:
162 
163```bash
164azd ai agent endpoint update --dry-run # preview
165azd ai agent endpoint update --force # apply
166```
167 
168Idempotent.
169 
170## Multi-environment deploys
171 
172```bash
173azd env list
174azd env select prod
175azd deploy --no-prompt
176```
177 
178Each env has its own `AGENT_<SVC>_*` vars.
179 
180## Common failure modes -- Hosted
181 
182| Error | Fix |
183|-------|-----|
184| `missing_project_endpoint` | Run `azd env set AZURE_AI_PROJECT_ENDPOINT <url>`, or run `azd provision` for a new project. |
185| `invalid_agent_manifest` | `azd ai agent doctor`; fix the named field. |
186| `invalid_connection` | Inspect with `azd ai agent connection show <name>`. |
187| Docker daemon not running | You are on the container path. Add/fix `code_configuration` and retry direct code deploy. Only install Docker or try remote image build if you specifically need container deploy. |
188| ACR push 403 | Foundry project RBAC is missing `AcrPush` for your identity. Consider switching to direct code deployment to avoid ACR entirely. |
189| `container registry endpoint not found` | ACR is not configured. Use `azd env set AZURE_CONTAINER_REGISTRY_ENDPOINT <url>`, or switch to direct code deployment. |
190| Agent version poll times out | Build still running; retry `azd ai agent show` after a minute. |
191| `session_not_ready` (424) | Cold start or readiness delay. Wait 15-30 seconds and retry. If persistent, use `1` CPU / `2Gi` memory minimum, verify the model deployment name, capability host, and agent identity role. |
192| `invalid value "json" for --output` from `azd ai agent invoke` | Invoke supports only `default` and `raw` currently. Retry without `--output json`. |
193| `could not resolve agent service in azd project: no azure.ai.agent service named '<agentName>' found in azure.yaml` from `azd ai agent invoke` | Name mismatch. Use the service name, update `agent.yaml`, or invoke through the Foundry MCP `agent_invoke` tool. |
194| `subscription quota exceeded` | Ask user to request quota; do not auto-retry. |
195| Bicep deploy errors | Forward `error.details[]` verbatim to the user. |
196| `RoleAssignmentUpdateNotPermitted` during provision | A role assignment already exists but conflicts. Check for existing role assignments with `az role assignment list --scope <resource-scope>`. The provision may have succeeded for all resources except RBAC — verify with `azd ai project show` and manually assign the `Cognitive Services User` role to the agent identity if needed. |
197| `eval generate`: `one of --gen-instruction ... is required` | Retry with `--gen-instruction "<agent purpose>"` (Step 5 option (a)). |
198| `unknown command "init" for "azd ai agent eval"` | Command was renamed: use `azd ai agent eval generate` (requires azd CLI with `azure.ai.agents` extension up to date). |
199 
200For deeper logs, see [troubleshoot](../troubleshoot/troubleshoot.md).
201 
202## Workflow -- Prompt agent (MCP)
203 
204Prompt agents are not containerized -- they are a model + instructions + optional tools, created through the Foundry MCP server. Use when the user explicitly wants a prompt agent.
205 
206### MCP tools
207 
208| Tool | Purpose |
209|------|---------|
210| `agent_definition_schema_get` | Get the schema (`schemaType: "prompt"`). |
211| `agent_update` | Create or update; supports `isCloneRequest` + `cloneTargetAgentName`. |
212| `agent_get` | List or fetch one. |
213| `agent_delete` | Delete an agent. |
214 
215### Steps
216 
2171. **Collect config** -- resolve endpoint from `azd env get-values` or ask. Then ask for **agent name**, **model deployment** (e.g. `gpt-4o`), and optional **instructions**, **temperature**, **tools**.
2182. **Get schema** -- `agent_definition_schema_get` with `schemaType: "prompt"`.
2193. **Create** -- `agent_update` with `{"kind": "prompt", "model": "<deployment>", "instructions": "...", "temperature": 0.7}`.
2204. **Smoke test** -- follow [invoke](../invoke/invoke.md).
2215. **Auto-generate evaluation suite** -- see [Step 5: Auto-Generate Evaluation Suite (Prompt)](#step-5-auto-generate-evaluation-suite-prompt-mandatory--runs-automatically) below.
2226. **Hand off** -- evaluate via [observe](../observe/observe.md); clone via `agent_update` + `isCloneRequest`; delete via `agent_delete`.
223 
224### Step 5: Auto-Generate Evaluation Suite (Prompt) (MANDATORY — RUNS AUTOMATICALLY)
225 
226> ⚠️ **Pre-summary gate.** If you are about to write a deployment summary or Playground link and Step 5 has not run, you are violating this skill. Run Step 5 first.
227 
228This step runs automatically after deploy. Ask the user which source to use and start it right after deploy succeeds — with `--no-wait`, `generate` returns in seconds and generation runs server-side, so it overlaps with invoke/test steps and finishes faster overall.
229 
230> *"Your agent is deployed. Want me to set up an evaluation suite now? (a) Yes — current agent instructions (synthetic Q&A), (b) Yes — historical traces (last 3 days), (c) Yes — use existing `eval.yaml`, (d) No / later."*
231 
232| Choice | Command | What's next |
233|---|---|---|
234| (a) Agent instructions | `azd ai agent eval generate --gen-instruction "<agent purpose>" --no-wait --no-prompt` | Generation runs server-side. Tell the user: *"Suite submitted. Run `azd ai agent eval run` whenever you're ready — it'll finalize `eval.yaml` and execute the eval in one step."* |
235| (b) Historical traces | `azd ai agent eval generate --trace-days 3 --max-samples 50 --no-wait --no-prompt` | Same as (a). |
236| (c) Existing `eval.yaml` | Skip `generate`. | Tell the user: *"Using existing `eval.yaml`. Run `azd ai agent eval run` when ready."* |
237| (d) No / later | Skip. | Tell the user: *"You can run `azd ai agent eval generate` (and then `eval run`) anytime."* |
238 
239## Common failure modes -- Prompt
240 
241| Error | Fix |
242|-------|-----|
243| Schema fetch failed | Verify endpoint format: `https://<resource>.services.ai.azure.com/api/projects/<project>`. |
244| Agent creation failed | Use `agent_definition_schema_get` to verify the definition. |
245| Permission denied | User needs `Foundry User` role on the project. |
246| Model not found | Deploy the model first via [models/deploy-model](../../models/deploy-model/SKILL.md). |
247 
248## Display agent details (both flows)
249 
250After a successful deploy, show the agent's name, version, status, and endpoints in a table. Include a Playground link:
251 
252```
253https://ai.azure.com/nextgen/r/{encodedSubId},{resourceGroup},,{accountName},{projectName}/build/agents/{agentName}/build?version={agentVersion}
254```
255 
256`encodedSubId` is the subscription GUID as URL-safe base64 (no `=`):
257 
258```bash
259python -c "import base64,uuid;print(base64.urlsafe_b64encode(uuid.UUID('<SUBSCRIPTION_ID>').bytes).rstrip(b'=').decode())"
260```
261 
262For hosted agents, `playground_url` is in `azd ai agent show --output json`.
263 
264## After Deployment — Auto-Generate Evaluation Suite
265 
266> Reference for Step 5 options (a) and (b) — start `generate` right after deploy so its server-side generation overlaps with invoke/test steps and finishes faster. Options (c) and (d) skip `generate` and go straight to section 3 (run) or stop.
267 
268### 1. Inspect existing eval.yaml
269 
270Check the selected agent root for `eval.yaml`:
271 
272- **Exists and matches the selected agent** → skip `generate`; go to step 3 (run).
273- **Missing or stale** → continue to step 2.
274 
275### 2. Submit generation (asynchronous, server-side)
276 
277Run `azd ai agent eval generate --no-wait` with the user's chosen flags (see the Step 5 table). The command:
278 
279- Submits dataset + evaluator generation jobs server-side.
280- Returns in seconds.
281- Writes pending operation IDs to local azd state.
282- Writes a placeholder `eval.yaml` at the agent root (override with `--out-file <path>`).
283 
284No skill-side polling, terminal handle, or later-turn re-check is needed. `azd ai agent eval run` (section 3) automatically resumes a pending generation, downloads artifacts, finalizes `eval.yaml`, then runs the eval.
285 
286If the user wants to wait synchronously instead (e.g., to inspect `eval.yaml` before running), drop `--no-wait` — `generate` will then submit the jobs, wait for completion, download review artifacts, and write the finalized `eval.yaml` before returning (typically several minutes).
287 
288### 3. Run the suite
289 
290```bash
291azd ai agent eval run
292```
293 
294Use `azd ai agent eval show -O results.json` to inspect run details, or `azd ai agent eval list` to see history.
295 
296### 4. Refresh datasets/evaluators (later)
297 
298When local files under `datasets/<suite>/` or `evaluators/<suite>/` change, run `azd ai agent eval update --dataset-only` or `--evaluator-only` to upload new versions. azd bumps the `version` fields in `eval.yaml`.
299 
300### 5. Prompt User
301 
302*"Your agent is deployed and evaluation suite generation is **submitted server-side** (still running, takes several minutes). Would you like to run an evaluation now? `azd ai agent eval run` will wait for generation to finish, then execute the eval."*
303 
304- **Yes** → run `azd ai agent eval run` (this resumes the pending generation, then runs the eval — may take several minutes the first time), then follow the [observe skill](../observe/observe.md) to interpret results.
305- **No** → stop. The user can return later via `azd ai agent eval run` — it will pick up wherever the pending generation is.
306- **Production trace analysis** → follow the [trace skill](../trace/trace.md).
307 
308## Non-Interactive / YOLO Mode
309 
310- Hosted: always pass `--no-prompt`. If `azd ai agent invoke` prints a `confirmation_required` envelope, summarize `changes[]` and re-run with `--force` after the user consents -- never auto-append `--force`.
311- Prompt: all required values (project endpoint, agent name, model deployment) must come from the user message or `azd env get-values`; missing values should fail loudly rather than prompt.
312