1# Use Toolbox in a Hosted Agent
2 
3Hosted agents access Foundry-managed tools through a **Toolbox MCP endpoint**. Unlike prompt agents that wire tools directly, hosted agents connect to a single MCP-compatible endpoint that exposes all configured tools. The platform handles credential injection, token refresh, and policy enforcement.
4 
5> 🚦 **Toolbox creation gate:** before creating a toolbox/connection, you MUST read the boundary rules in [create-hosted.md → Toolbox creation boundary](../create-hosted.md#toolbox-creation-boundary) and follow them, then continue with the rest of this file.
6 
7> 📘 For endpoint format, MCP protocol details, auth, OAuth consent handling, testing, citation pattern, and troubleshooting, see [toolbox-reference.md](toolbox-reference.md).
8>
9> 📘 For wiring a remote tool (catalog tile or generic MCP server) into a project connection that a toolbox can attach to, see [foundry-tool-catalog.md](foundry-tool-catalog.md).
10>
11> 📘 For the full list of supported tool types and their per-type fields, see [agent-tools.md](agent-tools.md) and the per-tool `tool-*.md` files.
12 
13> 💡 **This skill is scoped to *consuming* an existing toolbox from agent code** — endpoint resolution, env-var contract, payload shape gathered before agent runtime, verification, and tracing. **Toolbox and connection CRUD belongs in [Foundry Toolkit (VS Code)](https://code.visualstudio.com/docs/intelligentapps/tool-catalog) or the [Foundry Portal](https://ai.azure.com/)** — those surfaces give you tool browsing, metadata, connection wizards, and validation. Use the imperative `azd ai` CLI only for *operational* tasks (retarget the default version, smoke-test an endpoint).
14 
15## ✨ Recommendation: enable Tool Search
16 
17**Before adding more than ~5 tools to a toolbox, add `{ "type": "toolbox_search_preview" }` to the toolbox.** This replaces the full `tools/list` shown to the model with two meta-tools — `tool_search` (natural-language discovery) and `call_tool` (invoke a discovered tool) — so context cost stays flat as the toolbox grows.
18 
19- The `toolbox_search_preview` entry **doesn't count** toward the unnamed-tool-per-type limit.
20- All other tools in the toolbox are hidden from the initial `tools/list` and surfaced only by `tool_search` (or by per-user auto-pinning of hot tools).
21- Pin specific high-traffic tools or add ranking-only keywords via `tool_configs.{tool_name}` (with `pin: true` and `additional_search_text`).
22- In the agent's system prompt, instruct the model to call `tool_search` whenever a needed capability isn't already visible.
23 
24Full configuration recipe in [tool-tool-search.md](tool-tool-search.md) and the public [Tool Search (preview) docs](https://learn.microsoft.com/azure/foundry/agents/how-to/tools/tool-search).
25 
26## Quick Reference
27 
28| Property | Value |
29|----------|-------|
30| **Toolbox Docs** | https://learn.microsoft.com/azure/foundry/agents/how-to/tools/toolbox |
31| **Tool Catalog Docs** | https://learn.microsoft.com/azure/foundry/agents/concepts/tool-catalog |
32| **Tool Search Docs** | https://learn.microsoft.com/azure/foundry/agents/how-to/tools/tool-search |
33| **Foundry Toolkit (VS Code) — set up tools/toolboxes** | https://code.visualstudio.com/docs/intelligentapps/tool-catalog |
34| **Foundry Portal** | https://ai.azure.com/ |
35| **Default Sample (Python, Agent Framework + toolbox)** | https://github.com/microsoft-foundry/foundry-samples/tree/main/samples/python/hosted-agents/agent-framework/responses/04-foundry-toolbox |
36| **Python Hosted Agent — `responses` (BYO)** | https://github.com/microsoft-foundry/foundry-samples/tree/main/samples/python/hosted-agents/bring-your-own/responses |
37| **Python Hosted Agent — `invocations` (BYO)** | https://github.com/microsoft-foundry/foundry-samples/tree/main/samples/python/hosted-agents/bring-your-own/invocations |
38| **C# (.NET) Hosted Agent + toolbox** | https://github.com/microsoft-foundry/foundry-samples/tree/main/samples/csharp/hosted-agents/agent-framework/foundry-toolbox-server-side |
39| **Supported Toolbox Scenarios (sample-side reference)** | https://github.com/microsoft-foundry/foundry-samples/blob/main/samples/python/hosted-agents/SUPPORTED_TOOLBOX_SCENARIOS.md |
40 
41## Resolve Toolbox Endpoint
42 
43If the user provides a toolbox name or endpoint URL, or the project already references a toolbox (e.g., in `.env` or `agent.manifest.yaml`) → use it directly.
44 
45Otherwise, ask one question:
46 
47> _"Would you like to provide your toolbox endpoint? (you can create one with the [Foundry Toolkit in VS Code](https://code.visualstudio.com/docs/intelligentapps/tool-catalog) or the [Foundry Portal](https://ai.azure.com/))"_
48 
49Once the user supplies the toolbox name/endpoint — either an existing one or a new one they create via the Foundry Toolkit or Foundry Portal — set it on the agent (e.g., `TOOLBOX_ENDPOINT` in `.env`) and continue with verification.
50 
51> Use the env var name **`TOOLBOX_ENDPOINT`** (no `FOUNDRY_` prefix). The Foundry platform reserves `FOUNDRY_`-prefixed env vars and may silently overwrite them at runtime — see [toolbox-reference.md § Agent env contract](toolbox-reference.md#agent-env-contract).
52 
53> **When asking the question, always include the doc links inline** for the manual options — the [Foundry Toolkit in VS Code](https://code.visualstudio.com/docs/intelligentapps/tool-catalog) and the [Foundry Portal](https://ai.azure.com/) — so the user knows where to go to create a tool/toolbox themselves. Don't just name the options; render them as clickable links every time.
54 
55> **Before printing out any step-by-step guidance** for the Foundry Toolkit (VS Code) path, fetch and read [Use Tool Catalog to connect tools and Toolboxes in Foundry Toolkit](https://code.visualstudio.com/docs/intelligentapps/tool-catalog) first, then summarize the relevant steps for them. Don't paraphrase from memory — the Toolkit UI changes; quote the current doc.
56 
57## Available tool types
58 
59The full set is documented in [agent-tools.md](agent-tools.md) and — authoritatively — in the public [Toolbox docs (Configure tools)](https://learn.microsoft.com/azure/foundry/agents/how-to/tools/toolbox#configure-tools). At time of writing the supported `type` values are:
60 
61| `type` | Tool | Connection required? | Detail |
62|---|---|---|---|
63| `mcp` | Remote MCP server (third-party via catalog, BYO OAuth, or generic) | Optional (none / static key / project MI / OAuth) | [tool-mcp.md](tool-mcp.md) |
64| `web_search` | Web search (basic Bing; optional `web_search.custom_search_configuration` for Bing Custom Search to scope grounding to specific domains) | No (basic); Yes for Custom Search | [tool-web-search.md](tool-web-search.md) |
65| `azure_ai_search` | Azure AI Search index | Yes (Search service connection) | [tool-azure-ai-search.md](tool-azure-ai-search.md) |
66| `code_interpreter` | Sandboxed Python execution | No | [tool-code-interpreter.md](tool-code-interpreter.md) |
67| `file_search` | Vector-store-backed retrieval over uploaded files | No (vector store is part of the toolbox) | [tool-file-search.md](tool-file-search.md) |
68| `openapi` | REST API exposed via an OpenAPI 3.x spec | Conditional (`connection` requires `project_connection_id`; `managed_identity` does not — uses project MI + `audience`) | [tool-openapi.md](tool-openapi.md) |
69| `a2a_preview` | Call another Foundry agent as a tool | Optional | [tool-a2a.md](tool-a2a.md) |
70| `work_iq_preview` | Microsoft 365 work context (mail / meetings / files / chats) via Work IQ | Yes (Work IQ `RemoteA2A` OAuth connection; BYO Entra app; M365 Copilot license per user) | [tool-work-iq.md](tool-work-iq.md) |
71| `fabric_iq_preview` | Microsoft Fabric data (Ontology / Fabric data agent / Power BI semantic model) | Yes (Fabric IQ OAuth connection; tenant admin consent) | [tool-fabric-iq.md](tool-fabric-iq.md) |
72| `toolbox_search_preview` | **Tool Search** — a directive (not a tool) that swaps `tools/list` for `tool_search` + `call_tool` meta-tools | No | [tool-tool-search.md](tool-tool-search.md) |
73 
74**Adjacent (not a `type` in a toolbox version):**
75 
76- **Agent Memory** — use the `MemorySearchTool` SDK class on prompt agents; for hosted agents, configure the memory store via the project (separate from the toolbox). See [tool-memory.md](tool-memory.md).
77- **Skills** — reusable behavioral guidelines (`SKILL.md`) attached via the `skills[]` array, exposed as MCP resources (`skill://` URIs); the Agent Framework SDK wraps these into a `load_skill` tool for progressive disclosure. See [skill-toolbox.md](skill-toolbox.md).
78- **Routines (preview)** — not a tool; an agent **trigger** (`schedule` / `timer` / `github_issue` / `custom`) that invokes an existing agent. See the [public Routines docs](https://learn.microsoft.com/azure/foundry/agents/how-to/use-routines).
79 
80## Information to Gather Before Building a Toolbox Payload
81 
82When the user asks to "add an MCP tool" or similar, **never guess**. Confirm each field before generating any JSON or `azure.yaml` snippet:
83 
84| # | Question | Why needed |
85|---|----------|------------|
86| 1 | **MCP server URL?** | The `server_url` field on the `mcp` tool entry |
87| 2 | **Auth type?** `none` / `key` / `mi` / `oauth` | Determines whether a project connection is required and which shape to create (see [foundry-tool-catalog.md](foundry-tool-catalog.md)) |
88| 3 | **Project connection name** (if auth ≠ `none`) | The `project_connection_id` field; must already exist in the Foundry project |
89| 4 | **`server_label`** | Short prefix for the tool names exposed by this server (e.g. `myserver`) |
90| 5 | **Toolbox name** | The container that will hold the tool entries |
91| 6 | **Foundry project endpoint** | Where the toolbox is created — read from `PROJECT_ENDPOINT` / `AZURE_AI_PROJECT_ENDPOINT` (avoid `FOUNDRY_`-prefixed names) |
92| 7 | **Many tools planned?** (> ~5) | If yes, also add `{ "type": "toolbox_search_preview" }` so the model uses [Tool Search](#-recommendation-enable-tool-search) instead of seeing the full list. |
93 
94### Toolbox payload — MCP with a project connection
95 
96```json
97{
98  "name": "<TOOLBOX_NAME>",
99  "description": "MCP server with key or OAuth auth",
100  "tools": [
101    {
102      "type": "mcp",
103      "server_label": "<LABEL>",
104      "server_url": "<SERVER_URL>",
105      "require_approval": "never",
106      "project_connection_id": "<CONNECTION_NAME>"
107    }
108  ]
109}
110```
111 
112### Toolbox payload — public MCP (no auth)
113 
114```json
115{
116  "name": "api-specs",
117  "description": "Public MCP server, no connection needed",
118  "tools": [
119    {
120      "type": "mcp",
121      "server_label": "api_specs",
122      "server_url": "https://gitmcp.io/Azure/azure-rest-api-specs",
123      "require_approval": "never"
124    }
125  ]
126}
127```
128 
129### Toolbox payload — large toolbox with Tool Search
130 
131```json
132{
133  "name": "big-toolbox",
134  "description": "Many tools — model uses tool_search to discover",
135  "tools": [
136    { "type": "toolbox_search_preview" },
137    { "type": "web_search" },
138    { "type": "azure_ai_search", "name": "docs_index", "project_connection_id": "search-conn", "index_name": "docs" },
139    {
140      "type": "mcp", "server_label": "github", "server_url": "<github-mcp-url>",
141      "project_connection_id": "gh-conn",
142      "tool_configs": {
143        "search_issues": { "pin": true, "additional_search_text": "GitHub issues bug tracking" },
144        "*":             { "additional_search_text": "GitHub repositories code" }
145      }
146    }
147  ]
148}
149```
150 
151### Declarative path via `azd`
152 
153If the project already uses `azd ai agent init`, prefer declaring the toolbox in `azure.yaml` so `azd deploy` provisions it and injects `TOOLBOX_ENDPOINT` automatically:
154 
155```yaml
156# Declare secret parameters first; azd will prompt for the value on `azd up`
157# (or read it from `AZURE_<NAME>` env vars) and never store it in plaintext.
158params:
159  - name: github_pat
160    type: securestring
161 
162resources:
163  - kind: connection
164    name: <CONNECTION_NAME>
165    target: <MCP_SERVER_URL>
166    category: remoteTool
167    credentials:
168      type: CustomKeys
169      keys:
170        # Header name comes from the catalog entry's x-ms-connection-parameters.
171        # {{ github_pat }} is resolved from the `params` block above.
172        Authorization: "Bearer {{ github_pat }}"
173 
174  - kind: toolbox
175    name: agent-tools
176    tools:
177      - type: toolbox_search_preview   # recommended for any toolbox > ~5 tools
178      - type: web_search
179      - type: mcp
180        server_label: <LABEL>
181        server_url: <MCP_SERVER_URL>
182        project_connection_id: <CONNECTION_NAME>
183```
184 
185See [azd `params` reference](https://learn.microsoft.com/azure/developer/azure-developer-cli/azd-schema#params) for the full parameter syntax.
186 
187## Operational helpers via `azd ai` CLI
188 
189> The `azd ai` CLI also exposes `agent connection create`, `toolbox create`, `toolbox list`, and `toolbox delete`. Prefer **Foundry Toolkit (VS Code)** or the **Foundry Portal** for those — the UI gives you tool browsing, connection wizards, and validation. The two commands below are the ones the skill should still drive directly because they're *operational*, not setup.
190 
191> All commands require `--project-endpoint <PROJECT_ENDPOINT>` (the value of `PROJECT_ENDPOINT`, e.g. `https://<account>.services.ai.azure.com/api/projects/<project>`). To avoid repeating it, export it once:
192>
193> ```pwsh
194> $PE = "https://<account>.services.ai.azure.com/api/projects/<project>"
195> ```
196 
197### Retarget the default version — `azd ai toolbox update`
198 
199Each toolbox version is **immutable**. The version an agent actually hits is the one marked `*` in `version list` — i.e. the **default version**. Use `update` to point that pointer at any existing version (e.g. rollback to a known-good version after a bad publish).
200 
201```pwsh
202# Inspect first — current default is marked with '*'
203azd ai toolbox version list my-toolbox --project-endpoint $PE
204 
205# Retarget the default
206azd ai toolbox update my-toolbox --default-version 20 --project-endpoint $PE --no-prompt
207 
208# Verify (Default version / Shown version / Endpoint all reflect the new value)
209azd ai toolbox show my-toolbox --project-endpoint $PE
210```
211 
212- `--default-version` is the only field `update` accepts today.
213- Validated: switched `default-tb` from version 21 → 20 → 21; both `show` and the computed MCP endpoint (`.../toolboxes/<name>/versions/<n>/mcp?api-version=v1`) tracked the change immediately.
214 
215### End-to-end smoke test
216 
217After the toolbox is created (via Toolkit / Portal / `azd`), hit the MCP endpoint directly to confirm the tool is reachable before pointing an agent at it:
218 
219```pwsh
220$TOK = az account get-access-token --resource "https://ai.azure.com" --query accessToken -o tsv
221$H   = @{
222  Authorization      = "Bearer $TOK"
223  "Content-Type"     = "application/json"
224  "Foundry-Features" = "Toolboxes=V1Preview"
225}
226$URL = "$PE/toolboxes/my-toolbox/mcp?api-version=v1"
227$body = @{ jsonrpc = "2.0"; id = 1; method = "tools/list"; params = @{} } | ConvertTo-Json
228(Invoke-RestMethod -Method POST -Uri $URL -Headers $H -Body $body).result.tools | Select-Object name
229```
230 
231`?api-version=v1` and the `Foundry-Features: Toolboxes=V1Preview` header are both required.
232 
233## Code Integration Patterns
234 
235The sample repo provides integration patterns for both Python and C#. Read the sample code and adapt it to the user's project.
236 
237**Python samples:**
238 
239| Sample | Framework | Protocol | When to use |
240|--------|-----------|----------|-------------|
241| [`agent-framework/responses/04-foundry-toolbox/`](https://github.com/microsoft-foundry/foundry-samples/tree/main/samples/python/hosted-agents/agent-framework/responses/04-foundry-toolbox) — recommended | Agent Framework (MAF) | Responses | **Default choice** |
242| [`bring-your-own/responses/langgraph-toolbox/`](https://github.com/microsoft-foundry/foundry-samples/tree/main/samples/python/hosted-agents/bring-your-own/responses/langgraph-toolbox) | LangGraph (BYO) | Responses | LangGraph hosted agent with toolbox |
243| [`bring-your-own/responses/bring-your-own-toolbox/`](https://github.com/microsoft-foundry/foundry-samples/tree/main/samples/python/hosted-agents/bring-your-own/responses/bring-your-own-toolbox) | Generic MCP (BYO) | Responses | Raw `httpx` MCP client — works with any framework |
244| [`bring-your-own/invocations/toolbox/`](https://github.com/microsoft-foundry/foundry-samples/tree/main/samples/python/hosted-agents/bring-your-own/invocations/toolbox) | Generic MCP (BYO) | Invocations | Toolbox via Invocations protocol |
245 
246**C# (.NET) samples:**
247 
248| Sample | Description |
249|--------|-------------|
250| [`csharp/hosted-agents/agent-framework/foundry-toolbox-server-side/`](https://github.com/microsoft-foundry/foundry-samples/tree/main/samples/csharp/hosted-agents/agent-framework/foundry-toolbox-server-side) — recommended | Agent Framework agent with toolbox MCP (Responses protocol) |
251 
252**Notes** (apply to all patterns, both Python and C#):
253 
254- Auth: Inject a bearer token with scope `https://ai.azure.com/.default` on every request (Python: `httpx.Auth` subclass; C#: `DefaultAzureCredential` + `BearerTokenAuthenticationPolicy`).
255- Header: Always include `Foundry-Features: Toolboxes=V1Preview`.
256- MCP client: Pass `load_prompts=False` — the toolbox endpoint does not support `prompts/list`.
257- Endpoint: Construct from `{project_endpoint}/toolboxes/{toolbox_name}/mcp?api-version=v1`.
258- Multi-tool toolboxes: at most one tool per unnamed type, and unique `server_label` per MCP tool (see [toolbox-reference.md](toolbox-reference.md#multi-tool-toolbox-constraint)). `toolbox_search_preview` doesn't count toward this limit.
259- Tool naming: MCP-sourced tools are prefixed `{server_label}.{tool_name}`; **all other tool types** use the entry's `name` field value (or the default tool name).
260 
261> 💡 **Tip:** If MCP tools have `require_approval: "always"` in `_meta.tool_configuration`, the agent runtime must ask the user for confirmation before invoking. The toolbox endpoint does not enforce this — your agent code is responsible.
262 
263## Tracing
264 
265All toolbox samples emit OpenTelemetry traces. No code changes are required to enable export to Azure Monitor — it's purely a configuration step.
266 
267- **Local development:** set `APPLICATIONINSIGHTS_CONNECTION_STRING` in the agent's `.env`.
268- **Deployed:** the platform injects `APPLICATIONINSIGHTS_CONNECTION_STRING` automatically when the Foundry project is linked to an Application Insights resource.
269- **Per-framework instrumentation hooks** (already present in the samples):
270  - `maf` — `main.py` calls `enable_instrumentation()`.
271  - `langgraph` / `azd` — auto-instrumented by `azure-ai-agentserver-core[tracing]`.
272- **Viewing traces:** Azure Portal → Application Insights → **Investigate → Transaction search** (per-trace) or **Application map** (dependency graph).
273