1## Container Apps Observability
2 
3Observability guide for apps running in Azure Container Apps.
4 
5## Environment-Level Log Analytics
6 
7By default, Container Apps environments use a Log Analytics workspace. Configure it at environment creation (`--logs-workspace-id` expects the workspace **Customer ID** (GUID), not the ARM resource ID):
8 
9```bash
10WORKSPACE_ID=$(az monitor log-analytics workspace show \
11  --resource-group <rg> --workspace-name <workspace-name> \
12  --query customerId -o tsv)
13 
14WORKSPACE_KEY=$(az monitor log-analytics workspace get-shared-keys \
15  --resource-group <rg> --workspace-name <workspace-name> \
16  --query primarySharedKey -o tsv)
17 
18az containerapp env create \
19  --name <env-name> \
20  --resource-group <rg> \
21  --logs-workspace-id $WORKSPACE_ID \
22  --logs-workspace-key $WORKSPACE_KEY \
23  --logs-destination log-analytics
24```
25 
26> 💡 **Tip:** All apps in the same environment share the workspace. Use `--logs-destination none` only for BYOB (bring-your-own-backend) scenarios.
27 
28## System Logs vs Application Logs
29 
30| Log Table | Content | Retention |
31|-----------|---------|-----------|
32| `ContainerAppConsoleLogs_CL` | stdout/stderr from containers | Workspace default |
33| `ContainerAppSystemLogs_CL` | Platform events (scaling, restarts, image pulls) | Workspace default |
34 
35> ⚠️ **Note:** The `_CL` suffix and `_s` column suffixes apply to the **Log Analytics** destination. Environments using the newer **Azure Monitor** destination use `ContainerAppConsoleLogs` / `ContainerAppSystemLogs` (no `_CL`, no `_s` suffixes). Check your environment's log destination to use the correct table name.
36 
37System logs capture events outside your code—replica scheduling, health probe results, and revision activation. Console logs capture everything your app writes to stdout/stderr.
38 
39## Built-in Metrics
40 
41Container Apps exposes these metrics without any SDK:
42 
43| Metric | Description | Dimensions |
44|--------|-------------|-----------|
45| `Replicas` | Current replica count | `revision` |
46| `Requests` | HTTP request count | `statusCode`, `statusCodeCategory`, `revision`, `replica` |
47| `UsageNanoCores` | CPU usage per replica | `revision`, `replica` |
48| `WorkingSetBytes` | Memory usage per replica | `revision`, `replica` |
49| `RestartCount` | Container restart count | `revision`, `replica` |
50| `RxBytes` / `TxBytes` | Network I/O | `revision`, `replica` |
51 
52> ⚠️ **Warning:** Built-in metrics cover infrastructure only. For request-level tracing, response times, and dependency tracking, add Application Insights SDK.
53 
54## Application Insights SDK Setup
55 
56Set `APPLICATIONINSIGHTS_CONNECTION_STRING` as an environment variable on the container app, then add the SDK per language:
57 
58| Language | Package | Init Pattern |
59|----------|---------|-------------|
60| Node.js | `@azure/monitor-opentelemetry` | Call `useAzureMonitor()` before app startup |
61| Python | `azure-monitor-opentelemetry` | Call `configure_azure_monitor()` at entry |
62| .NET | `Azure.Monitor.OpenTelemetry.AspNetCore` | `builder.Services.AddOpenTelemetry().UseAzureMonitor()` |
63| Java | Agent JAR (manual) | Set `JAVA_TOOL_OPTIONS=-javaagent:/agent/applicationinsights-agent.jar` |
64 
65```bash
66# Store as a secret (recommended — keeps value out of az show output and portal config)
67az containerapp secret set -n <app-name> -g <rg> \
68  --secrets "appinsights-conn=<conn-string>"
69 
70az containerapp update \
71  --name <app-name> \
72  --resource-group <rg> \
73  --set-env-vars "APPLICATIONINSIGHTS_CONNECTION_STRING=secretref:appinsights-conn"
74```
75 
76## Distributed Tracing Across Microservices
77 
78Container Apps with multiple services need correlation. The OpenTelemetry SDK propagates `traceparent` headers automatically through HTTP calls. Ensure:
79 
801. Every microservice has the SDK initialized with the **same** Application Insights resource
812. HTTP clients use instrumented libraries (e.g., `requests` in Python, `fetch`/`axios` in Node.js)
823. Verify end-to-end traces in the **Application Map** blade
83 
84> 💡 **Tip:** Use `operation_Id` in KQL queries to trace a single request across all services.
85 
86## Dapr Observability
87 
88For apps using Dapr sidecars, Dapr generates tracing spans for service invocation, pub/sub, and state operations when tracing is configured. Note that `samplingRate: "1"` means 100% sampling — consider lowering for production workloads.
89 
90Configure Dapr tracing in the Container Apps environment. The YAML below represents the config spec — in ACA, apply it via `az containerapp env dapr-component set` or ARM/Bicep (not as a raw YAML file):
91 
92```yaml
93# Dapr tracing config spec (apply via CLI or Bicep, not raw kubectl)
94apiVersion: dapr.io/v1alpha1
95kind: Configuration
96metadata:
97  name: appconfig
98spec:
99  tracing:
100    samplingRate: "1"
101    otel:
102      endpointAddress: "<otlp-collector-endpoint>"
103      isSecure: true
104      protocol: grpc
105```
106 
107> ⚠️ **Note:** `endpointAddress` should point to an OpenTelemetry Collector (not Application Insights directly). Configure the collector with the Azure Monitor exporter to forward traces to App Insights.
108 
109Dapr generates spans for:
110- **Service invocation** — caller → Dapr sidecar → target sidecar → target app
111- **Pub/sub** — publisher → broker → subscriber
112- **Bindings** — input/output binding operations
113 
114## ARG Queries — Monitoring Status
115 
116Discover Container Apps and their monitoring configuration:
117 
118```kql
119// Container Apps without App Insights configured (checks all containers)
120resources
121| where type == "microsoft.app/containerapps"
122| mv-expand container = properties.template.containers
123| mv-expand envVar = container.env
124| where isnotempty(envVar)
125| summarize hasAppInsights = countif(envVar.name == "APPLICATIONINSIGHTS_CONNECTION_STRING") by name, resourceGroup
126| where hasAppInsights == 0
127```
128 
129> **Note:** This query only covers apps with existing environment variables. Apps with no env vars are excluded by the `mv-expand` and should be identified separately (e.g., filter for containers where `env` is null or empty).
130 
131## KQL Query Library
132 
133### Console log errors
134 
135```kql
136ContainerAppConsoleLogs_CL
137| where Log_s contains "error" or Log_s contains "exception"
138| project TimeGenerated, ContainerAppName_s, RevisionName_s, Log_s
139| order by TimeGenerated desc
140| take 50
141```
142 
143### Replica restart events
144 
145```kql
146ContainerAppSystemLogs_CL
147| where EventSource_s == "ContainerAppController" and Reason_s == "Restarting"
148| summarize restarts = count() by ContainerAppName_s, RevisionName_s, bin(TimeGenerated, 1h)
149| order by TimeGenerated desc
150```
151 
152### Scaling events
153 
154```kql
155ContainerAppSystemLogs_CL
156| where Reason_s in ("ScalingUp", "ScalingDown")
157| project TimeGenerated, ContainerAppName_s, Reason_s, Log_s
158| order by TimeGenerated desc
159```
160 
161### Console log volume by revision
162 
163```kql
164ContainerAppConsoleLogs_CL
165| where isnotempty(Log_s)
166| summarize logCount = count() by RevisionName_s, bin(TimeGenerated, 5m)
167| render timechart
168```
169 
170### Request latency by instance (requires Application Insights SDK)
171 
172```kql
173requests
174| where cloud_RoleName has "<app-name>"
175| summarize avgDuration = avg(duration), p95 = percentile(duration, 95) by cloud_RoleInstance, bin(timestamp, 5m)
176| render timechart
177```
178 
179> 💡 **Tip:** Console logs don't contain latency data. For request-level latency and dependency analysis, query the `requests` and `dependencies` tables from Application Insights.
180