1# Spot And Zone Issues
2 
3Use this guide when workload placement, evictions, or zonal behavior is causing node-pool instability.
4 
5## Spot Node Pool Evictions
6 
7AKS spot nodes use Azure Spot VMs - they can be evicted with 30 seconds notice when Azure needs capacity.
8 
9**Diagnose spot eviction:**
10 
11```bash
12# Spot nodes carry this taint automatically
13kubectl describe node <node> | grep "Taint"
14# kubernetes.azure.com/scalesetpriority=spot:NoSchedule
15 
16# Check eviction events
17kubectl get events -A --field-selector reason=SpotEviction
18kubectl get events -A | grep -i "evict\|spot\|preempt"
19```
20 
21**Spot workload pattern:** pods must tolerate the spot taint. Prefer PDBs and avoid stateful PVC workloads on spot.
22 
23```yaml
24tolerations:
25  - key: "kubernetes.azure.com/scalesetpriority"
26    operator: Equal
27    value: spot
28    effect: NoSchedule
29```
30 
31Add this preferred node affinity when you want the workload to bias toward spot nodes:
32 
33```yaml
34affinity:
35  nodeAffinity:
36    preferredDuringSchedulingIgnoredDuringExecution:
37      - weight: 1
38        preference:
39          matchExpressions:
40            - key: kubernetes.azure.com/scalesetpriority
41              operator: In
42              values: ["spot"]
43```
44 
45---
46 
47## Multi-AZ Node Pool & Zone-Related Failures
48 
49**Check zone distribution:**
50 
51```bash
52kubectl get nodes -L topology.kubernetes.io/zone
53```
54 
55**Zone-related failure patterns:**
56 
57| Symptom                                          | Cause                                                | Fix                                                          |
58| ------------------------------------------------ | ---------------------------------------------------- | ------------------------------------------------------------ |
59| Pods stack on one zone after node failures       | Scheduling imbalance after zone failure              | `kubectl rollout restart deployment/<n>` to rebalance        |
60| PVC pending with `volume node affinity conflict` | Azure Disk is zonal; pod scheduled in different zone | Use ZRS storage class or ensure PVC and pod are in same zone |
61| Service endpoints unreachable from one zone      | Topology-aware routing misconfigured                 | Check `service.spec.trafficDistribution` or TopologyKeys     |
62| Upgrade causing zone imbalance                   | Surge nodes in one zone                              | Configure `maxSurge` in node pool upgrade settings           |
63 
64Use `Premium_ZRS` or `StandardSSD_ZRS` in custom StorageClasses to reduce zonal PVC conflicts. See [AKS storage best practices](https://learn.microsoft.com/azure/aks/operator-best-practices-storage).
65