1# Pipelines Gotchas
2 
3Non-obvious failure modes (not well covered by docs). For current limits and error semantics, pull `https://developers.cloudflare.com/pipelines/platform/limits/`.
4 
5## Events accepted but never appear (most common)
6 
7HTTP 200 / `send()` resolves, but no data in the sink. Causes:
8 
91. **Schema validation failure** — structured streams accept then **silently drop** invalid events during processing. Validate client-side (Zod) and monitor `pipelinesUserErrorsAdaptiveGroups`.
102. **First-flush warm-up** — first data takes **3–7 minutes** (warm-up + namespace/table creation) even with `--roll-interval 10`. Poll ≥5 min in tests.
113. **Roll interval not elapsed** — default 300s.
124. **Silent sink failure** — deleted bucket or expired token. Check `recordsWritten > 0` but `filesWritten = 0`; inspect `failure_reason` via `GET /pipelines/{id}`.
13 
14## Everything is immutable
15 
16Cannot modify stream schema, pipeline SQL, or sink config — delete and recreate. Use version naming (`events_v1`) and keep SQL in version control.
17 
18```bash
19curl -X DELETE "$BASE_URL/pipelines/{id}" -H "Authorization: Bearer $API_TOKEN"
20curl -X DELETE "$BASE_URL/sinks/{id}"     -H "Authorization: Bearer $API_TOKEN"
21curl -X DELETE "$BASE_URL/streams/{id}"   -H "Authorization: Bearer $API_TOKEN"
22```
23 
24## Worker binding undefined (`env.MY_STREAM`)
25 
261. Use the **stream ID**, not pipeline ID, in `wrangler.jsonc`.
272. Binding field is `"stream"` (June 2026); old `"pipeline"` still works.
283. Redeploy after adding the binding.
29 
30## REST API field names ≠ CLI flags
31 
32`r2_data_catalog` vs `--type r2-data-catalog`, `table_name` vs `--table`, `token` vs `--catalog-token`, and `format` is required in REST but implied in CLI. See [configuration.md](configuration.md#option-c-rest-api-programmatic).
33 
34## `wrangler pipelines delete` defaults to "no"
35 
36Non-interactive environments answer "no" automatically — use REST `DELETE` for CI/automation.
37 
38## Behavioral Notes
39 
40- **`__ingest_ts` auto-added** (TIMESTAMP, day-partitioned). Don't put it in your schema.
41- **Sinks can't target existing tables** — the sink creates its own. Use PySpark to write to existing tables.
42- **JSON-only input** — no Avro/Protobuf/CSV.
43- **Naming:** streams/sinks/pipelines use underscores; buckets use hyphens.
44- **Metrics lag 5–10 min** after creation.
45- **Pipeline SQL is row-level only** — no GROUP BY/aggregation/window functions (do aggregation in [R2 SQL](../r2-sql/) at query time). CTEs and `UNNEST` are supported.
46 
47## Debug Checklist
48 
49- [ ] Stream exists: `wrangler pipelines streams list`
50- [ ] Pipeline `running` (not `initializing`/`failed`): `GET /pipelines/{id}`, check `failure_reason`
51- [ ] SQL matches schema; sink token valid; bucket + catalog exist
52- [ ] Worker redeployed; binding uses **stream ID** under `"stream"`
53- [ ] Waited ≥5 min (first flush)
54- [ ] Sink metrics: `filesWritten > 0`; error metrics show no drops
55 
56## See Also
57 
58- [configuration.md](configuration.md) · [api.md](api.md) · [patterns.md](patterns.md)
59