1---
2name: benchmarking
3description: Write benchmarks with the v5 bench test-context fixture (Tinybench)
4---
5 
6# Benchmarking (v5)
7 
8In v5 the benchmark API was rewritten: `bench` is **no longer a top-level import**. It is a [test-context fixture](features-context.md) used inside a regular `test()`, available only in files matched by `benchmark.include` (default `**/*.{bench,benchmark}.?(c|m)[jt]s?(x)`). Benchmarks are powered by [Tinybench](https://github.com/tinylibs/tinybench).
9 
10## Defining & Running
11 
12```ts
13import { expect, test } from 'vitest'
14 
15test('parse performance', async ({ bench }) => {
16  // bench() registers; .run() executes and returns the result
17  const result = await bench('parse', () => {
18    const data = JSON.parse('{"key":"value"}')
19    use(data) // consume the result — engines may eliminate dead code
20  }).run()
21 
22  expect(result.throughput.mean).toBeGreaterThan(10_000)
23})
24```
25 
26Run benchmarks:
27 
28```bash
29vitest bench           # only benchmarks (implicitly enables them)
30vitest bench parser    # filter by filename
31vitest bench -t JSON   # filter by test name
32```
33 
34Set `benchmark: { enabled: true }` to run them alongside regular tests in a separate isolated group.
35 
36## Comparing Implementations
37 
38```ts
39test('compare parsers', async ({ bench }) => {
40  const result = await bench.compare(
41    bench('JSON.parse', () => { JSON.parse(input) }),
42    bench('custom', { beforeEach: () => reset() }, () => { customParse(input) }),
43    { iterations: 100, time: 1000 }, // shared Tinybench options (last arg)
44  )
45 
46  // Assertion matchers (delta avoids flaky failures)
47  expect(result.get('JSON.parse')).toBeFasterThan(result.get('custom'), { delta: 0.1 })
48  expect(result.get('custom')).toBeSlowerThan(result.get('JSON.parse'))
49})
50```
51 
52`bench.compare` interleaves iterations to reduce environmental bias and prints a comparison table after the test.
53 
54## Storing & Replaying Baselines
55 
56```ts
57test('compare against baseline', async ({ bench }) => {
58  await bench.compare(
59    bench('current', { writeResult: './benchmarks/parse.json' }, () => parse(input)),
60    bench.from('previous', './benchmarks/parse.json'),       // reads a stored result, no run
61    bench.from('remote', () => fetch(url).then(r => r.json())),
62  )
63})
64```
65 
66- `writeResult` overwrites the JSON file on every successful run (no skip-when-cached).
67- `bench.from(name, source)` reads a stored result without invoking any function.
68- For multi-project workspaces, pass `{ perProject: true }` and use `${projectName}` in `writeResult` paths to collect a cross-project comparison table.
69 
70## Stability Notes
71 
72- Benchmark files run sequentially and never in parallel; `retry` and the `delta` option reduce flakiness.
73- Consume the result inside the bench fn — JS engines eliminate side-effect-free code.
74- In Node mode every imported binding goes through Vite's module-runner getter; store hot references locally (`const _parse = parse`), benchmark the built package, or disable `experimental.viteModuleRunner` for the bench project.
75 
76## v5 Migration
77 
78- `bench` top-level import → `({ bench })` from the test context
79- `bench.skip/only/todo` removed → use `test.skip/only/todo` on the surrounding test
80- `benchmark.reporters`/`outputFile`/`compare`/`outputJson` and `--compare`/`--outputJson` removed → use `--reporter=json --outputFile` (JSON now has a `benchmarks` field)
81 
82## Key Points
83 
84- Benchmarks live in `*.bench.ts` files and run inside `test()` via `{ bench }`
85- Use `bench.compare` + `toBeFasterThan`/`toBeSlowerThan` (with `delta`) for relative perf
86- Persist baselines with `writeResult` and replay with `bench.from`
87 
88<!--
89Source references:
90- https://vitest.dev/guide/benchmarking
91- https://vitest.dev/guide/test-context#bench
92-->
93