Source from repo
Playwright Best Practices

Comprehensive Playwright testing guide covering E2E, component, API, visual, accessibility, and security tests.
currents-devGitHub currents-devSource repo Original GitHub link Publisher page
Files
Skill
n/a
Size
755.6 KB
Entrypoint
SKILL.md
Format
git-repo
Open file
infrastructure-ci-cd/other-providers.md

Syntax-highlighted preview of this file as included in the skill package.
Rendered Source
markdown522 linesFree
infrastructure-ci-cd/other-providers.md
1# CI: CircleCI, Azure DevOps, and Jenkins
2 
3> **When to use**: Running Playwright tests in CI platforms other than GitHub Actions or GitLab.
4 
5## Table of Contents
6 
71. [Common Commands](#common-commands)
82. [Jenkins](#jenkins)
93. [CircleCI](#circleci)
104. [Azure DevOps](#azure-devops)
115. [JUnit Reporter Config](#junit-reporter-config)
126. [Platform Comparison](#platform-comparison)
137. [Troubleshooting](#troubleshooting)
148. [Anti-Patterns](#anti-patterns)
15 
16---
17 
18## Common Commands
19 
20```bash
21npx playwright install --with-deps    # browsers + OS dependencies
22npx playwright test --shard=1/4       # parallel sharding
23npx playwright merge-reports ./blob-report  # combine shard results
24npx playwright test --reporter=dot,html     # multiple reporters
25```
26 
27## Jenkins
28 
29### Declarative Pipeline
30 
31```groovy
32// Jenkinsfile
33pipeline {
34    agent {
35        docker {
36            image 'mcr.microsoft.com/playwright:v1.48.0-noble'
37            args '-u root'
38        }
39    }
40 
41    environment {
42        CI = 'true'
43        HOME = '/root'
44        npm_config_cache = "${WORKSPACE}/.npm"
45    }
46 
47    options {
48        timeout(time: 30, unit: 'MINUTES')
49        disableConcurrentBuilds()
50    }
51 
52    stages {
53        stage('Install') {
54            steps {
55                sh 'npm ci'
56            }
57        }
58 
59        stage('Test') {
60            steps {
61                sh 'npx playwright test'
62            }
63            post {
64                always {
65                    junit allowEmptyResults: true,
66                         testResults: 'results/junit.xml'
67                    archiveArtifacts artifacts: 'pw-report/**',
68                                     allowEmptyArchive: true
69                    archiveArtifacts artifacts: 'results/**',
70                                     allowEmptyArchive: true
71                }
72            }
73        }
74    }
75 
76    post {
77        failure {
78            echo 'Tests failed!'
79        }
80        cleanup {
81            cleanWs()
82        }
83    }
84}
85```
86 
87### Parallel Shards
88 
89```groovy
90// Jenkinsfile (sharded)
91pipeline {
92    agent none
93 
94    environment {
95        CI = 'true'
96        HOME = '/root'
97    }
98 
99    options {
100        timeout(time: 30, unit: 'MINUTES')
101    }
102 
103    stages {
104        stage('Test') {
105            parallel {
106                stage('Shard 1') {
107                    agent {
108                        docker {
109                            image 'mcr.microsoft.com/playwright:v1.48.0-noble'
110                            args '-u root'
111                        }
112                    }
113                    steps {
114                        sh 'npm ci'
115                        sh 'npx playwright test --shard=1/4'
116                    }
117                    post {
118                        always {
119                            archiveArtifacts artifacts: 'blob-report/**',
120                                             allowEmptyArchive: true
121                        }
122                    }
123                }
124                stage('Shard 2') {
125                    agent {
126                        docker {
127                            image 'mcr.microsoft.com/playwright:v1.48.0-noble'
128                            args '-u root'
129                        }
130                    }
131                    steps {
132                        sh 'npm ci'
133                        sh 'npx playwright test --shard=2/4'
134                    }
135                    post {
136                        always {
137                            archiveArtifacts artifacts: 'blob-report/**',
138                                             allowEmptyArchive: true
139                        }
140                    }
141                }
142                stage('Shard 3') {
143                    agent {
144                        docker {
145                            image 'mcr.microsoft.com/playwright:v1.48.0-noble'
146                            args '-u root'
147                        }
148                    }
149                    steps {
150                        sh 'npm ci'
151                        sh 'npx playwright test --shard=3/4'
152                    }
153                    post {
154                        always {
155                            archiveArtifacts artifacts: 'blob-report/**',
156                                             allowEmptyArchive: true
157                        }
158                    }
159                }
160                stage('Shard 4') {
161                    agent {
162                        docker {
163                            image 'mcr.microsoft.com/playwright:v1.48.0-noble'
164                            args '-u root'
165                        }
166                    }
167                    steps {
168                        sh 'npm ci'
169                        sh 'npx playwright test --shard=4/4'
170                    }
171                    post {
172                        always {
173                            archiveArtifacts artifacts: 'blob-report/**',
174                                             allowEmptyArchive: true
175                        }
176                    }
177                }
178            }
179        }
180    }
181}
182```
183 
184## CircleCI
185 
186### Basic Pipeline
187 
188```yaml
189# .circleci/config.yml
190version: 2.1
191 
192executors:
193  pw:
194    docker:
195      - image: mcr.microsoft.com/playwright:v1.48.0-noble
196    working_directory: ~/app
197 
198jobs:
199  install:
200    executor: pw
201    steps:
202      - checkout
203      - restore_cache:
204          keys:
205            - deps-{{ checksum "package-lock.json" }}
206      - run: npm ci
207      - save_cache:
208          key: deps-{{ checksum "package-lock.json" }}
209          paths:
210            - node_modules
211      - persist_to_workspace:
212          root: .
213          paths:
214            - node_modules
215 
216  test:
217    executor: pw
218    parallelism: 4
219    steps:
220      - checkout
221      - attach_workspace:
222          at: .
223      - run:
224          name: Run tests
225          command: |
226            npx playwright test --shard=$((CIRCLE_NODE_INDEX + 1))/$CIRCLE_NODE_TOTAL
227      - store_artifacts:
228          path: pw-report
229          destination: pw-report
230      - store_artifacts:
231          path: results
232          destination: results
233      - store_test_results:
234          path: results/junit.xml
235 
236workflows:
237  test:
238    jobs:
239      - install
240      - test:
241          requires:
242            - install
243```
244 
245### Using Orbs
246 
247```yaml
248# .circleci/config.yml
249version: 2.1
250 
251orbs:
252  node: circleci/node@latest
253 
254executors:
255  pw:
256    docker:
257      - image: mcr.microsoft.com/playwright:v1.48.0-noble
258 
259jobs:
260  e2e:
261    executor: pw
262    parallelism: 4
263    steps:
264      - checkout
265      - node/install-packages
266      - run:
267          name: Run tests
268          command: npx playwright test --shard=$((CIRCLE_NODE_INDEX + 1))/$CIRCLE_NODE_TOTAL
269      - store_artifacts:
270          path: pw-report
271      - store_test_results:
272          path: results/junit.xml
273 
274workflows:
275  main:
276    jobs:
277      - e2e
278```
279 
280## Azure DevOps
281 
282### Basic Pipeline
283 
284```yaml
285# azure-pipelines.yml
286trigger:
287  branches:
288    include:
289      - main
290 
291pr:
292  branches:
293    include:
294      - main
295 
296pool:
297  vmImage: "ubuntu-latest"
298 
299variables:
300  CI: "true"
301  npm_config_cache: $(Pipeline.Workspace)/.npm
302 
303steps:
304  - task: NodeTool@0
305    inputs:
306      versionSpec: "20.x"
307    displayName: "Install Node.js"
308 
309  - task: Cache@2
310    inputs:
311      key: 'npm | "$(Agent.OS)" | package-lock.json'
312      restoreKeys: |
313        npm | "$(Agent.OS)"
314      path: $(npm_config_cache)
315    displayName: "Cache npm"
316 
317  - script: npm ci
318    displayName: "Install dependencies"
319 
320  - script: npx playwright install --with-deps
321    displayName: "Install browsers"
322 
323  - script: npx playwright test
324    displayName: "Run tests"
325 
326  - task: PublishTestResults@2
327    condition: always()
328    inputs:
329      testResultsFormat: "JUnit"
330      testResultsFiles: "results/junit.xml"
331      mergeTestResults: true
332      testRunTitle: "E2E Tests"
333    displayName: "Publish results"
334 
335  - task: PublishPipelineArtifact@1
336    condition: always()
337    inputs:
338      targetPath: pw-report
339      artifact: pw-report
340      publishLocation: "pipeline"
341    displayName: "Upload report"
342```
343 
344### With Sharding
345 
346```yaml
347# azure-pipelines.yml
348trigger:
349  branches:
350    include:
351      - main
352 
353pr:
354  branches:
355    include:
356      - main
357 
358variables:
359  CI: "true"
360 
361stages:
362  - stage: Test
363    jobs:
364      - job: E2E
365        pool:
366          vmImage: "ubuntu-latest"
367        strategy:
368          matrix:
369            shard1:
370              SHARD: "1/4"
371            shard2:
372              SHARD: "2/4"
373            shard3:
374              SHARD: "3/4"
375            shard4:
376              SHARD: "4/4"
377        steps:
378          - task: NodeTool@0
379            inputs:
380              versionSpec: "20.x"
381 
382          - script: npm ci
383            displayName: "Install dependencies"
384 
385          - script: npx playwright install --with-deps
386            displayName: "Install browsers"
387 
388          - script: npx playwright test --shard=$(SHARD)
389            displayName: "Run tests (shard $(SHARD))"
390 
391          - task: PublishPipelineArtifact@1
392            condition: always()
393            inputs:
394              targetPath: blob-report
395              artifact: blob-report-$(System.JobPositionInPhase)
396            displayName: "Upload blob report"
397 
398  - stage: Report
399    dependsOn: Test
400    condition: always()
401    jobs:
402      - job: MergeReports
403        pool:
404          vmImage: "ubuntu-latest"
405        steps:
406          - task: NodeTool@0
407            inputs:
408              versionSpec: "20.x"
409 
410          - script: npm ci
411            displayName: "Install dependencies"
412 
413          - task: DownloadPipelineArtifact@2
414            inputs:
415              patterns: "blob-report-*/**"
416              path: all-blob-reports
417            displayName: "Download blob reports"
418 
419          - script: npx playwright merge-reports --reporter=html ./all-blob-reports
420            displayName: "Merge reports"
421 
422          - task: PublishPipelineArtifact@1
423            inputs:
424              targetPath: pw-report
425              artifact: pw-report
426            displayName: "Upload merged report"
427```
428 
429## JUnit Reporter Config
430 
431All platforms benefit from JUnit output for native test result display:
432 
433```typescript
434// playwright.config.ts
435import { defineConfig } from "@playwright/test";
436 
437export default defineConfig({
438  reporter: process.env.CI
439    ? [
440        ["dot"],
441        ["html", { open: "never" }],
442        ["junit", { outputFile: "results/junit.xml" }],
443      ]
444    : [["html", { open: "on-failure" }]],
445});
446```
447 
448## Platform Comparison
449 
450| Feature           | CircleCI                                        | Azure DevOps                     | Jenkins                |
451| ----------------- | ----------------------------------------------- | -------------------------------- | ---------------------- |
452| Docker support    | `docker:` executor                              | `vmImage` or container jobs      | Docker Pipeline plugin |
453| Parallelism       | `parallelism: N` + `CIRCLE_NODE_INDEX`          | `strategy.matrix`                | `parallel` stages      |
454| Artifact upload   | `store_artifacts`                               | `PublishPipelineArtifact@1`      | `archiveArtifacts`     |
455| JUnit integration | `store_test_results`                            | `PublishTestResults@2`           | `junit` step           |
456| Shard variable    | `$((CIRCLE_NODE_INDEX + 1))/$CIRCLE_NODE_TOTAL` | Define in matrix: `SHARD: '1/4'` | Hardcode per stage     |
457| Cache key         | `checksum "package-lock.json"`                  | `Cache@2` with key template      | `stash`/`unstash`      |
458| Secrets           | Context + env variables                         | Variable groups                  | Credentials plugin     |
459 
460## Troubleshooting
461 
462### Jenkins: "Browser closed unexpectedly"
463 
464Running as non-root in container causes sandbox issues.
465 
466```groovy
467agent {
468    docker {
469        image 'mcr.microsoft.com/playwright:v1.48.0-noble'
470        args '-u root'
471    }
472}
473environment {
474    HOME = '/root'
475}
476```
477 
478### CircleCI: "Executable doesn't exist"
479 
480Image version mismatch with `@playwright/test` version. Use `latest` tag or match versions:
481 
482```yaml
483docker:
484  - image: mcr.microsoft.com/playwright:v1.48.0-noble
485```
486 
487### Azure DevOps: Test results not showing
488 
489Missing JUnit reporter or `PublishTestResults@2` task:
490 
491```typescript
492reporter: [['junit', { outputFile: 'results/junit.xml' }]],
493```
494 
495```yaml
496- task: PublishTestResults@2
497  condition: always()
498  inputs:
499    testResultsFormat: "JUnit"
500    testResultsFiles: "results/junit.xml"
501```
502 
503### Shard index off by one
504 
505CircleCI's `CIRCLE_NODE_INDEX` is 0-based, Playwright's `--shard` is 1-based:
506 
507```yaml
508# CircleCI - add 1
509command: npx playwright test --shard=$((CIRCLE_NODE_INDEX + 1))/$CIRCLE_NODE_TOTAL
510```
511 
512## Anti-Patterns
513 
514| Anti-Pattern                        | Problem                                   | Solution                                             |
515| ----------------------------------- | ----------------------------------------- | ---------------------------------------------------- |
516| Missing `--with-deps` on bare metal | OS libs missing, browser launch fails     | Use Playwright Docker image or `--with-deps`         |
517| No JUnit reporter                   | CI can't display test results             | Add `['junit', { outputFile: 'results/junit.xml' }]` |
518| No job timeout                      | Hung tests consume resources indefinitely | Set explicit timeout (20-30 min)                     |
519| No artifact upload on success       | Can't verify passing results              | Always upload reports (`condition: always()`)        |
520| Non-root in container without setup | Permission errors on browser binaries     | Run as root or configure permissions                 |
521| Hardcoded shard count               | Must update multiple places               | Use CI-native variables                              |
522
Preparing the source view

Playwright Best Practices

infrastructure-ci-cd/other-providers.md
