Your Web News in One Place

Help Webnuz

Referal links:

Sign up for GreenGeeks web hosting
January 24, 2023 05:48 pm GMT

Fast and reliable end-to-end tests with Playwright on GitHub Actions

Playwright is a powerful web testing tool supporting Chromium, Firefox and WebKit engines. The project has excellent documentation and many examples.

I use Playwright for E2E tests on the DEV environment. Today I'm going to show how you can integrate Playwright on GitHub Actions more efficiently.

Playwright by Microsoft

The documentation suggests using the following snippet to run Playwright inside GitHub Actions:

steps:  - uses: actions/[email protected]  - uses: actions/[email protected]    with:      node-version: '18'  - name: Install dependencies    run: npm ci  - name: Install Playwright    run: npx playwright install --with-deps  - name: Run your tests    run: npx playwright test  - name: Upload test results    if: always()    uses: actions/[email protected]    with:      name: playwright-report      path: playwright-report

What is notable here?

  • It's supposed to run all tests on the same type of GitHub Runner. ubuntu-latest by default.
  • All supported browser engines (chromium, firefox, WebKit) will be installed on the Linux runner.
  • Playwright will execute tests for all browser engines from the same runner.
  • The last step will prepare an archive with an HTML report.

After several months of experiments I come up with the following configuration:

# ...jobs:  test:    name:  ${{ matrix.project }} E2E Tests    runs-on: ${{ matrix.os }}    timeout-minutes: 20    strategy:      fail-fast: false      matrix:        include:          - project: chromium            os: ubuntu-latest            cache_dir: ~/.cache/ms-playwright          - project: firefox            os: ubuntu-latest            cache_dir: ~/.cache/ms-playwright          - project: webkit            os: macos-12            cache_dir: ~/Library/Caches/ms-playwright    steps:    - uses: actions/[email protected]    - uses: actions/[email protected]      with:        node-version-file: '.nvmrc'        cache: 'npm'    - name:  Cache NPM dependencies      uses: actions/[email protected]      id: cache-primes      with:        path: node_modules        key: ${{ runner.os }}-npm-${{ hashFiles('package-lock.json') }}    - name:  Cache playwright binaries      uses: actions/[email protected]      id: playwright-cache      with:        path: ${{ matrix.cache_dir }}        key: ${{ runner.os }}-${{ matrix.project }}-pw-${{ hashFiles('**/.playwright-version') }}    - name:  Install deps      if: steps.cache-primes.outputs.cache-hit != 'true'      run: make install    - name:  Install ${{ matrix.project }} with Playwright      if: steps.playwright-cache.outputs.cache-hit != 'true'      run: npx playwright install --with-deps ${{ matrix.project }}    - name:  Playwright tests      run: npx playwright test --project=${{ matrix.project }}      env:        DEBUG: pw:api,pw:browser*    - name:  Upload test results      if: failure()      uses: actions/[email protected]      with:        name: playwright-report-${{ matrix.project }}        path: playwright-report

What I've changed in GitHub Workflow for Playwright tests?

  • I use a matrix with three browser engines to run the jobs simultaneously. Instead of ubuntu-latest, I run WebKit tests on macos-12 runner. I had to make the change to avoid fragile tests. I noticed that the page.goto() API works differently on Linux and macOS for the WebKit engine. WebKit didn't wait for load event, which causes failures from time to time. At the same time, I've never caught such a problem locally on my MacBook. If you're interested, the ticket is still open in the Playwright bug tracker.
  • The fail-fast parameter is set to false to wait for all testing jobs to complete, even if one of them fails for some reason.
  • I believe it is essential to specify timeout-minutes for E2E tests so that the tests will crash if the infrastructure hangs or behaves incorrectly.
  • I use the .nvmrc file inside the repository to specify the NodeJS version. DRY all over the place!
  • Configured aggressive cache for npm dependencies and binary files with browsers. I added a special target to Makefile to calculate the Playwright browsers hash:
bump-playwright:  npm install --save-dev @playwright/test  npx playwright --version > .github/.playwright-version

After executing make bump-playwright command, the latest PW version will be installed and the value of the installed release will be written to the .github/.playwright-version file. GitHub Actions calculate the hash for the cache from the file content. Example file:

$ cat .github/.playwright-versionVersion 1.29.2
  • Each runner installs only one browser and it's pretty handy cause the HTML report will have the results for that browser easy to navigate and spot the issues. This distinction also allows you to have an idea of the test execution speed in each browser and to notice flickering tests specific to a particular engine.
  • Playwright tests run with enabled DEBUG, so that in case of a crash, you can quickly get additional information for diagnostics.
  • In contrast to the recommended configuration from the documentation, I prefer to upload artifacts with the report only in case of test failures. I don't see the point in storing those artifacts on the cloud, given how often the tests are run.

I shared my findings for configuring E2E playwright tests in GitHub Actions. Feel free to ask questions about the example GitHub workflow.

What Playwright tricks do you use in your daily work? Would love to hear about your hidden gems .

Cover photo by Denny Mller.


Original Link: https://dev.to/drakulavich/fast-and-reliable-end-to-end-tests-with-playwright-on-github-actions-2mkh

Share this article:    Share on Facebook
View Full Article

Dev To

An online community for sharing and discovering great ideas, having debates, and making friends

More About this Source Visit Dev To