The hours of Protractor are numbered: The Angular team has announced to stop development by the end of the year. So I started looking for alternatives and found Playwright.
Image published by
Gerd Altmann
on Pixabay
under a Pixaybay license.
I could also have picked up my Cypress tests again, but after a first short love affair, my enthusiasm for Cypress cooled down mainly because it's huge. Running npm install in a Cypress-powered project takes a while. The first download of Playwrite is also heavy, but it feels more lightweight at first glance. It downloads three browsers: Firefox, Chrome, and Safari. By default, the test runs with all three browsers. Nice!
Getting started
Getting to speed is simple. Just follow these instructions and play around with the framework for an hour. If you're familiar with other e2e test frameworks, the source code won't puzzle you:
import { expect, test } from '@playwright/test'; test.beforeEach(async ({ page }) => { await page.goto('http://localhost:4200'); }); test.describe('Smoketest', () => { test('at least one page renders', async ({ page }) => { await page.waitForSelector('.visiblePageIsLoading'); await page.locator('[data-page-number]="19" > canvas'); await expect(page.locator('small')) .toContainText('Copyright hint: the PDF file has been published by'); await expect(page.locator('#primaryPageRotateCw')) .not.toBeDisabled(); await expect(page.locator('[data-page-number="19"]')) .toBeVisible(); await expect(page.locator(':nth-match(.loadingInProgress,1)')) .not.toBeVisible(); expect(await page.screenshot()) .toMatchSnapshot({ maxDiffPixelRatio: 0.001 }); }); });When running the tests from the command line (npx playwright test), the tests run in a headless browser.
Debugging
What I particularly like is the debugger. It's a Visual Studio Code plugin and works like a charm.
VS Code allows you to run the test with a single click. In this case, the tests run in a headless browser, just as they do in the command line. However, error messages show in your source code window. While the test runs, it highlights the line under test in the editor. You always know what's going on.
Running the debugger requires you to use the context menu. The debugger opens a browser window, giving you more control during the debugging session.
Screenshots and visual comparisons
Playwright allows you to take snapshots during the test. Even better, you can compare your screenshots against pre-recorded screenshots:
expect(await page.screenshot()) .toMatchSnapshot({ maxDiffPixelRatio: 0.001 });The maDiffPixelRation is essential. Modern browsers cut corners when rendering, so the result is not 100% reproducible. Often, there are differences in subpixels - something you don't notice, but it allows the browser to render faster. So it makes sense to allow for a certain number of different pixels. In any case, anti-aliased pixels are ignored by default if I understand the documentation correctly.
Under the hood, Playwright uses the pixelmatch library. That's a node.js counterpart of jLineUp, but it's a lot smaller and a better match in a node.js environment.
Visual comparisons are black-box tests
Screenshot tests are controversial. Most developers consider them too much work because they're always broken. I don't think so: if you're writing a web application, the visual page counts, not the HTML statements and CSS rules rendering the page. So I consider most UI tests bad because they test the wrong things. The whole idea of automated tests is to make changes hard. That's logical: you don't want to add errors to your application. Unfortunately, most tests freeze the implementation instead of the behavior.
I want my test to check my application does the right thing. What I do not wish to do is fixate the implementation. That kind of test is detrimental. Next time you abstain from refactoring code because it breaks too many tests, you'll know what I'm talking about.
That's what I like about visual comparisons: they are black-box tests. Plus, it takes almost no time to write them.
Comparing components visually
By the way, we can do something about the "broken all the time" bit. Instead of taking a screenshot of the entire page, it's better to take a screenshot of the component under test, thus increasing the chances that only the test relating to your source code change breaks. In Angular application you can use the component selector as CSS selector. This code snippets tests the three components <pdf-toolbar>, <pdf-sidebar>, and <pdf-document-container>:
Wrapping it up
I've been playing with Playwright for only a few hours, and it already feels good. Its simple API, VS Code integration, and speed convinced me within minutes. Plus, testing three browsers simultaneously and headlessly is a big plus, too.
For the sake of completeness, there are alternatives: NightwatchJS, Cypress, Testcafe, WebdriverIO, and - if you're not interested in running client-side JavaScript - even good old Jest. I've been working with all of them except WebdriverIO in the past, and they do the trick. However, for the moment, I'll stick to Playwright because of the VS Code integration and the debugger.