[Feature] Component testing experiments #12799
Comments
|
@Rich-Harris I was mentioning this experiment to you over Discord. Svelte examples are in https://github.com/microsoft/playwright/tree/main/tests-components, not yet in the examples folder. Curious what you think about Svelte version and how we can make it shine. |
|
@yyx990803: I was wondering if you are interested in looking at the Vue examples:
We are not using vue-test-utils (we bring our own automation and assertions), but it feels like there are bits of it that we should be using. Similarly, it'd be nice to provide out-of-the box experience that does not require managing testable components gallery. |
|
Out of interest, have you considered Storybook integration as an option? That might solve the gallery problem at the same time. |
Yes, we have considered it, but we could not satisfy our requirements (mainly Experience section). |
|
Is Web Components (using lit) in scope? Seems so, just checking. |
I am curious about this too |
|
Nice! Just a few questions/comments/input:
Does that mean this example will not work: import { MyClass } from './some-path';
import { MyComponent } from './my-component'
test('My test', async ({ mount }) => {
const myComplexType = new MyClass(); // Is this class instantiated in node and then tried to be serialized to the page?
const comp = await mount(<MyComponent someProp={myComplexType} />);
});Or how is the component instantiated in the page?
Maybe the same question as above, but I do not follow. Do we need to register all components that are used in the tests? What if I have two components with the same name? How can I register both test.jsIf I, as a user, need to manually register all components in a special file I would much more like a solution that looks something like this: https://github.com/kivra/playwright-react#option-2-recomended-right-now-import-an-external-component-component-under-test ie. creating an extra file which will be executed in the browser. Test executionIt would be nice if Snapshot testing (nice to have)I would define snapshot (image) testing as a special type of component testing. The idea is to take an image snapshot of a component and then test that it has not changed. This can of course be achieved by a normal component test: test('Counter component should look the same on desktop', async ({ mount, page }) => {
await page.setViewportSize({ width: 1600, height: 1200 });
await mount(<div className="screenshot-wrapper"><Theme><Counter /></Theme></div>);
expect(await page.screenshot()).toMatchSnapshot();
});
test('Counter component should look the same on mobile', async ({ mount, page }) => {
await page.setViewportSize({ width: 900, height: 1200 });
await mount(<div className="screenshot-wrapper"><Theme><Counter /></Theme></div>);
expect(await page.screenshot()).toMatchSnapshot();
});But this becomes quite cumbersome. It would be nice to have an api that looks something like this: // MyText.snap.tsx
import React from 'react';
import { MyText } from '../MyText';
export const tests = [{
name: 'Counter on desktop',
size: { width: 1600, height: 1200 }
render: () => <Counter />
}, {
name: 'Counter on mobile',
size: { width: 900, height: 1200 }
render: () => <Counter />
}]And add a path to a wrapper component ( We have, at my company, a few hundred tests with this syntax with this (hacky) addon to Playwright: https://github.com/kivra/playwright-react The documentation is not complete and the code might be hard to read but I would gladly improve it if it can be helpful. EDIT: Is it possible to put component code into multiple files? Something like this: // util/base.ts
import { test as base } from '@playwright/test';
import { Wrapper } from './some-path';
type MyFixtures = {
snapshot: (component: JSX.Element) => Promise<void>;
};
export const test = base.extend({
snapshot: async ({ page, mount }, use) => {
const snapshot = async (component: JSX.Element) => {
await mount(<Wrapper>{component}</Wrapper>); // <--- Put jsx here as well
// ... some more code
}
await use(snapshot);
},
});// my-component.spec.ts
import { test } from './uil/base';
test('Counter component', async ({ snapshot }) => {
await snapshot(<Counter />);
});If so, that would be really nice! |
Correct, you can only do: await mount(<MyComponent ..>...</MyComponent>);
or
await mount(MyComponent, { props: ...., slots: ... });
This is not solved atm, but I can imagine that we can do something about it. Is that a real use case? (I assume it is). For now, you can work around it via using separate endpoints for those,
That's what actually happens,
I would expect that you put them into a separate Playwright project:
or tag them
We are improving our visual regression story. I'm sure we can make it ergonomically sound while maintaining general shape of the tests.
No, this would not work atm. All that Thanks for all the great questions and comments! |
|
Thanks for your answers!
Just so we are talking about the same thing. My question was if it would be possible to pass non-serialized props to a component from a test. Like a class instance. If I understand you correctly this is not possible? For example, let's say you have the following react component: // my-component.tsx
inport type { Person } from './person';
interface Props {
person: Person;
}
export function MyComponent({ person }: Props) {
return <p>{person.fullName}</p>
}and the following class: // person.ts
export class Person {
first: string;
last: string;
constructor(first: string, last: string) {
this.first = first;
this.last = last;
}
get fullName() { return `${this.first} ${this.last}` }
}If I understand it correctly it is not possible to write tests like this, since // my-component.spec.tsx
import { test, expect } from '@playwright/experimental-ct-react/test'
import { MyComponent } from './my-component'
import { Person } from './person'
test('Super Mario', async ({ mount }) => {
const person = new Person('Super, 'Mario');
const component = await mount(<MyComponent person={person}></MyComponent>)
await expect(component).toContainText('Super Mario')
})
test('Luigi Mario', async ({ mount }) => {
const person = new Person('Luigi, 'Mario');
const component = await mount(<MyComponent person={person}></MyComponent>)
await expect(component).toContainText('Super Mario')
})// test.ts
import register from '@playwright/experimental-ct-react/register'
import { MyComponent } from './my-component'
register({ MyComponent })So in order to test the component above, I need to create two new components in a test file: // my-component-test.ts
const mario = new Person('Super, 'Mario');
export const MyComponentWithSuperMario = () => {
return <MyComponent person={mario}></MyComponent>
}
const luigi = new Person('Super, 'Luigi');
export const MyComponentWithLuigiMario = () => { /** ... */ }// tests.ts
import register from '@playwright/experimental-ct-react/register'
import { MyComponentWithSuperMario, MyComponentWithLuigiMario } from './my-component-test'
register({
MyComponentWithSuperMario,
MyComponentWithLuigiMario
})And then in the test: // my-component.spec.tsx
import { test, expect } from '@playwright/experimental-ct-react/test'
import { MyComponentWithSuperMario, MyComponentWithLuigiMario } from './my-component-test'
test('Super Mario', async ({ mount }) => {
const component = await mount(<MyComponentWithSuperMario />)
await expect(component).toContainText('Super Mario')
})
test('Luigi Mario', async ({ mount }) => {
const component = await mount(<MyComponentWithLuigiMario />)
await expect(component).toContainText('Super Mario')
})So the file structure will be something like this: Sorry for the long questing I just want to see if I get this right.
I guess this can be done with vite (and webpack, and probably other budlers as well): // test.ts
const modules = import.meta.globEager('./**/*-test.tsx');
register(Object.values(modules).reduce(...));
Will this work with React as well? Even though React is not using native events? For example, take this component: // my-button.tsx
interface Props {
onClick: () => void;
}
export function MyButton({ onClick }: Props) {
return <button onClick={onClick}>Click me</button>
}Will this just work in React? // my-button.spec.tsx
import { test, expect } from '@playwright/experimental-ct-react/test'
import { MyButton } from './my-button'
test('My button', async ({ mount }) => {
let clickCount = 0
const component = await mount(<MyButton onClick={() => clickCount++}></MyButton>)
await component.click()
expect(clickCount).toEqual(1)
})
|
Correct, this is not possible, your tests are running in Node, while your components run inside the browser.
Yes, that would work.
Yes, that would work.
Yes, before doing that we'd like to validate the overall story though. But the idea is to have a vite (and webpack) plugins that would perform registration. It might be a bit more involved than on your snippet (we need to consider multiple components in the same .tsx file, etc), but the overall idea is the same.
Yes, we do that here: https://github.com/microsoft/playwright/blob/main/packages/html-reporter/src/chip.spec.tsx#L46
Yes, it will. |
|
These are just some early thoughts. I'm sharing them here before they escape me. Currently, the PWT architecture is 1) run the CLI 2) playwright loads a Playwright vite plugin together with the framework plugin for vite 3) Vite plugin checks for changed test files from cache directory 4) Playwright reruns new tests + changed files. If I'm developing, I already have Vite running...which has a watcher running...and a detailed module graph of all files changing. Why not reuse all of those in Playwright Test? It might be interesting to expose some Playwright Test API for other tools bring in, rather than (or in addition to) bringing in the other tools into Playwright. Playwright test does it's own transforms for I spent a few hours hacking on a Vite plugin that runs playwright test when files change. It's somewhat crude at the moment, and it reruns all tests if any file changes but some book keeping with Vite's How much of this can be achieved while maintaining the core priorities is not something I've thought too much about. TL;DR A JavaScript API to the Playwright Test runner internals would be welcome. |
We don't yet do (3) and (4), i.e. we don't yet have a watch mode in place. So far we re-build component registry upon running tests (if necessary).
I think it makes a lot of sense. However:
So while there definitely is a room for vite-centric plugin that reuses infra, we would still need to a) have a build mode which is different from your main app build mode and b) maintain vite builder for non-vite users
Oh wow, I'll definitely take a look at it. Watch is the next logical step for us, so once we have a validation of the existing component story, it will follow. Open to patches from experienced plugin authors!
I don't think you are violating anything.
You can experiment with running Playwright in a subprocess with CLI being an API. It gives you pretty much everything you need. But if we want to make it efficient, we would need to keep the browser open, which means that the watch mode would need to be deeply integrated into the test runner itself. |
|
This is now largely out-of-date, so I'm closing it. Component testing preview is scheduled to ship in v1.22 shortly. |
We are exploring the ways where Playwright can be helpful when testing framework-specific components.
Core Priorities
Proposal
See examples:
git clone [email protected]:pavelfeldman/playwright-ct-vue-example.gitTypical workflow for enabling component testing with Playwright
User creates
tests.htmlfile that defines component theming:User creates
tests.jsthat uses@playwright/experimental-ct-{react,vue,svelte}to register components for testing:User exposes
tests.htmlas atests/endpoint by means of the underlying framework.testsrouteUser adds
playwright.config.tsthat runs respective dev server before running tests.All the steps above can be handled by the script automation in either respective frameworks or Playwright itself.
Under the hood
Playwright relies upon the underlying framework to serve
tests.htmlas/tests(configurable) endpoint. This is a blank page that includes all the components from thetests.jsbundle. It exposes APIs that Playwright can use to mount these components during tests. Before each test, Playwright opens this page, it then mounts given component on that target page. Parameters, slots and events are supported.Playwright tests still run in Node, while components are instantiated in the page, so there is no fundamental difference between how Playwright works for regular e2e scenario and the components scenario. Events are rpc-ed over the process boundary, and the test script receives serialized payloads of the respective native and component events.
Challenges
tests.jstestable components registry is manual, which is a huge pain point. Framework-specific plugins are required to collect components of interest and keep them running a dev server on a separate endpoint.The text was updated successfully, but these errors were encountered: