playwright
playwright copied to clipboard
microsoft
Enable custom runners [Feature]
The objective is to use playwright for load testing. playwright will keep running the test suites in a loop for a given duration. Currently teams have a set of e2e functional set and a set of load tests. Duplicating a lot of code/work
This feature will allow people to use their functional tests to generate load.
Can you elaborate on what is currently missing in the playwright test runner? (e.g. you can already run the same suites repeatedly using repeatEach parameter, you can also generate as many copies of the tests you need still having a good control over their structure)
Let me describe a simple load test scenario I have a web application where users can 1) submit a form (takes 2 seconds to run), 2) generate a report (takes 10 seconds to run)
I want to setup a concurrent load test. Test duration : 1 hour 5 users submitting the form 2 users generating the report
While running the test, every minute I want to capture 1) the response time of each scenario. 2) The failure rate of each scenario. So that I can find any degradation in performance over time. Furthermore my web application needs some warm up time when there will be increased timeouts, I want to ignore these errors. I do not want all the concurrent users to start at the same time. I want to have a configurable ramp up. Also I want to have a configurable success criteria for the whole test run. Less than 2% errors for a scenario is considered Successful execution. I want each of these executions to use a different set of data. Sequentially taken from 2 csv file.
As you can see a single variable cannot capture these requirements. People use k6/load runner/artillery to achieve these types of tests. They usually simulate real browser, Without the real javascript in browser interaction, these tests are super fragile. Now a days the hardware has become cheap, and playwright is very efficient in using browser resources. It is possible to achieve a good load using playwright only. People will not have to maintain 2 test suites.
Thanks for detailed description! We are not planning to support custom runners but we may add support for load testing that you could leverage for such scenarios. Will keep this request open to collect more feedback.
For anyone looking for a way to run Playwright tests as load tests today:
https://github.com/artilleryio/artillery-engine-playwright
This lets you define load phases, ramps, and tracks and reports failure rates for scenarios, as well as Core Web Vital metrics (and how they change under load).
Would love any feedback from the community on this.
Why was this issue closed?
Thank you for your involvement. This issue was closed due to limited engagement (upvotes/activity), lack of recent activity, and insufficient actionability. To maintain a manageable database, we prioritize issues based on these factors.
If you disagree with this closure, please open a new issue and reference this one. More support or clarity on its necessity may prompt a review. Your understanding and cooperation are appreciated.
