app icon indicating copy to clipboard operation
app copied to clipboard

Parametrized tests having [whitespace] character in its parameter value are not properly registered in test run

Open mrogachenko-cg opened this issue 1 year ago • 3 comments

Describe the bug When parametrized test have a parameter value with whitespace (e.g. "some value") are registered in test run, all of them are grouped as the same test with 'retry', while it is expected to be registered as separate test (separate execution line).

Prerequisites

  • playwright project is properly configured to pass result into testomat
  • tests are properly synced
  • testrun is properly prepared and used for syncing result

To Reproduce Steps to reproduce the behavior:

  1. Use the following parametrized test as example (replace test case id with an actual for your project)
let valuesList3: string[] = [
	'NoSpaces',
	"Under_score",
	'I have a space',
	'Me too have a space'
];

for (const value of valuesList3) {
	test(`This is an example of parametrized test with ${value} that could have spaces @T1d3b757f`, async ({
		page,
	}) => {
    await page.goto('https://playwright.dev/');
    await page.waitForTimeout(100);
    // Expect a title "to contain" a substring.
    await expect(page).toHaveTitle(/Playwright/);
	});
}
  1. Run tests in a way that will sync result into prepared test run report (from prerequisites)
  2. Wait until test are finished
  3. Check the testomat's test run report

Expected behavior Each test execution should be registered as separate run

Actual behavior For single word parameter it is working as expected, for multy-word parameter it is registered as 'retry' for the same test with 'empty' parameters value

Screenshots image

Additional context Probably another characters (not only [whitespace] may trigger the same problem)

mrogachenko-cg avatar Mar 29 '24 14:03 mrogachenko-cg

Known issue.

We can't detect that multi-word parameter matches the test, so let's say if we have two tests:

  • User clicks link to login
  • User #{param}

If we allow param to contain whitespaces, receiving report for User clicks link to login we can match User #{param} instead.

This is why we would like to ask to use camelCase or underscore_case (underscore chars are ok as params).

DavertMik avatar Apr 02 '24 22:04 DavertMik

@DavertMik I'm sorry, it's not quite clear why you need to detect test case matching by multi-word parameters instead of matching it by provided test case id? Not sure that current issue is the same you've mentioning.

ghost avatar Apr 03 '24 05:04 ghost

re-opened per user request

TetianaKhomenko avatar Apr 03 '24 06:04 TetianaKhomenko

@DavertMik telegram-cloud-photo-size-2-5469737106424322982-y

TetianaKhomenko avatar Jun 27 '24 12:06 TetianaKhomenko

@mrogachenko-cg Hi, this feature implemented and already live on production Please check it

@TetianaKhomenko already on app.testomat.io

DavertMik avatar Jul 02 '24 10:07 DavertMik

@DavertMik Hi Yes, it is working now

ghost avatar Jul 03 '24 12:07 ghost