microsoft
[BUG] aria snapshot's regex marked as errors if the test fails
hello, i'm using aria snapshots to validate some sections and when a check fails in the error message all the regex are shown as differences making it difficult to find identify the error.
in the example below the real error is:
- - button "Colore neutro Garantisce solo modelli di colore neutro + € 1 "
+ - button "Colore neutro Garantisce solo modelli di colore neutro + € 0 "
and as you can see also all the regex are marked as errors.
if in this text i change the "1" to "0" the test is marked as passed
My code:
await expect(this.page.locator(this.boxConfigurationContainer)).toMatchAriaSnapshot(`
- text: Configura La tua Subbyx Box
- img
- text: /\\d+ % Cosa potresti ricevere/
- button "" [disabled]
- text: /Galaxy S24\\+ Risparmi \\d+,\\d+ € al mese 👍🏼 Molto probabile/
- button ""
- button "Standard Contiene usato PreLoved di qualsiasi grado estetico"
- button /Excellent Contiene solo usato PreLoved di grado estetico Eccellente \\+ \\d+,\\d+ €/:
- paragraph: /\\+ \\d+,\\d+ €/
- link " Quali sono i gradi dei PreLoved?"
- paragraph: Blocca una
- paragraph: Preferenza
- paragraph: Max 1 selezione
- button "Marca Garantisce un modello della marca selezionata + € 0 Seleziona un brand "
- button "Colore neutro Garantisce solo modelli di colore neutro + € 1 "
- button "Schermo base Garantisce uno schermo di dimensioni standard o più ridotte + € 2 "
- button "Schermo grande Garantisce uno schermo più grande rispetto al modello base + € 3 "
- button /Tanta memoria Garantisce una memoria da \\d+ GB o superiore \\+ € 2 /
- button "Continua senza preferenze Ricevi una configurazione casuale Gratis "
- button "Abbonati senza vincoli"
`);
The error:
Locator: locator('//*[@class="w-full flex flex-col justify-center items-center px-5 md:px-[10%] lg:px-[12%] py-14 bg-sbx-gray-light gap-5"]')
- Expected - 6
+ Received + 7
- text: Configura La tua Subbyx Box
- img
- - text: /\d+ % Cosa potresti ricevere/
+ - text: 34 % Cosa potresti ricevere
- button "" [disabled]
- - text: /Galaxy S24\+ Risparmi \d+,\d+ € al mese 👍🏼 Molto probabile/
+ - text: Galaxy S24+ Risparmi 13,92 € al mese 👍🏼 Molto probabile
- button ""
- button "Standard Contiene usato PreLoved di qualsiasi grado estetico"
- - button /Excellent Contiene solo usato PreLoved di grado estetico Eccellente \+ \d+,\d+ €/:
+ - button "Excellent Contiene solo usato PreLoved di grado estetico Eccellente + 3,00 €":
- - paragraph: /\+ \d+,\d+ €/
+ - text: Excellent Contiene solo usato PreLoved di grado estetico Eccellente
+ - paragraph: + 3,00 €
- link " Quali sono i gradi dei PreLoved?"
- paragraph: Blocca una
- paragraph: Preferenza
- paragraph: Max 1 selezione
- button "Marca Garantisce un modello della marca selezionata + € 0 Seleziona un brand "
- - button "Colore neutro Garantisce solo modelli di colore neutro + € 1 "
+ - button "Colore neutro Garantisce solo modelli di colore neutro + € 0 "
- button "Schermo base Garantisce uno schermo di dimensioni standard o più ridotte + € 2 "
- button "Schermo grande Garantisce uno schermo più grande rispetto al modello base + € 3 "
- - button /Tanta memoria Garantisce una memoria da \d+ GB o superiore \+ € 2 /
+ - button "Tanta memoria Garantisce una memoria da 256 GB o superiore + € 2 "
- button "Continua senza preferenze Ricevi una configurazione casuale Gratis "
- button "Abbonati senza vincoli"
Please break this down into individual cases that we can run and evaluate each scenario. Without a full, minimal repro we are unable to help.
This is already an individual case. The issue occurs when performing a validation using an ARIA snapshot. If there is a difference between the actual and expected output, the error log also highlights regex patterns as differences, which is unexpected. You can see this behavior in the log I attached.
It is an individual case, but it is very large, and you have not provided any of the backing DOM or the actual test contents. I need to be able to run the test and get the same output as you. Please provide a smaller ARIA snapshot so it's more readable.
You can reproduce the issue by running the following code.
import { test, expect } from '@playwright/test';
test('test', async ({ page }) => {
await page.goto('https://www.subbyx.com/en/box/premium?id=subbyxboxsmartphoneandroid');
await page.getByLabel('Close this notice').click();
await expect(page.locator('(//*[@class="w-full flex flex-col gap-3"])[1]')).toMatchAriaSnapshot(`
- button "Standard Contains PreLoved devices of any aesthetic gradeE"
- button /Excellent Contains only PreLoved devices with Excellent aesthetic grade \\+ €\\d+\\.\\d+/:
- paragraph: /\\+ €\\d+\\.\\d+/
- link " What are the grades of PreLoveds?"
`);
});
In this test, the only expected difference between the actual and expected output should be:
Expected: "gradeE" Actual: "grade" However, the error log also highlights regex patterns as differences, which should not happen.
To confirm that regex handling is correct, you can modify the expected value from "gradeE" to "grade", and the test will pass. This proves that the regex patterns themselves are valid and functioning correctly.
Just wanting to add on to this. Any removed attributes that you do not want to assert against are also shown in the diff even though they are not causing an error such as the following example.
- rowgroup:
- - row:
+ - row "table-cell-initials table-cell-profileData table-cell-date table-cell-testName table-cell-position table-cell-reps":
- cell "table-cell-initials":
- paragraph: QP
- - cell "table-cell-date": /\d+\/\d+\/\d+ \d+:\d+ (AM|PM)/
+ - cell "table-cell-date": 12/03/2025 8:44 AM
- - img: /66 ° 66 ° 73 ° 62 °/
+ - img: 66 ° 66 ° 62 ° 73 °
In the above example the only line causing an error is the img: as the numbers are in an incorrect order. However the report showing a diff for both the excluded properties from the row; and any regex makes it harder to identify the problem.
