playwright
playwright copied to clipboard
microsoft
[Feature]: Custom AsymmetricMatchers
🚀 Feature Request
The current custom matcher extend function does not seem to allow custom asymmetric matchers.
E.g. fixtures.ts
import { expect as baseExpect } from '@playwright/test';
export { test } from '@playwright/test';
export const expect = baseExpect.extend({
async toBeNumberOrNull(received) {
const assertionName = `toBeNullOrType`;
let pass: boolean;
let matcherResult: any;
const expected = 'Null or Number';
let isNumber = false;
let isNull = false;
try {
await baseExpect(received).toEqual(baseExpect.any(Number));
isNumber = true;
} catch (e: any) {
matcherResult = e.matcherResult;
isNumber = false;
}
try {
await baseExpect(received).toEqual(null);
isNull = true;
} catch (e: any) {
matcherResult = e.matcherResult;
isNull = false;
}
pass = isNull || isNumber;
const message = pass
? () => this.utils.matcherHint(assertionName, undefined, undefined, { isNot: this.isNot }) +
'\n\n' +
`Expected: ${this.isNot ? 'not' : ''}${this.utils.printExpected(expected)}\n` +
(matcherResult ? `Received: ${this.utils.printReceived(matcherResult.actual)}` : '')
: () => this.utils.matcherHint(assertionName, undefined, undefined, { isNot: this.isNot }) +
'\n\n' +
`Expected: ${this.utils.printExpected(expected)}\n` +
(matcherResult ? `Received: ${this.utils.printReceived(matcherResult.actual)}` : '');
return {
message,
pass,
name: assertionName,
expected,
actual: matcherResult?.actual,
};
}
});
This works when writing an symmetric assertion such as:
test('Test matcher', async () => {
await expect(1).toBeNumberOrNull();
});
This does not work when writing an asymmetric assertion such as:
test('Gets order', async ({ request }) => {
const response = await request.get('/order');
expect(response.status()).toBe(200);
const response = await response.json();
expect(response.data).toMatchObject({
id: expect.any(String),
name: expect.any(String),
item: {
id: expect.any(Number),
price: expect.any(Number),
discount: expect.toBeNumberOrNull(), // this custom asymmetric matcher is not currently supported
},
});
});
Example
No response
Motivation
This will allow custom matchers to be used asymmetrically, e.g. where expected values could be a few different values such as the example above.
Technically asymmetric matchers are already supported, it's just that their type definitions are not correctly retained. You can work around the issue by explicitly adding type casts on your export const expect.
Your example:
import { expect as baseExpect } from '@playwright/test'
type ValueOf<T> = T[keyof T]
// Playwright does not export the type of the return value of the custom matcher
type MatcherReturnType = Exclude<ReturnType<ValueOf<Parameters<typeof baseExpect.extend>[0]>>, Promise<unknown>>
const toBeNumberOrNull = (received: unknown): MatcherReturnType => {
const pass = typeof received === 'number' || received === null
const message = pass ? () => `Expected '${received}' not to be a number or null` : () => `Expected ${received} to be a number or null`
return { pass, message, actual: received }
}
const customMatchers = {
/**
* Check if the received value is a `Number` or `null`.
*/
toBeNumberOrNull,
// for better readability when using as asymmetric matcher
numberOrNull: toBeNumberOrNull,
}
const customExpect = baseExpect.extend(customMatchers)
type OmitFirstArg<F> = F extends (x: any, ...args: infer P) => infer R ? (...args: P) => R : never
type AsymmetricMatcher = Record<string, unknown>
type CustomMatchers = {
[K in keyof typeof customMatchers]: K extends string
? OmitFirstArg<(...args: Parameters<(typeof customMatchers)[K]>) => AsymmetricMatcher>
: never
}
// Playwright expect.extend() misses the type info of custom asymmetric matchers,
// so we need to explicitly cast it here...
export const expect = customExpect as typeof customExpect & CustomMatchers
Usage:
expect(42).toBeNumberOrNull() // OK
expect(null).toBeNumberOrNull() // OK
expect({ foo: 42, bar: null }).toEqual(expect.objectContaining({
foo: expect.numberOrNull(),
})) // OK
The above workaround works until playwright 1.47.1. ~In latest main it does no longer work due to the recently merged https://github.com/microsoft/playwright/pull/32366 which changes how expect.extend(…) is implemented.~ It's now fixed again in main via https://github.com/microsoft/playwright/pull/32795 . 🎉
However, I've created https://github.com/microsoft/playwright/pull/32740 which should make the above workaround unnecessary.
@dgozman as you mentioned on #32740 you…
…are not sure we'd like to keep this API for asymmetric matchers going forward.
Can you give an example why this API for asymmetric matchers is not desirable? I like to better understand your concerns. IMHO, extensibility is one of the great features of playwright.
Can you give an example why this API for asymmetric matchers is not desirable? I like to better understand your concerns. IMHO, extensibility is one of the great features of playwright.
Could you give us a couple of use cases where they would make most sense to you? We'd like to make sure there are no reasonable alternatives that would not involve asymmetric matchers.
Can you give an example why this API for asymmetric matchers is not desirable? I like to better understand your concerns. IMHO, extensibility is one of the great features of playwright.
Could you give us a couple of use cases where they would make most sense to you? We'd like to make sure there are no reasonable alternatives that would not involve asymmetric matchers.
I didn't ask the original question, but for any custom matcher it is useful to have both symmetric and asymmetric versions, so that it can be used from within other matchers like toMatchObject or other custom matchers.
Example:
expect(obj).toMatchObject({
prop1: 'value1',
prop2: expect.myCustomMatcher(...)
);
I'd love to hear alternative ways to do this that don't need this feature.
Really expect this to be a feature to make playwright more friendly for API testing. It is more concise to write this then extract each and every field separately. It bursts readability, productivity.
expect(obj).toMatchObject({
prop1: 'value1',
prop2: expect.myCustomMatcher(...)
);
Is there some technical limitation to fully support custom asymetric matchers with types ? Why do you doubt to support asymmetric matchers?
I'm trying to type an asymmetrical custom matcher and the workaround keeps giving me nevers. I just wish this was supported out of the box. My example:
import { expect as baseExpect } from "@playwright/test";
import { format, parseISO } from "date-fns";
type ValueOf<T> = T[keyof T];
// Playwright does not export the type of the return value of the custom matcher
type MatcherReturnType = Exclude<
ReturnType<ValueOf<Parameters<typeof baseExpect.extend>[0]>>,
Promise<unknown>
>;
const toMatchISODate = (
actual: string,
expected: string
): MatcherReturnType => {
const actualIsoDate = format(parseISO(actual), "yyyy-MM-dd");
const check = actualIsoDate === expected;
if (check) {
return {
message: () => "passed",
pass: true,
};
} else {
return {
message: () =>
`toMatchISODate() assertion failed.\nYou expected '${actualIsoDate}' to be '${expected}'`,
pass: false,
};
}
};
const customMatchers = {
toMatchISODate,
};
const customExpect = baseExpect.extend(customMatchers);
/** @see https://github.com/microsoft/playwright/issues/32562 for this type workaround for asymmetric matchers */
type OmitFirstArg<F> = F extends (x: unknown, ...args: infer P) => infer R
? (...args: P) => R
: never;
type AsymmetricMatcher = Record<string, unknown>;
type CustomMatchers = {
[K in keyof typeof customMatchers]: K extends string
? OmitFirstArg<
(...args: Parameters<(typeof customMatchers)[K]>) => AsymmetricMatcher
>
: never;
};
export const expect = customExpect as typeof customExpect & CustomMatchers;
However this still gives me type errors stating toMatchISODate is type never:
await expect(jsonBlob).toEqual({
startDate: expect.toMatchISODate("2025-01-21"),
endDate: expect.toMatchISODate("2025-02-14"),
});
This is needed because the dates are encoded as ISO Date/Time in UTC which may be one day behind or ahead in local time so trying to string match fails. By parsing and then formatting the date field we always get the correct local date. We do this in a number of places and may even be deep down inside a json object which is why we don't want to have to check individual fields.
@J5 I am using this, it works matchersTypes.d.ts
import { expect as baseExpect } from '@playwright/test';
import { customMatchers } from './customMatchers';
declare global {
namespace PlaywrightTest {
interface Matchers<R, T> extends CustomMatcher {}
interface AsymmetricMatchers extends CustomMatcher {}
}
}
type OmitFirstArg<F> = F extends (x: any, ...args: infer P) => infer R
? (...args: P) => R
: never;
type CustomMatcher = {
[K in keyof typeof customMatchers]: K extends string
? OmitFirstArg<(...args: Parameters<(typeof customMatchers)[K]>) => void>
: never;
};
const customExpect = baseExpect.extend({
...customMatchers,
});
export const expect = customExpect as typeof customExpect & CustomMatcher;
customMatchers.ts
import { DateTime, Duration, DurationUnits } from 'luxon';
import _ from 'lodash';
import { DurationLikeObject } from 'luxon/src/duration';
import validator from 'validator';
import { expect } from '@matchers';
export const customMatchers = {
/**
* Verify that actual date is not older than current date by amount passed as a duration. The actual date should be in the past.
*
* This check will fail
* const someDateToVerify = DateTime.now().minus({ minutes: 4 }).toJSDate();
* expect(someDateToVerify).dateTimeNotLessThan({ minutes: 3 });
*
* with error
* Error: Mon Aug 26 2024 13:20:40 GMT+0200 (Central European Summer Time) is not in a valid Date format or less than {"minutes":3}, actual time: 2024-08-26T13:24:40.461+02:00.
*
* This check will fail because actual time is in the future to the current time
* const someDateToVerify = DateTime.now().plus({ minutes: 1 }).toJSDate();
* expect(someDateToVerify).dateTimeNotLessThan({ minutes: 3 });
*
* This check will pass
* const someDateToVerify = DateTime.now().minus({ minutes: 2 }).toJSDate();
* expect(someDateToVerify).dateTimeNotLessThan({ minutes: 2, seconds: 5 });
*
* @param received js Date object, milliseconds since UNIX Epoch or js Date date time string that is eligible to Date.parse(received)
* it will be sent to the function automatically.
* @param duration DurationLikeObject, can be that complex {hours: 1, minutes: 15, seconds: 50} or just {minutes: 3}
*/
dateTimeNotLessThan(
received: string | Date | number,
duration: DurationLikeObject
) {
const failure = {
message: () =>
`${received} is not in a valid Date format or less than ${JSON.stringify(
duration
)}, actual time: ${DateTime.now().toISO()}.`,
pass: false,
};
const success = {
message: () =>
`${received} is in a valid Date format and less than ${JSON.stringify(
duration
)}, actual time: ${DateTime.now().toISO()}.`,
pass: true,
};
let dateTimeReceived: DateTime;
if (typeof received === 'number') {
dateTimeReceived = DateTime.fromMillis(received);
} else if (received instanceof Date) {
dateTimeReceived = DateTime.fromJSDate(received);
//eslint-disable-next-line unicorn/prefer-number-properties,unicorn/no-negated-condition
} else if (!isNaN(Date.parse(received))) {
dateTimeReceived = DateTime.fromJSDate(new Date(received));
} else {
return failure;
}
const timeDiffInMillis = dateTimeReceived
.diffNow(Object.keys(duration) as DurationUnits)
.toMillis();
return timeDiffInMillis <= 0 &&
timeDiffInMillis >= ~Duration.fromDurationLike(duration).toMillis() + 1
? success
: failure;
},
/**
* Verify that actual string is a string with all letters in lower case
* Example:
* expect(oktaUser!.profile!.email!).toBeLowerCase();
*
* @param received actual string is passed to the function automatically
*/
toBeLowerCase(received: string | any) {
const failure = {
message: () => `${received} is not a string or is not in a lower case.`,
pass: false,
};
const success = {
message: () => `${received} is a string and is in a lower case.`,
pass: true,
};
if (!_.isString(received)) {
return failure;
}
const lowerCaseString = received.toLowerCase();
return received === lowerCaseString ? success : failure;
},
/**
* Verify that actual string is a string with all letters in upper case
* Example:
* expect(oktaUser!.profile!.email!).toBeUpperCase();
*
* @param received actual string is passed to the function automatically
*/
toBeUpperCase(received: string | any) {
const failure = {
message: () => `${received} is not a string or is not in a upper case.`,
pass: false,
};
const success = {
message: () => `${received} is a string and is in a upper case.`,
pass: true,
};
if (!_.isString(received)) {
return failure;
}
const upperCaseString = received.toUpperCase();
return received === upperCaseString ? success : failure;
},
/**
* Verifies that provided actual value is NOT null, undefined, NaN and object, array, string,
* or jQuery-like collection with a length 0
*
* expect(0).valueShouldNotBeEmpty(); -> passes
* expect('text').valueShouldNotBeEmpty(); -> passes
* expect([1]).valueShouldNotBeEmpty(); -> passes
* expect({a: 123}).valueShouldNotBeEmpty(); -> passes
* expect([]).valueShouldNotBeEmpty(); -> fails
* expect({}).valueShouldNotBeEmpty(); -> fails
* expect('').valueShouldNotBeEmpty(); -> fails
* expect(null).valueShouldNotBeEmpty(); -> fails
* expect(undefined).valueShouldNotBeEmpty(); -> fails
* expect(NaN).valueShouldNotBeEmpty(); -> fails
*
* @param received actual value is passed to the function automatically
*/
valueShouldNotBeEmpty(received: any) {
const result =
_.isNil(received) ||
_.isNaN(received) ||
(!_.isNumber(received) && _.isEmpty(received));
const failure = {
message: () => `Actual argument is empty: ${received}.`,
pass: false,
};
const success = {
message: () => `${received} is not empty.`,
pass: true,
};
return result ? failure : success;
},
/**
* Verifies that actual string is a correct UUID.
* expect({
* a: '6ec0bd7f-11c0-43da-975e-2a8ad9ebae0b',
* }).toMatchObject({
* a: expect.isValidUUID()
* });
* expect('6ec0bd7f-11c0-43da-975e-2a8ad9ebae0b').isValidUUID();
* @param received actual string that is expected to be UUID. Actual value is passed to the function automatically
*/
isValidUUID(received: string) {
const result = validator.isUUID(received);
const failure = {
message: () => `Actual argument is not UUID string: ${received}.`,
pass: false,
};
const success = {
message: () => `${received} is a correct UUID string.`,
pass: true,
};
return result ? success : failure;
},
/**
* Verifies that actual string is a correct email address.
* @param received actual string that is expected to be email address. Actual value is passed to the function automatically
*/
isValidEmail(received: string) {
const result = validator.isEmail(received);
const failure = {
message: () =>
`Actual argument is not email address string: ${received}.`,
pass: false,
};
const success = {
message: () => `${received} is a correct email address string.`,
pass: true,
};
return result ? success : failure;
},
/**
* Use .toBeOneOf when checking if actual value is a member of a given Array.
* expect(1).toBeOneOf([1, 2, 3]);
* expect(1).toBeOneOf([expect.any(Number), null]);
* @param received actual value
* @param list array of expected values
*/
toBeOneOf: (received: any, list: any[]) => {
const failure = {
message: () => `Actual argument <${received}> is not equal to expected.`,
pass: false,
};
const success = {
message: () => `Actual argument <${received}> is equal to expected.`,
pass: true,
};
let result = false;
for (const listElement of list) {
try {
expect(listElement).toEqual(received);
result = true;
} catch {
//ignore
}
}
return result ? success : failure;
},
/**
* Check if the string is a valid date. e.g. [2002-07-15, new Date()].
*/
toBeValidDate(received: any) {
const result = validator.isDate(received);
const failure = {
message: () => `Actual argument is not Date string: ${received}.`,
pass: false,
};
const success = {
message: () => `${received} is a correct Date string.`,
pass: true,
};
return result ? success : failure;
},
/**
* Use .toBeNil when checking a value is null or undefined.
*
* @param received actual value
*/
toBeNil(received: any) {
const result = received === undefined || received === null;
const failure = {
message: () => `Actual argument is not null or undefined: ${received}.`,
pass: false,
};
const success = {
message: () => `${received} is null or undefined.`,
pass: true,
};
return result ? success : failure;
},
/**
* Use .fail when you want exception to fail.
*
* @param received actual value, will be passed anyway
* @param message message why or what is failing
*/
fail(received: any, message: string) {
return {
message: () => `${message}.`,
pass: false,
};
},
/**
* Compare two objects excluding certain keys. Keys can be nested.
*
* Example:
* const actual = { 'a': 1, 'b': {f: 123, e: 6}, 'c': 3 };
* const expected = { 'a': 1, 'b': {f: 456, e: 6}, 'c': 8 };
* expect(actual).toEqualExcludingKeys(expected, ['b.f', 'c']);
*
* @param received actual object, will be passed anyway
* @param expected object to compare
* @param excluding object keys to ignore during comparisons
*/
toEqualExcludingKeys(received: any, expected: any, excluding: string[]) {
try {
expect(_.omit(received, excluding)).toEqual(_.omit(expected, excluding));
return {
message: () =>
`Actual argument ${received} is equal to expected ${expected}.`,
pass: true,
};
} catch (error) {
return {
message: () =>
`Actual object is not equal to expected excluding ${excluding}. \n${error}`,
pass: false,
};
}
},
};
Still didn't work for me but I was just able to hand roll an interface that works
type AsymmetricCustomMatcher = {
toMatchISODate: (expected: string) => unknown;
};
export const expect = customExpect as typeof customExpect &
AsymmetricCustomMatcher;