[Feature] allow client certificate selection and settings from Javascript

[frzme]

Similarly to https://github.com/puppeteer/puppeteer/issues/540

Currently when navigating to a page that requires client certificates and client certificates are available a popup is shown in Firefox and Chrome which asks to select which certificate to use. It would be beneficial to provide an API to select the correct certificate to use (or use none).

[yyvess]

You can apply same workaround as for puppeteer. First client certificate popup should only open if you have more than one client certificate installed. You can remove client certificate to avoid the popup be open.

Secondly you can inject dynamically an certificate as is =>

const fs = require('fs');
const request = require('request');
const browser = await chromium.launch();
const ctx = await browser.newContext();
await ctx.route('**/*', (route, req) => {
    const options = {
      uri: req.url(),
      method: req.method(),
      headers: req.headers(),
      body: req.postDataBuffer(),
      timeout: 10000,
      followRedirect: false,
      agentOptions: {
        ca: fs.readFileSync('./certs/ca.pem'),
        pfx: fs.readFileSync('./certs/user_cert.p12'),
        passphrase: fs.readFileSync('./certs/user_cert.p12.pwd'),
      },
    };
    let firstTry = true;
    const handler = function handler(err, resp, data) {
      if (err) {
        /* Strange random connection error on first request, do one re-try */
        if (firstTry) {
          firstTry = false;
          return request(options, handler);
        }
        console.error(`Unable to call ${options.uri}`, err.code, err);
        return route.abort();
      } else {
        return route.fulfill({
          status: resp.statusCode,
          headers: resp.headers,
          body: data,
        });
      }
    };
    return request(options, handler);
  });

[gepd]

You can apply same workaround as for puppeteer. First client certificate popup should only open if you have more than one client certificate installed. You can remove client certificate to avoid the popup be open.

Hey @yyvess do you have an example of how remove client certificate programmatically?

[yyvess]

Hi @gepd, as I know you cannot remove it programmatically. But if you have only one certificate installed, the browser should not ask you to select one. Note that the popup is not present when you run your test in background.

[mxschmitt]

Hey folks!

We are currently evaluating and digging into this feature and have a few questions so it will fit your needs:

• Is selecting a certificate out of the given ones from the operating system programatically enough? (like e.g. a clientCertificate event where you can select out of the loaded ones or a mapping from hosts to client certificates)
• Do you want to load a custom certificate manually or is it already in your operating system certificate storage?
• If you add them manually, are you using PEM or PFX certificate file format?
• Is it enough if its on browser.launch level? (context level with multiple certificates on each context would be the alternative)
• Do you want to validate against a CA?
• (your use-case would also help a lot)

Thank you! ❤️ And sorry for the ping.

cc @osmenia, @SMN947, @delsoft, @dcr007, @inikulin, @BredSt, @matthias-ccri, @nlack, @sdeprez, @bramvanhoutte, @yyvess, @gepd

[sdeprez]

👋   great news! So to talk about our own usecase:

• programmatical selection is enough yes
• on the fly loading would be very nice, because for us the certificate is ultimately provided by our customers and as such is dynamic, so we would like to avoid tooling to constantly add and remove certificates from the OS storage
• only PEM
• browser.launch is enough, we have one browser per session (and only one certificate per session)
• no need to validate, probably some certificates will be self-signed I can imagine

[yyvess]

Hi,

Thanks for your interest on this features !

On my case applications use client certificate to do the authentication of users.

Today as work around we inject certificate as I described before on this thread. Only small issue that still is that when we run test in no headless, the browser requests a certificate selection on a popup. We can select manually any certificate proposed by the browser as after an other is inject by the proxy . But it's force to add a wait to gave time to select manually one other tests faild.

• Is selecting a certificate out of the given ones from the operating system programatically enough?

Will more easy to pass a list (or map<key,file>) of certificate files to Playwright during the initialization. On CI our test are executed on a dedicated docker image. But when a developer run test manually, it's run directly on his Os.

• If you add them manually, are you using PEM or PFX certificate file format?

Converting certificate is not an issue.

• Is it enough if its on browser.launch level?

We have some test that require different users with different privilege. Then we should be able to select or switch certificate between two different tests.

• Do you want to validate against a CA?

Not really needed

[gepd]

Hey!

Thanks for considerate this!

in my use case:

• programmatical selection is enough
• like @yyvess on the fly loading would be very nice
• We use PFX
• we have one browser and certificate per session
• If we can validate would be cool, but not mandatory

Our use case; we need to download a file from a gov website, this website only works creating a session with a PFX certificate. Each user doesn't need to download that file very often, but we have many users, that is why on the fly loading is good for our use case

Thanks again!

[callmemagnus]

The solution @yyvess proposed works for most use cases.

I have an issue with multipart POST request, like file uploads. The new request does not contain the multipart file content. We need the formData content available, if we would like this solution to be sustainable.

[arekzelechowski]

I'd like to add my thoughts to this discussion:

Is selecting a certificate out of the given ones from the operating system programatically enough? (like e.g. a clientCertificate event where you can select out of the loaded ones or a mapping from hosts to client certificates) For our use case OS certs are not enough. The event idea seems awesome.

Do you want to load a custom certificate manually or is it already in your operating system certificate storage? We'd like to do that manually.

If you add them manually, are you using PEM or PFX certificate file format? Actually it does not matter. Converting between PEMs and PFX/P12 is quite easy. However, take a look at https.Agent and tls.connect() implementations. Using the same interface could be beneficial - ca, cert and key fields in PEM format

Is it enough if its on browser.launch level? (context level with multiple certificates on each context would be the alternative) For our use case it is enough

Do you want to validate against a CA? Yes, we do. We have our own CA that issues both client and server certificates.

(your use-case would also help a lot)

We use Playwright in two ways: E2E tests and automated scripts. Current solution for E2E is ok, but automated scripts runtime is somewhat problematic. Our intention is to write scripts with APIs as much as possible, however, some of our legacy apps do not have these. In that case, we use playwright as an workaround. All of these legacy apps are behind HTTPS (sometimes with certs issued by local CA) and client certs auth.

One more important factor is Docker. We run E2E tests with Gitlab CI runner that uses docker containers. We would also like to run these scripts in docker containers as you do not need to install node, playwright, no updates required. This is why asking OS for cert is not sufficient.

PS. Using custom TLS certs for APIs is not a problem at all.

[Xen0byte]

Hi @mxschmitt, I've raised on the dotnet repo the issue of not being able to import certificates or to use already-existing certificates in headless mode (https://github.com/microsoft/playwright-dotnet/issues/1601). That issue got merged into this one, which is when I started having some concerns that this somewhat unrelated issue would be bundled with the actual problem that I've raised on the dotnet repo. For my organisation, certificate selection is a nice-to-have, and this can be easily worked around via the registry in Windows and via config on UNIX-based systems, in lieu of programmatic means, however not being able to import certificates in dotnet or to use already-existing certificates in headless mode is currently a blocker on one of our products. Hope this information helps towards development on this; for me and my organisation, this would be the most important feature since the first release on the stable channel. Cheers.

[fargraph]

This is also our use case, including docker:

Is selecting a certificate out of the given ones from the operating system programatically enough? (like e.g. a clientCertificate event where you can select out of the loaded ones or a mapping from hosts to client certificates) For our use case OS certs are not enough. The event idea seems awesome.

Do you want to load a custom certificate manually or is it already in your operating system certificate storage? We'd like to do that manually.

If you add them manually, are you using PEM or PFX certificate file format? Actually it does not matter. Converting between PEMs and PFX/P12 is quite easy. However, take a look at https.Agent and tls.connect() implementations. Using the same interface could be beneficial - ca, cert and key fields in PEM format

Is it enough if its on browser.launch level? (context level with multiple certificates on each context would be the alternative) For our use case it is enough

Do you want to validate against a CA? Yes, we do. We have our own CA that issues both client and server certificates.

(your use-case would also help a lot)

We use Playwright in two ways: E2E tests and automated scripts. Current solution for E2E is ok, but automated scripts runtime is somewhat problematic. Our intention is to write scripts with APIs as much as possible, however, some of our legacy apps do not have these. In that case, we use playwright as an workaround. All of these legacy apps are behind HTTPS (sometimes with certs issued by local CA) and client certs auth.

One more important factor is Docker. We run E2E tests with Gitlab CI runner that uses docker containers. We would also like to run these scripts in docker containers as you do not need to install node, playwright, no updates required. This is why asking OS for cert is not sufficient.

PS. Using custom TLS certs for APIs is not a problem at all.

[fargraph]

If you came here looking for a solution and you saw the solution from @yyvess, you may also be looking for where to implement that solution. It took me a while trying to integrate it in the global config, or even a setup/teardown file, but I just couldn't find the correct apis that would let me do that. So I finally landed on creating a custom fixture that provides a context and then also re-exports all of @playwright/test. The pattern is similar to testing-library in which they recommend creating fixtures as needed to reduce code.

I have a global config that I'll include for completeness, but it really doesn't do anything special. The real magic is in the fixture which is used by the test for all of the Playwright imports.

Note, if you need to do MFA Authentication and want to use the documented persistent context, but can't figure out how to tell playwright to launch a persistent context, you can use this same solution, instead of using the provided context in the fixture, you would just create one in the fixture and it will be provided to any tests that use the fixture.

// playwright.config.ts

import { PlaywrightTestConfig } from '@playwright/test'

const config: PlaywrightTestConfig = {
    testDir: 'tests',
    use: {
        channel: 'chrome',
        ignoreHTTPSErrors: true,
    },
}
export default config

// caAuthenticationFixture.ts

import { test as base, chromium, BrowserContext } from '@playwright/test'
import fs from 'fs'
import request, { CoreOptions } from 'request'

export const test = base.extend({
    context: async ({ context }, use) => {

       // I use the context that is created using my base config here, just adding the route, but you could also create
       // a context first if you needed even more customizability.

        await context.route('**/*', (route, req) => {
            const options = {
                uri: req.url(),
                method: req.method(),
                headers: req.headers(),
                body: req.postDataBuffer(),
                timeout: 10000,
                followRedirect: false,
                agentOptions: {
                    ca: fs.readFileSync('./certs/ca.pem'),
                    pfx: fs.readFileSync('./certs/user.p12'),
                    passphrase: fs.readFileSync('./certs/user.p12.passwd', 'utf8'),
                },
            }
            let firstTry = true
            const handler = function handler(err: any, resp: any, data: any) {
                if (err) {
                    /* Strange random connection error on first request, do one re-try */
                    if (firstTry) {
                        firstTry = false
                        return request(options, handler)
                    }
                    console.error(`Unable to call ${options.uri}`, err.code, err)
                    return route.abort()
                } else {
                    return route.fulfill({
                        status: resp.statusCode,
                        headers: resp.headers,
                        body: data,
                    })
                }
            }
            return request(options, handler)
        })
        use(context)
    },
})

export * from '@playwright/test'

// login.spec.ts

import { test, expect } from './certAuthenticationFixture'  // <-- note the import of everything from our fixture.

test('login test', async ({ page, context }) => {
  
    await page.goto('https://my.page.net')

    const title = page.locator('title')

    await expect(title).toHaveText('My Title')
})

[philga7]

Hi, @fargraph:

In using Node v14.18.0, were you (or anyone else) able to get beyond the Node-level (but not Node error) message SELF_SIGNED_CERT_IN_CHAIN Error: self signed certificate in certificate chain?

In other automation frameworks that I've used, they typically had cert auth baked in, so having to rely upon Node wasn't necessarily an issue.

Thanks!

[fargraph]

I believe that error is bypassed by this line in the playwright config:

// playwright.config.ts

import { PlaywrightTestConfig } from '@playwright/test'

const config: PlaywrightTestConfig = {
    testDir: 'tests',
    use: {
        channel: 'chrome',
        ignoreHTTPSErrors: true, // <-- bypass the SELF_SIGNED_CERT_IN_CHAIN Error
    },
}
export default config

See this issue for reference: https://github.com/microsoft/playwright/issues/2814

[philga7]

@fargraph: Unfortunately, no, ignoreHTTPSErrors: true does not deal with the SELF_SIGNED_CERT_IN_CHAIN error.

This is something likely very simple that I'm simply overlooking.

Appreciate the help.

[randomactions]

While the request is being implemented, is there a way to simply click "Cancel" in the "Select a certificate" dialog?

[radu-nicoara-bayer]

We would also need just a way to cancel the popup. I am currently implementing a login tests, and it always gets stuck when the browser is asking for a certificate, that we are trying to cancel ether way. Just that playwright does not offer the possibility when using page.on('dialog', dialog => dialog.dismiss());. Nothing happens.

[enrialonso]

Hi guys, few days ago I involved on the same problem and want to share here my workaround for this.

Important, this workaround is really weak and need improve to attach the real goal: select the certificate programmatically

• Only work on headless=false
• Only tested on chromium browser

In this repository explain better the workaround >> playwright-auto-select-certificates-for-url

Basically manage the policies of the chromium browser in the path /etc/chromium/policies/managed and filter the certificated used for a URL inside of the json policy.

{
  "AutoSelectCertificateForUrls": ["{\"pattern\":\"*\",\"filter\":{}}"]
}

The browser launch read this policy and automatic select the cert for the url.

Any mistake pls sorry and sorry for my english.

[langri-sha]

~I've been using the same workaround from @yyvess successfully, but on the latest release started getting the error again.~

:point_up: just realized that request and page.request use a separate context entirely.

[IanVS]

@philga7, best way I've found so far to avoid that error, is to use NODE_TLS_REJECT_UNAUTHORIZED=0 npm test to temporarily set that environment variable for my test command. I'm curious if there's a better way, but that seems to do the trick.

[Xen0byte]

It would be nice if implementing this feature would get some traction.

[amirabramovich]

How can we inject it in Java?

[baermathias]

Any news in terms of implementing this feature? @mxschmitt

[ansghof]

That would be a really helpful feature. Cypress also did it recently but I don't want to go back to Cypress.

[ChrisSku]

I really love playwright, but for enterprise software where authenticating with SSL certificates is common, it is hard to use playwright. Of course you can use the solutions described here to overwrite the route behaviour and adding an https agent, but it is error prune when you make changes and it clutters the the reports full with events and makes error taking hard.

A solution as Cypress has added would be nice https://docs.cypress.io/guides/references/client-certificates.

I also don't want to move to cypress because handling login over multi domains is a pain there despite the whole cypress chain.

[konradekk]

I believe that error is bypassed by this line in the playwright config:

…

See this issue for reference: #2814

I’ve got this covered by following instructions from mr. wheeler (Playwright uses Ubuntu underneath)

Highly recommended – better than ”ignoring HTTPS errors” or anything like that!

[Kremliovskyi]

One more vote here, the workaround is not working, keycloak auth is logging out immediately after loging and redirect to home page. Please implement this https://docs.cypress.io/guides/references/client-certificates in playwright, this is blocker for using playwright at all.

[gwenne]

Would really be great to get a proper solution for this. It's blocking our testing, massively.

[janostgren]

Very important feature. Go for similar solution as Cypress. It is ok to just support the Node runtime.

[Tmodurr]

@janostgren any idea of Cypress provides this support (as defined here)