go crypto/x509: TestPlatformVerifier/revoked

I'm seeing the TestPlatformVerifier/revoked_leaf test in crypto/x509 failing at tip, 1.19.4, and 1.18.9 on darwin/arm64 running macOS 13.1 (22C65):

$ go test crypto/x509
--- FAIL: TestPlatformVerifier (1.39s)
    --- FAIL: TestPlatformVerifier/revoked_leaf (0.18s)
        root_darwin_test.go:116: unexpected verification error: got "x509: “revoked.badssl.com” certificate is expired", want "x509: “revoked.badssl.com” certificate is revoked"
FAIL
FAIL	crypto/x509	3.321s
FAIL

CC @rolandshoemaker.

Dec 21 '22 18:12 dmitshur

This appears to be a combination of https://github.com/chromium/badssl.com/issues/515, and a change in the verification error precedence in macOS 13 (previously revoked seemed to take precedence over expiration, whereas now it's reversed).

This just further shows the importance of #52108, which I still don't have a good answer for. Probably something to seriously look at in the new year.

Dec 21 '22 18:12 rolandshoemaker

Also further motivation for #35678 / #49055, since that test is skipped in short mode so doesn't actually run on any builders we have today. (This explains why it wasn't reported by the darwin-amd64-13 builder earlier.)

Dec 21 '22 18:12 dmitshur

The new darwin-amd64-longtest builder is indeed catching this! 🙃

(And leaf_missing_SCTs, too: https://build.golang.org/log/8815b334c89791bc3c30410757d4c1020e927e96)

Apr 06 '23 19:04 bcmills

Change https://go.dev/cl/482165 mentions this issue: crypto/x509: skip broken darwin root tests

Apr 10 '23 15:04 gopherbot

crypto/x509: TestPlatformVerifier/revoked_leaf fails on macOS 13.1