Added extremely basic support for CRLs
(extremely untested, generated with zed's agent stuff + some basic cleanup)
:shipit: No regressions found.
@woodruffw a) what am I supposed to do about this datetime lint thing? b) are you up for giving this a review to see if it looks like a vaguely sane architecture and is in line with teh rest of the codebase's design patterns? c) is there somewhere I can see a table of how every harness does on this testcase?
c) is there somewhere I can see a table of how every harness does on this testcase?
Huh, that's a weird one: I think it's complaining because the other files do from datetime import datetime and this one only does import datetime. So if you change it to the former it should be fine.
b) are you up for giving this a review to see if it looks like a vaguely sane architecture and is in line with teh rest of the codebase's design patterns?
Sure, can do tonight 🙂
c) is there somewhere I can see a table of how every harness does on this testcase?
The easiest way to do that is probably to run the various make test-* targets and look at the results locally with make site, although there's still some kind of pathological behavior in the site building due to what I think is a perf regression in mkdocs or mkdocs-material. I haven't had time to chase that down, though.
I feel like it'd be helpful to be able to get that table in CI. I'm developing on a mac where the docker based harnesses are not going to run.
I feel like it'd be helpful to be able to get that table in CI. I'm developing on a mac where the docker based harnesses are not going to run.
Yeah, I need to look into that. Right now we generate a subset of the table for the regression detection, so actually that might be sufficient in this case (since there should be fewer testcases than the table limit). That'll show up once you regenerate the limbo.json.
New testcases
There are new testcases in this change.
openssl-3.3.3
| Testcase | Expected Result | Actual Result | Context |
|---|---|---|---|
crl::revoked-certificate-with-crl |
FAILURE | SUCCESS | None |
gocryptox509-go1.23.9
| Testcase | Expected Result | Actual Result | Context |
|---|---|---|---|
crl::revoked-certificate-with-crl |
FAILURE | SKIPPED | CRLs not supported |
openssl-3.4.1
| Testcase | Expected Result | Actual Result | Context |
|---|---|---|---|
crl::revoked-certificate-with-crl |
FAILURE | SUCCESS | None |
rust-webpki
| Testcase | Expected Result | Actual Result | Context |
|---|---|---|---|
crl::revoked-certificate-with-crl |
FAILURE | SKIPPED | CRLs are not supported by this API |
pyca-cryptography-45.0.2
| Testcase | Expected Result | Actual Result | Context |
|---|---|---|---|
crl::revoked-certificate-with-crl |
FAILURE | SKIPPED | testcase skipped (explicit unsupported feature) |
openssl-3.5.0
| Testcase | Expected Result | Actual Result | Context |
|---|---|---|---|
crl::revoked-certificate-with-crl |
FAILURE | SUCCESS | None |
gnutls-certtool-3.8.3
| Testcase | Expected Result | Actual Result | Context |
|---|---|---|---|
crl::revoked-certificate-with-crl |
FAILURE | SKIPPED | CRLs not supported yet |
certvalidator-0.11.1
| Testcase | Expected Result | Actual Result | Context |
|---|---|---|---|
crl::revoked-certificate-with-crl |
FAILURE | SKIPPED | testcase skipped (explicit unsupported feature) |
openssl-3.2.4
| Testcase | Expected Result | Actual Result | Context |
|---|---|---|---|
crl::revoked-certificate-with-crl |
FAILURE | SUCCESS | None |
rustls-webpki
| Testcase | Expected Result | Actual Result | Context |
|---|---|---|---|
crl::revoked-certificate-with-crl |
FAILURE | SKIPPED | CRLs are not supported by this API |
openssl-3.0.16
| Testcase | Expected Result | Actual Result | Context |
|---|---|---|---|
crl::revoked-certificate-with-crl |
FAILURE | SUCCESS | None |
openssl-1.1
| Testcase | Expected Result | Actual Result | Context |
|---|---|---|---|
crl::revoked-certificate-with-crl |
FAILURE | SUCCESS | None |
grumble, so this doesn't even fail correctly in openssl
I think the comment has all the info I want, I think it'd be more readable if the table was inverted though: one table per test case, with the table having all the harnesses.
I think the comment has all the info I want, I think it'd be more readable if the table was inverted though: one table per test case, with the table having all the harnesses.
True, IDR why I did it like that. I'll refactor it to one table tonight or tomorrow.