REST API certificate based authentication

[Gunni]

Since mntner objects can be locked using key-cert objects, and Client Certificate authentication is a thing.

I hereby propose its addition to the RIPE REST API.

In addition to being able to create, update, and delete objects when authenticated you could also allow requesting the full/unfiltered/unmodified object owned by the authenticated mntner.

[eshryane]

Hello @Gunni thanks for your suggestion, I'll add this to our product backlog for further investigation.

The RIPE REST API currently only supports passwords and/or SSO credentials. This sounds like a good alternative. How likely is this to be adopted by users?

There is some related discussion around authentication in the NWI-8 proposal on the DB-WG.

[eshryane]

Hello @Gunni, it appears that support for PGP keys was dropped from Client Certificate authentication in TLS 1.3.

"The OpenPGP certificate type [RFC6091] MUST NOT be used with TLS 1.3."

https://crypto.stackexchange.com/questions/63981/why-did-tls-1-3-prohibit-pgp-authentication https://tools.ietf.org/html/rfc8446#section-4.4.2

Does it still make sense to consider this for Whois?

[Gunni]

Hey @eshryane, i did not mean PGP keys, i meant using x509 certificates.

https://www.ripe.net/manage-ips-and-asns/db/support/documentation/ripe-database-documentation/rpsl-object-types/4-3-descriptions-of-secondary-objects/4-3-3-description-of-the-key-cert-object.

[eshryane]

@Gunni Thanks for clarifying! Indeed we already support X509 key-certs, we will investigate this further.

[Gunni]

Has there been any luck with this @eshryane? x509 cert auth is very well defined and most clients I know of support it. And is infinately better than having to add a md5 password to the mntner.

[eshryane]

Hi @Gunni No update, but I haven't forgotten (we were discussing it last week in fact).

There's a number of areas we'll need to investigate in order to implement this:

• Configure the Apache httpd proxy to pass the the X509 certificate in a header

  • We don't want to create a dependency on this solution however, in case it makes it difficult to migrate away from httpd later
  • We also don't want to allow weak keys (short length or insecure algorithm) to be allowed (as it would bypass the existing SSL restrictions).
  • We need to do this without disturbing the existing use of SSL.
  • We need to test this fully, and document supported use case(s), to allow our users to actually implement it on the client side.

• Configure Whois to make use of the X509 certificate header

  • This is relatively straightforward, we can create a new credential type, and look for a matching X509 key-cert on a maintainer when doing the authentication.

It's not a huge amount of work in total, but enough that it hasn't been a priority. I agree that it's long overdue to replace MD5 (not sure what the migration path will look like however, there probably won't be huge take-up of client certificate authentication).

Now that you've reminded me, I will setup a test in our Release Candidate environment.

[eshryane]

@Gunni I've completed the httpd proxy configuration to forward a client certificate to the backend Whois server (if provided). We'll work on the necessary changes to Whois next, and deploy to RC in the next few weeks for testing.

[eshryane]

@Gunni We didn't have time to implement the necessary Whois changes before the 1.97 release, and we are waiting for a security review of our implementation, but I'm still planning to deploy this before the next RIPE meeting.

[eshryane]

@Gunni We didn't have time to add this feature to Whois 1.97.1, but we still plan to implement it. I will have an update for the DB-WG session at RIPE80 next week.

[eshryane]

@Gunni We've merged initial support for this feature.

We will include the change in the Whois 1.97.2 release, but it will be turned off in production for now, until the community have tested it, and we also plan a security review.

[daniel-brenot]

Has this been enabled for production yet or is it still for development only?

[eshryane]

Hi @daniel-brenot @Gunni We have implemented this feature, but we haven't enabled in production yet.

We are planning to do a security review, and we are now also replacing our httpd proxy layer with F5, which we will depend on for SSL termination and forwarding the client certificate information. We hope to enable this soon.

[Gunni]

We hope to enable this soon.

What about enabling it on test? Are we able to test it there? Do you have a link to docs for test db rest api?

[eshryane]

Hi @Gunni We plan to initially enable it in the Release Candidate environment, but we're going to wait until we switch the proxy layer from httpd to F5, hopefully we'll do this by the end of the year.

Once the community have tried it out, and after it's passed a security review, we plan to enable it in production.

[eshryane]

Hi @Gunni I went ahead and enabled client certificate authentication in Release Candidate, so feel free to test: https://www.ripe.net/manage-ips-and-asns/db/release-notes/rc-release-candidate-environment

You'll need to generate a key-pair, then create a keycert object with the public key and add that to a maintainer. Then you should be able to connect over TLS with your key-pair, and authenticate a Whois update as the maintainer.

Client certificate authentication should work on rest-rc.db.ripe.net and syncupdates-rc.db.ripe.net.

RC is still using httpd to proxy requests, but we are investigating client certificate support in F5 now.

[eshryane]

The whois code changes are in #637

[Gunni]

I have verified that CREATE, UPDATE, DELETE objects works with client cert auth.

Nice!

However I still get Filtered objects back when READing, even if I am authenticated while querying.

    }, {
      "name" : "auth",
      "value" : "SSO",
      "comment" : "Filtered"
    }, {

Please ensure getting unfiltered objects works when requests are authenticated!

Edit: Additionally, when doing updates like this, the update emails that normally say: Changed by SSO account: [email protected] has nothing about who updated the object.

[eshryane]

Hi @Gunni Thanks for testing! Good to hear it's working for you.

We haven't implemented client certificate authentication for reads yet (GET requests), only for updates (POST, PUT, DELETE). We will add this in the next release.

[eshryane]

Hi @Gunni this feature is still on our list, we had some problems making it optional with the F5 loadbalancer we're currently using. We plan now to move HTTPS support to the backend application and allow client certificate authentication there.

[Gunni]

Hey, it's been a year, are there any news?

[eshryane]

Hi @Gunni Happy New Year. Yes I presented on client certificate authentication at RIPE87 in Rome: https://ripe87.ripe.net/programme/meeting-plan/db-wg/

We've implemented Client Certificate authentication together with backend HTTPS in the Whois 1.109 release.

Unfortunately there is a blocking issue that's preventing us from enabling it just yet: the client certificate request in the TLS handshake causes browsers to ask the user to choose a certificate, which annoys users who don't want to use this feature. This happens not only when clicking on Whois REST API links but also for requests within single page web applications.

We're going to avoid this issue by creating a separate fully-qualified domain name just for client certificate authentication, and deploy in the next Whois release.

Apologies for the delay.

[eshryane]

Hi @Gunni I've documented how to use Client Certificate Authentication in the DB documentation with an example: https://apps.db.ripe.net/docs/Appendices/Appendix-I--Client-Certificate-Authentication/

Use the host name rest-cert.db.ripe.net instead of rest.db.ripe.net, we separated the service as a popup will appear in browsers asking users to choose a certificate when following (GET) REST API links, which is annoying.

[eshryane]

I'm closing the issue as client certificate authentication is now implemented and deployed in Whois.

[eshryane]

Reopening issue as @Gunni reported:

"But it still does not provide unfiltered fields for authenticated requests, which is what I need for to implement idempotency in clients that maintain records, I cannot see what auth sso are on a mntner so i would need to post the object every time to ensure it is as it should be."

Apologies I've confirmed this is not working, we'll fix in the next Whois release.

[MiguelAHM]

Hi @Gunni I hope you are doing well.

I am pleased to inform you that Client Certificate Authentication for lookups is now supported. With this update, you can retrieve unfiltered information for your objects using your certificate.

This feature is included in the 1.113 release.

Could you please confirm if this resolves the issue you were experiencing?

[Gunni]

It partially works!

I made my key-cert using this (RSA is so 1977) openssl req -x509 -newkey ed25519 -keyout RIPE.key -out RIPE.crt -sha512 -days 365 -nodes -subj '/C=IS/CN=...'

Then used this query: curl -v --header "Content-type: application/json" --key RIPE.key --cert RIPE.crt https://rest-cert.db.ripe.net/ripe/mntner/MNT-GUNNAR.json

What works is that i get the auth: sso emails, fantastic! However, other fields are missing/filtered, such as upd-to, mnt-nfy, and notify.

In addition to that, the last tag still says Filtered

{
      "name" : "source",
      "value" : "RIPE",
      "comment" : "Filtered"
}

It should not be filtered at all if I am authenticated for that object, so I'll give partially done.

Think of it this way, I should be able to generate an object on my side and diff every field, including those above, and remarks, and if not correct, I'd submit a PUT for it.

Notes

Things that I noticed while testing this

Documentation issue

Noticed while making my key, documentation of key-cert is outdated. Only specifies PGP keys. While online documentation is fine.

$ whois -h whois.ripe.net -- -v key-cert
% This is the RIPE Database query service.
% The objects are in RPSL format.
%
% The RIPE Database is subject to Terms and Conditions.
% See https://apps.db.ripe.net/docs/HTML-Terms-And-Conditions

The key-cert class:

      A key-cert object is a database public key certificate
      that is stored on the server and may be used with a mntner
      object for authentication when performing updates.
      Currently only PGP/GnuPG keys are supported.

key-cert:       [mandatory]  [single]     [primary/lookup key]
...

The content of the attributes of the key-cert class are defined below:

key-cert

   Defines the public key stored in the database.

     PGPKEY-<id>

     <id> is  the PGP key ID of the public key in 8-digit
     hexadecimal format without "0x" prefix.

method

   Defines the type of the public key.

     Currently, only PGP keys are supported.

Also the format of the key-cert field is not documented anywhere I could find. Turns out it must be AUTO-nnn.

Email notification

The email notification i get when someone updates a record that I watch using mnt-nfy does not contain what SSO account or x509 cert was used to perform the action.

p.s. Short OpenPGP key IDs are insecure. p.p.s. When this is done, do we get a News post for this feature? 😁

[eshryane]

Hi @Gunni thanks for your feedback!

REST API lookups are filtered by default, even if you're authenticated. You need to add the unfiltered flag to REST API lookups to request the entire object including attributes which may contain personal data. Authenticated lookups with a password work the same way. I do think it's confusing though, we will review this behaviour.

Thanks for pointing out the documentation issue, we will fix it.

I agree that short key-ids are insecure, but we are using it as a primary key, not for authentication, and it must be unique. I think using the full key fingerprint as the primary key would be unwieldy. We are open to feedback to change this though (just how we currently do it).

We do plan to deprecate MD5 hashes as announced at RIPE88, client certificate authentication is a good alternative, and this will be announced as part of the impact analysis.

Thanks again for testing our implementation.