Add support for mongodb+srv connection string

[Zer0-Tolerance]

trafficstars

Hi Marcel, I'm trying to connect to MongoDB Atlas instance but it's required to use a connection string like this one: Parsing error in url 'mongodb+srv://user:[email protected]/?retryWrites=true&w=majority' but the current driver don'ts know how to handle this connection and complains with this: 2024-02-25 19:11 UTC 58.886576 [F][1][Uri][380]: Parsing error in url 'mongodb+srv://host.o9jgs.mongodb.net/?retryWrites=true&w=majority' Would you be able to add support for this ? Or tell me how I can workaround this ?

Thanks a lot.

[MARTIMM]

the 'mongodb+srv' part throws the problem. I may be able to add a rule for it but can not test if there will be another problem caused by it. Also, will there be more extensions to take care of? Anyways I try to fix it and see what happens.

[MARTIMM]

I've published a new version 0.44.0. Must be available soon. Let me know if it works.

[Zer0-Tolerance]

Thanks a lot I'll do the testing today. But I think you need to handle the SSL connection as well because it is required by MongoDB Atlas.

the 'mongodb+srv' part throws the problem. I may be able to add a rule for it but can not test if there will be another problem caused by it. Also, will there be more extensions to take care of? Anyways I try to fix it and see what happens.

[Zer0-Tolerance]

btw you can also do some testing with the free tier MongoDB Atlas, the typical URI looks like this: mongodb+srv://<user>:<password>@host.o9jgs.mongodb.net/?retryWrites=true&w=majority&appName=yourcluster-cluster and from what I've been reading to far it is discovering the real servers with a DNS SRV request + a DNS TXT request to get some parameter for the connection. more info here

[Zer0-Tolerance]

First feedback on the new version, it doesn't seems to connect successfully. This is probably due to the lack of SSL / TLS support ?

[MARTIMM]

I do not have it implemented so that is a problem. Don't know how to do that in a short time. Need to investigate.

[MARTIMM]

I will do it slowly, step by step.

• Checks on parameters according to this document
• Try to get server polling working using the SRV DNS records following this document: initial dns seedlist discovery
• Apply some URI options to the result like srvMaxHosts and srvServiceName.

I also need to attend to two other projects though.

[MARTIMM]

Small update..... Done the above and some checking on URI options but I am now stuck for some time, setting up a connection to a server. The server starts properly using some certificate and key but I cannot convince OpenSSL to do the work I need. Errors like 'Private key does not match the public certificate' from OpenSSL and 'No SSL certificate provided by peer; connection rejected' on the server.

[Zer0-Tolerance]

Hi Marcel, thanks for the update, I didn't know Mongosh was doing some kind of certificate pinning. I'll try to do some more testing this week-end.

[MARTIMM]

hi @Zer0-Tolerance, at the moment I was doing some other work on other distros. It helps to empty my mind by doing something else before returning to this task.

[MARTIMM]

Taraaaah!. There is hope! My little test works. Now I only need to understand it. Of course, it must then be coded into the driver......

[Zer0-Tolerance]

Hi Marcel, I'm glad to hear that.

[MARTIMM]

I have pushed version 0.45.0 today. The URI accepts the tls, tlsCAFile, and tlsCertificateKeyFile options. I have tested it with homemade certificates so I am curious if you get it running with proper certificates and keys.

[Zer0-Tolerance]

Hi , this is great ! I'll test the new version asap.

[Zer0-Tolerance]

Hi Marcel, Just did the test, It fails with this error

$dburl="mongodb+srv://<user>:<password>@host.o9jgs.mongodb.net/?retryWrites=true&w=majority&appName=yourcluster-cluster";
--------------------------------------------------------------------------------
2024-05-24 22:30 UTC

43.664272 [F][1][Uri][516]: Found server 'test-dev-cluster-shard-00-00.o9jgs.mongodb.net' must be in same domain 'test-dev-cluster.o9jgs.mongodb.net'
Found server 'test-dev-cluster-shard-00-00.o9jgs.mongodb.net' must be in same domain 'test-dev-cluster.o9jgs.mongodb.net'
  in sub fatal-message at /Users/.rakubrew/versions/moar-2024.03/share/perl6/site/sources/02D61B5AC84C560843B1B47BFD76B5376184893F (MongoDB::Log) line 331
  in method get-srv-hosts at /Users/.rakubrew/versions/moar-2024.03/share/perl6/site/sources/A52B53AB41F36AF34A71F1BC5E4ECE5BFDA68301 (MongoDB::Uri) line 516
  in block  at /Users//.rakubrew/versions/moar-2024.03/share/perl6/site/sources/A52B53AB41F36AF34A71F1BC5E4ECE5BFDA68301 (MongoDB::Uri) line 290
  in submethod BUILD at /Users/.rakubrew/versions/moar-2024.03/share/perl6/site/sources/A52B53AB41F36AF34A71F1BC5E4ECE5BFDA68301 (MongoDB::Uri) line 280
  in submethod BUILD at /Users/.rakubrew/versions/moar-2024.03/share/perl6/site/sources/63FF6B179C583222B22CF1D14143D549E357423A (MongoDB::Client) line 205
  in method new at /Users//.rakubrew/versions/moar-2024.03/share/perl6/site/sources/63FF6B179C583222B22CF1D14143D549E357423A (MongoDB::Client) line 137
  in block <unit> at <unknown file> line 1
  in any <main> at /Users//.rakubrew/versions/moar-2024.03/bin/../share/perl6/runtime/perl6.moarvm line 1
  in any <entry> at /Users//.rakubrew/versions/moar-2024.03/bin/../share/perl6/runtime/perl6.moarvm line 1

[MARTIMM]

I do not see any certificate / key specs in your URI. Does it work like that? If so, I need to find a way to have more than one domain to test such a situation. I only have a 'localhost'.

[MARTIMM]

Shame on me, I looked in the wrong spot asking Google to find errors it could not possibly find. It was a test in the Uri class throwing this error. Should have looked better :-|.

I have repaired the problem but I cannot try this out myself. I will publish this version, 0.45.1 and hopefully, this one will work for you.

[Zer0-Tolerance]

Hi, there is a different error now :

--------------------------------------------------------------------------------
2024-06-15 23:55 UTC

39.752469 [F][1][Uri][516]: Found server 'test-dev-cluster-shard-00-01.o9jgs.mongodb.net' must be in same domain 'test-dev-cluster.o9jgs.mongodb.net'
Found server 'test-dev-cluster-shard-00-01.o9jgs.mongodb.net' must be in same domain 'test-dev-cluster.o9jgs.mongodb.net'
  in sub fatal-message at /Users/.rakubrew/versions/moar-2024.05/share/perl6/site/sources/02D61B5AC84C560843B1B47BFD76B5376184893F (MongoDB::Log) line 331
  in method get-srv-hosts at /Users/.rakubrew/versions/moar-2024.05/share/perl6/site/sources/A52B53AB41F36AF34A71F1BC5E4ECE5BFDA68301 (MongoDB::Uri) line 516
  in block  at /Users/.rakubrew/versions/moar-2024.05/share/perl6/site/sources/A52B53AB41F36AF34A71F1BC5E4ECE5BFDA68301 (MongoDB::Uri) line 290
  in submethod BUILD at /Users/.rakubrew/versions/moar-2024.05/share/perl6/site/sources/A52B53AB41F36AF34A71F1BC5E4ECE5BFDA68301 (MongoDB::Uri) line 280
  in submethod BUILD at /Users/.rakubrew/versions/moar-2024.05/share/perl6/site/sources/63FF6B179C583222B22CF1D14143D549E357423A (MongoDB::Client) line 205
  in method new at /Users/.rakubrew/versions/moar-2024.05/share/perl6/site/sources/63FF6B179C583222B22CF1D14143D549E357423A (MongoDB::Client) line 137
  in block <unit> at mongop5.raku line 40

to connect to the cluster with mongosh I just do this so I don't need to provide any cert or CA: mongosh "mongodb+srv://test-dev-cluster.o9jgs.mongodb.net/" --apiVersion 1 --username admin

[MARTIMM]

I am not sure but I get the feeling that you are not running the latest version. It should be version 0.45.1. The error above is from a previous version.

Furthermore, I assume the atlas server is started with the server TLS option disabled. However, the raku modules will automatically turn on TLS when srv records are used. So, when the encrypted connection isn't needed you have to explicitly turn TLS off … tls=false … in the URI.

[Zer0-Tolerance]

same error with the version 0.45.1 and tls=false

[MARTIMM]

Sorry that it still fails, I'll look into it

[MARTIMM]

I clipped too little from the owner domain (of the SRV records) where the server would be in. should have been o9jgs.mongodb.net instead of test-dev-cluster.o9jgs.mongodb.net. So, version 0.45.2 is in the making and perhaps available later on. I do not know how long it takes for zef to find in from CPAN.

[Zer0-Tolerance]

Quick update with latest version 0.45.2, new error now:

2024-06-22 23:07 UTC
42.885552 [F][1][Uri][538]: Only one TXT record is accepted for this domain 'test-dev-cluster.o9jgs.mongodb.net'

[MARTIMM]

I think I got this test right, but I'm curious about which TXT records there are for the specific domain. Could you please send these to me?

According to mongodb:

Like the standard connection string, the DNS seed list connection string supports specifying options as a query string. With a DNS seed list connection string, you can also specify the following options via a TXT record:

• replicaSet
• authSource

You may only specify one TXT record per mongod instance. If multiple TXT records appear in the DNS and/or if the TXT record contains an option other than replicaSet or authSource, the client will return an error.

You can find this text in the connection-string reference.

[MARTIMM]

Any luck? You could try dig test-dev-cluster.o9jgs.mongodb.net TXT.

[Zer0-Tolerance]

Hi, please find the output from dig darkrecon-dev-cluster.o9jgs.mongodb.net TXT:

; <<>> DiG 9.10.6 <<>> darkrecon-dev-cluster.o9jgs.mongodb.net TXT
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 18451
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 4, ADDITIONAL: 2

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;darkrecon-dev-cluster.o9jgs.mongodb.net. IN TXT

;; ANSWER SECTION:
darkrecon-dev-cluster.o9jgs.mongodb.net. 60 IN TXT "authSource=admin&replicaSet=DarkRecon-DEV-cluster-shard-0"

;; AUTHORITY SECTION:
mongodb.net.		172710	IN	NS	ns-1647.awsdns-13.co.uk.
mongodb.net.		172710	IN	NS	ns-1043.awsdns-02.org.
mongodb.net.		172710	IN	NS	ns-761.awsdns-31.net.
mongodb.net.		172710	IN	NS	ns-430.awsdns-53.com.

;; ADDITIONAL SECTION:
ns-761.awsdns-31.net.	5520	IN	A	205.251.194.249

;; Query time: 22 msec
;; SERVER: 10.4.0.1#53(10.4.0.1)
;; WHEN: Sat Jul 06 10:42:40 CEST 2024
;; MSG SIZE  rcvd: 291

[MARTIMM]

Fixed a minor something, caused by copy / paste code. It is a pity that I can not use a server of yours safely to test things more fully. Now it is a small step at a time until the next bug is found. Sorry for the inconvenience.

[MARTIMM]

Fixed a minor bug caused by copy/paste of code. It is a pity that I cannot use a server of yours safely so I can test everything more fully.