community.mongodb
community.mongodb copied to clipboard
Published
20 hours ago •
ansible-collections
ansible-collections
x509 auth always fails unless connection_options: - tlsAllowInvalidHostnames=true
SUMMARY
x509 auth always fails unless: connection_options: - tlsAllowInvalidHostnames=true
ISSUE TYPE
- Bug Report
COMPONENT NAME
x509 auth handling with pymongo
ANSIBLE VERSION
From mysql replica set primary node:
ansible --version
ansible [core 2.13.6]
config file = /etc/ansible/ansible.cfg
configured module search path = ['/root/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
ansible python module location = /usr/lib/python3/dist-packages/ansible
ansible collection location = /root/.ansible/collections:/usr/share/ansible/collections
executable location = /usr/bin/ansible
python version = 3.10.6 (main, Nov 2 2022, 18:53:38) [GCC 11.3.0]
jinja version = 3.0.3
libyaml = True
From controller host:
ansible --version
ansible [core 2.13.6]
config file = /Users/dwartell/freebird/rwsecure-pod-dev/ansible/ansible.cfg
configured module search path = ['/Users/dwartell/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
ansible python module location = /opt/homebrew/Cellar/ansible/6.6.0/libexec/lib/python3.10/site-packages/ansible
ansible collection location = /Users/dwartell/.ansible/collections:/usr/share/ansible/collections
executable location = /opt/homebrew/bin/ansible
python version = 3.10.8 (main, Oct 13 2022, 09:48:40) [Clang 14.0.0 (clang-1400.0.29.102)]
jinja version = 3.1.2
libyaml = True
Pymongo version on replica set master node
python3 -m pip list | grep mongo
pymongo 4.3.3
COLLECTION VERSION
ansible-galaxy collection list | grep mong
community.mongodb 1.4.2
CONFIGURATION
DEFAULT_HOST_LIST(/Users/dwartell/freebird/rwsecure-pod-dev/ansible/ansible.cfg) = ['/Users/dwartell/freebird/rwsecure-pod-dev/ansible/inventory']
DEFAULT_PRIVATE_KEY_FILE(/Users/dwartell/freebird/rwsecure-pod-dev/ansible/ansible.cfg) = /Users/dwartell/freebird/rwsecure-pod-dev/terraform/certs/bastionhost/id_rsa
DEFAULT_REMOTE_USER(/Users/dwartell/freebird/rwsecure-pod-dev/ansible/ansible.cfg) = ubuntu
OS / ENVIRONMENT
mysql replica set primary node:
cat /proc/version
Linux version 5.15.0-1023-aws (buildd@bos02-arm64-076) (gcc (Ubuntu 11.2.0-19ubuntu1) 11.2.0, GNU ld (GNU Binutils for Ubuntu) 2.38) #27-Ubuntu SMP Thu Oct 20 16:46:19 UTC 2022
lsb_release -r
Release: 22.04
mongosh --tls --host localhost --tlsCertificateKeyFile /etc/mongo-certs/mongo-rwadmin-user-cert-and-key.pem --tlsCAFile /etc/mongo-certs/mongo-ca-cert.pem --authenticationDatabase '$external' --authenticationMechanism MONGODB-X509 --eval "db.version()"
Current Mongosh Log ID: 6387f46f68b9c93944b9c978
Connecting to: mongodb://localhost:27017/?directConnection=true&serverSelectionTimeoutMS=2000&tls=true&tlsCertificateKeyFile=%2Fetc%2Fmongo-certs%2Fmongo-rwadmin-user-cert-and-key.pem&tlsCAFile=%2Fetc%2Fmongo-certs%2Fmongo-ca-cert.pem&authSource=%24external&authMechanism=MONGODB-X509&appName=mongosh+1.6.0
Using MongoDB: 6.0.3
Using Mongosh: 1.6.0
For mongosh info see: https://docs.mongodb.com/mongodb-shell/
------
The server generated these startup warnings when booting
2022-11-30T17:58:03.060+00:00: Using the XFS filesystem is strongly recommended with the WiredTiger storage engine. See http://dochub.mongodb.org/core/prodnotes-filesystem
------
6.0.3
Controller host: Macos 13.0.1 (22A400) arm64
STEPS TO REPRODUCE
#!/usr/bin/env ansible-playbook
- name: Check replica set status for all replica set nodes are in a healthy state
hosts: mongo
gather_facts: no
tasks:
- name: check mongo replica set state
become: true
community.mongodb.mongodb_status:
login_host: localhost
replica_set: rs0
auth_mechanism: "MONGODB-X509"
tls: true
tlsCAFile: /etc/mongo-certs/mongo-ca-cert.pem
tlsCertificateKeyFile: /etc/mongo-certs/mongo-rwadmin-user-cert-and-key.pem
login_database: "$external"
# this tlsAllowInvalidHostnames option is needed or the ansible community mongo ansible collection wont work with x509
# must be a defect, works correctly with mongosh
#connection_options:
# - "tlsAllowInvalidHostnames=true"
# mongosh --tls --host localhost --tlsCertificateKeyFile /etc/mongo-certs/mongo-rwadmin-user-cert-and-key.pem --tlsCAFile /etc/mongo-certs/mongo-ca-cert.pem --authenticationDatabase '$external' --authenticationMechanism MONGODB-X509 admin
Current Mongosh Log ID: 6387f52f3e68e0f7ec7a7caa
Connecting to: mongodb://localhost:27017/admin?directConnection=true&serverSelectionTimeoutMS=2000&tls=true&tlsCertificateKeyFile=%2Fetc%2Fmongo-certs%2Fmongo-rwadmin-user-cert-and-key.pem&tlsCAFile=%2Fetc%2Fmongo-certs%2Fmongo-ca-cert.pem&authSource=%24external&authMechanism=MONGODB-X509&appName=mongosh+1.6.0
Using MongoDB: 6.0.3
Using Mongosh: 1.6.0
For mongosh info see: https://docs.mongodb.com/mongodb-shell/
------
The server generated these startup warnings when booting
2022-11-30T17:58:03.060+00:00: Using the XFS filesystem is strongly recommended with the WiredTiger storage engine. See http://dochub.mongodb.org/core/prodnotes-filesystem
------
rs0 [direct: primary] admin> db.getSiblingDB("$external").auth({mechanism: "MONGODB-X509", user: "[email protected],CN=rwadmin,OU=mongo-user.dw.dev.stealthsoft.io,O=StealthMode Inc,L=Austin,ST=Texas,C=US"})
{ ok: 1 }
rs0 [direct: primary] admin> db.getSiblingDB("$external").getUsers();
{
users: [
{
_id: '[email protected],CN=rwadmin,OU=mongo-user.dw.dev.stealthsoft.io,O=StealthMode Inc,L=Austin,ST=Texas,C=US',
userId: new UUID("8599dc7b-8eb6-42c2-82ec-d6babd85c590"),
user: '[email protected],CN=rwadmin,OU=mongo-user.dw.dev.stealthsoft.io,O=StealthMode Inc,L=Austin,ST=Texas,C=US',
db: '$external',
roles: [ { role: 'root', db: 'admin' } ],
mechanisms: [ 'external' ]
},
{
_id: '[email protected],CN=rwapp,OU=mongo-user.dw.dev.stealthsoft.io,O=StealthMode Inc,L=Austin,ST=Texas,C=US',
userId: new UUID("f64f84c2-a6d4-441b-b1d8-7d236851c471"),
user: '[email protected],CN=rwapp,OU=mongo-user.dw.dev.stealthsoft.io,O=StealthMode Inc,L=Austin,ST=Texas,C=US',
db: '$external',
roles: [ { role: 'readWrite', db: 'rwsecure' } ],
mechanisms: [ 'external' ]
}
],
ok: 1,
'$clusterTime': {
clusterTime: Timestamp({ t: 1669854524, i: 1 }),
signature: {
hash: Binary(Buffer.from("0398d62ed09262dce376cd4aac40801db40a7445", "hex"), 0),
keyId: Long("7171583554449571845")
}
},
operationTime: Timestamp({ t: 1669854524, i: 1 })
}
rs0 [direct: primary] admin>
EXPECTED RESULTS
It should work like mongosh
# mongosh --tls --host localhost --tlsCertificateKeyFile /etc/mongo-certs/mongo-rwadmin-user-cert-and-key.pem --tlsCAFile /etc/mongo-certs/mongo-ca-cert.pem --authenticationDatabase '$external' --authenticationMechanism MONGODB-X509 admin
Current Mongosh Log ID: 6387f6426814dbd6590f5161
Connecting to: mongodb://localhost:27017/admin?directConnection=true&serverSelectionTimeoutMS=2000&tls=true&tlsCertificateKeyFile=%2Fetc%2Fmongo-certs%2Fmongo-rwadmin-user-cert-and-key.pem&tlsCAFile=%2Fetc%2Fmongo-certs%2Fmongo-ca-cert.pem&authSource=%24external&authMechanism=MONGODB-X509&appName=mongosh+1.6.0
Using MongoDB: 6.0.3
Using Mongosh: 1.6.0
For mongosh info see: https://docs.mongodb.com/mongodb-shell/
------
The server generated these startup warnings when booting
2022-11-30T17:58:03.060+00:00: Using the XFS filesystem is strongly recommended with the WiredTiger storage engine. See http://dochub.mongodb.org/core/prodnotes-filesystem
------
rs0 [direct: primary] admin> db.getSiblingDB("$external").auth({mechanism: "MONGODB-X509", user: "[email protected],CN=rwadmin,OU=mongo-user.dw.dev.stealthsoft.io,O=StealthMode Inc,L=Austin,ST=Texas,C=US"})
{ ok: 1 }
rs0 [direct: primary] admin>
rs0 [direct: primary] admin> rs.status();
{
set: 'rs0',
date: ISODate("2022-12-01T00:33:17.870Z"),
myState: 1,
term: Long("30"),
syncSourceHost: '',
syncSourceId: -1,
heartbeatIntervalMillis: Long("2000"),
majorityVoteCount: 2,
writeMajorityCount: 2,
votingMembersCount: 3,
writableVotingMembersCount: 3,
optimes: {
lastCommittedOpTime: { ts: Timestamp({ t: 1669854794, i: 1 }), t: Long("30") },
lastCommittedWallTime: ISODate("2022-12-01T00:33:14.626Z"),
readConcernMajorityOpTime: { ts: Timestamp({ t: 1669854794, i: 1 }), t: Long("30") },
appliedOpTime: { ts: Timestamp({ t: 1669854794, i: 1 }), t: Long("30") },
durableOpTime: { ts: Timestamp({ t: 1669854794, i: 1 }), t: Long("30") },
lastAppliedWallTime: ISODate("2022-12-01T00:33:14.626Z"),
lastDurableWallTime: ISODate("2022-12-01T00:33:14.626Z")
},
lastStableRecoveryTimestamp: Timestamp({ t: 1669854784, i: 3 }),
electionCandidateMetrics: {
lastElectionReason: 'priorityTakeover',
lastElectionDate: ISODate("2022-11-30T17:58:24.118Z"),
electionTerm: Long("30"),
lastCommittedOpTimeAtElection: { ts: Timestamp({ t: 1669831103, i: 1 }), t: Long("29") },
lastSeenOpTimeAtElection: { ts: Timestamp({ t: 1669831103, i: 1 }), t: Long("29") },
numVotesNeeded: 2,
priorityAtElection: 1,
electionTimeoutMillis: Long("10000"),
priorPrimaryMemberId: 1,
numCatchUpOps: Long("0"),
newTermStartDate: ISODate("2022-11-30T17:58:24.224Z"),
wMajorityWriteAvailabilityDate: ISODate("2022-11-30T17:58:25.243Z")
},
members: [
{
_id: 0,
name: 'mgo-wj-a.dw.dev.stealthsoft.io:27017',
health: 1,
state: 1,
stateStr: 'PRIMARY',
uptime: 23714,
optime: { ts: Timestamp({ t: 1669854794, i: 1 }), t: Long("30") },
optimeDate: ISODate("2022-12-01T00:33:14.000Z"),
lastAppliedWallTime: ISODate("2022-12-01T00:33:14.626Z"),
lastDurableWallTime: ISODate("2022-12-01T00:33:14.626Z"),
syncSourceHost: '',
syncSourceId: -1,
infoMessage: '',
electionTime: Timestamp({ t: 1669831104, i: 1 }),
electionDate: ISODate("2022-11-30T17:58:24.000Z"),
configVersion: 1,
configTerm: 30,
self: true,
lastHeartbeatMessage: ''
},
{
_id: 1,
name: 'mgo-wj-b.dw.dev.stealthsoft.io:27017',
health: 1,
state: 2,
stateStr: 'SECONDARY',
uptime: 23713,
optime: { ts: Timestamp({ t: 1669854794, i: 1 }), t: Long("30") },
optimeDurable: { ts: Timestamp({ t: 1669854794, i: 1 }), t: Long("30") },
optimeDate: ISODate("2022-12-01T00:33:14.000Z"),
optimeDurableDate: ISODate("2022-12-01T00:33:14.000Z"),
lastAppliedWallTime: ISODate("2022-12-01T00:33:14.626Z"),
lastDurableWallTime: ISODate("2022-12-01T00:33:14.626Z"),
lastHeartbeat: ISODate("2022-12-01T00:33:16.058Z"),
lastHeartbeatRecv: ISODate("2022-12-01T00:33:17.097Z"),
pingMs: Long("0"),
lastHeartbeatMessage: '',
syncSourceHost: 'mgo-wj-a.dw.dev.stealthsoft.io:27017',
syncSourceId: 0,
infoMessage: '',
configVersion: 1,
configTerm: 30
},
{
_id: 2,
name: 'mgo-wj-c.dw.dev.stealthsoft.io:27017',
health: 1,
state: 2,
stateStr: 'SECONDARY',
uptime: 23713,
optime: { ts: Timestamp({ t: 1669854794, i: 1 }), t: Long("30") },
optimeDurable: { ts: Timestamp({ t: 1669854794, i: 1 }), t: Long("30") },
optimeDate: ISODate("2022-12-01T00:33:14.000Z"),
optimeDurableDate: ISODate("2022-12-01T00:33:14.000Z"),
lastAppliedWallTime: ISODate("2022-12-01T00:33:14.626Z"),
lastDurableWallTime: ISODate("2022-12-01T00:33:14.626Z"),
lastHeartbeat: ISODate("2022-12-01T00:33:16.122Z"),
lastHeartbeatRecv: ISODate("2022-12-01T00:33:16.116Z"),
pingMs: Long("1"),
lastHeartbeatMessage: '',
syncSourceHost: 'mgo-wj-b.dw.dev.stealthsoft.io:27017',
syncSourceId: 1,
infoMessage: '',
configVersion: 1,
configTerm: 30
}
],
ok: 1,
'$clusterTime': {
clusterTime: Timestamp({ t: 1669854794, i: 1 }),
signature: {
hash: Binary(Buffer.from("a61dcdb51a7939c6f7e972afb4797805c66f5089", "hex"), 0),
keyId: Long("7171583554449571845")
}
},
operationTime: Timestamp({ t: 1669854794, i: 1 })
}
rs0 [direct: primary] admin>
ACTUAL RESULTS
It fails unless I include these lines in the playbook yaml
connection_options:
- "tlsAllowInvalidHostnames=true"
./mongo-status.yml
PLAY [Check replica set status for all replica set nodes are in a healthy state]
TASK [check mongo replica set state]
fatal: [mgo-wj-a]: FAILED! => {
"ansible_facts": {
"discovered_interpreter_python": "/usr/bin/python3"
},
"changed": false,
"msg": "Unable to determine if auth is enabled: Traceback (most recent call last):\n File \"/tmp/ansible_community.mongodb.mongodb_status_payload_qsf9hv51/ansible_community.mongodb.mongodb_status_payload.zip/ansible_collections/community/mongodb/plugins/module_utils/mongodb_common.py\", line 279, in is_auth_enabled\n myclient['admin'].command('listDatabases', 1.0)\n File \"/usr/local/lib/python3.10/dist-packages/pymongo/_csot.py\", line 105, in csot_wrapper\n return func(self, *args, **kwargs)\n File \"/usr/local/lib/python3.10/dist-packages/pymongo/database.py\", line 805, in command\n with self.__client._socket_for_reads(read_preference, session) as (\n File \"/usr/local/lib/python3.10/dist-packages/pymongo/mongo_client.py\", line 1296, in _socket_for_reads\n server = self._select_server(read_preference, session)\n File \"/usr/local/lib/python3.10/dist-packages/pymongo/mongo_client.py\", line 1257, in _select_server\n server = topology.select_server(server_selector)\n File \"/usr/local/lib/python3.10/dist-packages/pymongo/topology.py\", line 272, in select_server\n server = self._select_server(selector, server_selection_timeout, address)\n File \"/usr/local/lib/python3.10/dist-packages/pymongo/topology.py\", line 261, in _select_server\n servers = self.select_servers(selector, server_selection_timeout, address)\n File \"/usr/local/lib/python3.10/dist-packages/pymongo/topology.py\", line 223, in select_servers\n server_descriptions = self._select_servers_loop(selector, server_timeout, address)\n File \"/usr/local/lib/python3.10/dist-packages/pymongo/topology.py\", line 238, in _select_servers_loop\n raise ServerSelectionTimeoutError(\npymongo.errors.ServerSelectionTimeoutError: not enough values to unpack (expected 2, got 1), Timeout: 30s, Topology Description: <TopologyDescription id: 6387f1fed957c5ccc26eada3, topology_type: Single, servers: [<ServerDescription ('localhost', 27017) server_type: Unknown, rtt: None, error=ValueError('not enough values to unpack (expected 2, got 1)')>]>\n"
}
Hello @davidwartell,
Thank you for the very detailed report. Can you also tell us how you generate your certs? Does it work if you use the hostname, i.e. mgo-wj-b.dw.dev.stealthsoft.io?
@fischerscode Have you experienced any issues around this?
Cheers,
Rhys
Local test environment is broken impeding progress on this. Need to fix the versions of kubernetes (and related tools) to the versions in the CI.
Even if it is not the best solution, I overcame this by adding localhost into the list of SAN(s) in the csr.
To say it in openssl words the outcome is the following:
openssl req -new -newkey rsa:2048 -nodes -keyout replica.key -out replica.csr -addext "extendedKeyUsage = serverAuth, clientAuth" -addext "subjectAltName = DNS:<node1-fqdn>, DNS:<node2-fqdn>, DNS:<node3-fqdn>, DNS:localhost.
Hope it helps.
This might be an option for a local test env... https://github.com/nektos/act