stxnext
WIP: Quote credentials (as per pymongo documentation)
When using a password tin the pymongo URI such that the password contains special characters, authentication with MongoDB fails.
The pymongo documentation explains how to escape these special characters (see https://pymongo.readthedocs.io/en/stable/examples/authentication.html).
The password could technically be escaped directly in the URI on the
command-line, but this is very awkward and it would be quite convenient
if pymongo-migrate could do this automatically.
isn't this inherently broken?
special characters needed encoding are : / ? # [ ] @ so how urllib was supposed to parse such mangled URI in the first place?
for example: mongodb://[email protected]@P/a/ss?word/db
seems like it would be better to allow providing username/password as optional args
The password could technically be escaped directly in the URI on the command-line, but this is very awkward and it would be quite convenient if pymongo-migrate could do this automatically.
pymongo-migrate cannot do this automatically because it would double encode credentials that are already encoded, thus invalidating them.
Also, urllib.parse.urlparse will not work as expected on an URL with a "unescaped" password.
You must provide a valid (i.e. already escaped) URL on the command line.
@rooterkyberian Thanks for the feedback!
isn't this inherently broken? special characters needed encoding are : / ? # [ ] @ so how urllib was supposed to parse such mangled URI in the first place?
Indeed, it is. I encountered this while writing tests after pushing this (very prematurely :-). It's most obvious with # because everything after that interpreted as the fragment.
seems like it would be better to allow providing username/password as optional args
Yeah, I think so.
@pawel-slowik Thanks for the feedback!
pymongo-migratecannot do this automatically because it would double encode credentials that are already encoded, thus invalidating them
Indeed, urllib.parse.quote_plus() is not idempotent and it's not possible to detect if it has already been applied.
And clearly, it's now obvious that I cannot "fix" the password when it's passed to pymongo-migrate in the URI because an unescaped password creates an invalid URI anyways.
Are you open to other approaches for accepting and encoding credentials such as a dedicated command-line arguments or environment variables for the username and password? Or should I take your comment to mean that you think pymongo-migrate should not try to encode credentials at all?
While I don't think is essential, I'm open to any solution as long as its not too complicated or solves more problems than this particular corner case.
# solution with user/password args
pymongo-migrate migrate -u 'mongodb://localhost/test_db' --user User --password Password -m tests/migrations
# solution with URL generator - a pymongo-migrate-url script which only purpose is to construct mongodb connection URL
pymongo-migrate migrate -u "$(pymongo-migrate-url mongodb://localhost/test_db --user User --password Password)" -m tests/migrations
# solution with Config file in which we can have a support for extra arguments, such as user/password
pymongo-migrate migrate --config pymongo-migrate.ini
# solution where we just ignore this problem and force anyone encountering it to do encode it manually
One could argue that transmitting user/password through either ENVs or command line options is dangerous. We could try to kill two birds with one stone here. "Safe" options to provide application with credentials are:
- prompt
- config file - this is little bit less secure, but overall it can be useful for everyone
OK, I'll try to whip up another implementation based on what you proposed.
FWIW, I'm running pymongo-migrate in a Kubernetes job to which all secrets are passed as environment variables. My current workaround for this issue is roughly equivalent to your URL generator script:
# solution with URL generator - a pymongo-migrate-url script which only purpose is to construct mongodb connection URL
pymongo-migrate migrate -u "$(pymongo-migrate-url mongodb://localhost/test_db --user User --password Password)" -m tests/migrations
However, I would actually prefer the option of invoking pymongo-migrate directly as it allows more control in the job specification (in contrast with baking a shell script with the above snippet into the Docker image).
I'll update this PR with a solution based on this using with environment variables and we can discuss it again once I get that working:
# solution with user/password args
pymongo-migrate migrate -u 'mongodb://localhost/test_db' --user User --password Password -m tests/migrations