twisted
maxIdleTimeMS support
As far I can see in the code, maxIdleTimeMS option (url) is currently skipped (as well as keepalive). This is causing idle connection issues on Azure which suggests adding maxIdleTimeMS=120000 to connection string. Is it currently considered while connecting? Should I take care about these options or just let it reconnect again, re-issue query after AutoReconnect exception?
I can't say for sure, need to check the code if this setting is considered. But I think connection will reestablish automatically if dropped due to idleness.
Are your connections really idle? Do you send queries so rarely? I mean are you sure these issues are really caused by maxIdleTimeMS?
Btw, are you using MongoDB or CosmosDB? If latter, please try this version of TxMongo: https://github.com/twisted/txmongo/issues/261 . CosmosDB doesn't support legacy version of Mongo's wire protocol. This PR adds support for newer protocol.
Thanks for fast reply! I'm using CosmosDB and will try your solution. Thanks!
Here is what I've also noticed (while using legacy wire protocol):
When the server fresh starts, few documents are persisted to db, but after some time (+- 5 minutes), it just blocks. Then it takes 25 minutes for twisted to understand that something bad happened - it notifies multiple: Failure: pymongo.errors.AutoReconnect: TxMongo lost connection to MongoDB. From this time point everything repeats. Few documents persisted, block, restoring connection after 25 min. Very strange.
Also, I'm using only insert_one and update_one so I'm wondering if your fix will change anything in my case :(
EDIT: it did not work.
After some investigations, I've noticed that tx-mongo receives pings uninterruptedly for the whole time, but somehow during that, deferred are not resolving themselves (no callback is present) for a very long time and eventually connectionLost is invoked.
Is it possible to reproduce the issue with some minimal script?
It would be great if you can give me access to a test instance of CosmosDB so I could try to reproduce and investigate the issue. Unfortunately, I cannot simply create CosmosDB instance by myself because Azure don't work with private persons from Russia, as far as I know.