Upgrade from LL v2.6.2 to v5.4.0 loses control over preexisting Persona data

[ghost]

What version were you using?

LL: v5.4.0 (tagged commit) MongoDB: v4.0.16 NodeJS: Latest 8:buster Dockerhub image

What steps can we follow to reproduce the behaviour?

• Have a database on MongoDB v3.4 (EOL) and a LL on version 2.6.2 you want to migrate to v5.4.0 and a newer MongoDB. Because the official Deployment-Script uses MongoDB v4, we do as well.

• Create a clean deployment of LL 5.4.0 on MongoDB 4.0 (we're using a custom Docker image but is based around the information in the documentation and the learninglocker/deploy repo).

• Dump the v2.6.2-DB using mongodump (Version from MongoDB 4.x!)

• Reimport the DB on the new MongoDB 4.0 cluster using mongorestore. All collections and documents are being restored 100% into the new MongoDB 4.0 DB.

• Apply migrations on LL using yarn migrate

• The result is fully functional LL instance where statements, orgs, clients, users, and all other content is preserved. However, no Personas are shown in the web interface even though the database still contains the raw data.

• It is unclear why the data is not being retrieved by the service. There are no errors in the logs, at all (loglevel is DEBUG).

What is the actual behaviour?

The LL UI fails to show all existing personas which still are in the personas collection.

What is the expected behavior?

LL should show all preexisting personas without problems.

Is there any additional information that will help us replicate/understand the problem?

Is this an issue with the MongoDB version and/or connstring and/or MongoDB driver? I have strong suspicions that it is indeed the case. See e.g. https://github.com/LearningLocker/learninglocker/issues/1411.

Could you please instruct:

• Which is the exact supported MongoDB version range?
• Do mongo+srv:// connstrings work in general?
• Did I miss something on top of yarn migrate that could make the models/services work correctly? Any pointers?
• I am about to start bisecting builds down to the oldest LL version that is compatible with MongoDB >=v3.6. If that doesn't make sense (persona-service didn't change for a long time) please yell stop :)

[ghost]

I've tested the deployment against various MongoDB-Versions and Connstring combinations and it turns out that indeed, the standard mongodb:// connstring format must be used. mongodb+srv:// leads to spurious and hard-to-track-down bugs that occur without any meaningful error message both in the service and the database logs.

I suspect the root cause is indeed the old version of the MongoDB core driver that hides in your dependency tree.

I'm leaving this open; a MongoDB configuration hint should definitely be added to the documentation, either as a hint or as a warning.