OpenUserJS.org Migrate from MongoDB 4.x to @next

[ ] Check with backend to make sure VPS is supported (lsb_release -a with https://docs.mongodb.com/manual/tutorial/install-mongodb-on-debian/#platform-support)
[ ] Works with buster and focal (however not working with bullseye ¹ and jammy ² yet)
[ ] Schedule notice on OUJS
[ ] Take OUJS offline
[ ] Compact DB
[ ] Backup existing pro DB (ex. mongodump --gzip --db openuserjs_devel --archive=./dev/devDBdirty.gz but productions)
[ ] Backup VPS (will require additional time)
[ ] Purge 4.x (remove old signing key for 4.x)
[ ] Target new sources
[ ] Install 5.x (may need db tools from .deb package... local installation was a bit weird with apt)
[ ] Import pro DB (ex. mongorestore --gzip --db openuserjs_devel --archive=./dev/devDBdirty.gz but productions)
[ ] Backup VPS (will require additional time)
[ ] Restore VPS backup (storage opt)
[ ] Put OUJS online (possibly in read only mode just for initial checking)
[ ] Purge old VPS running backup when all is "well"

NOTE(S)

mongo command is deprecated for mongosh (No longer shows openssl version btw in new CLI with --version :-1: )
Time it took locally with Kubuntu LTS was about 1 hour ... so approximately 1'ish hour downtime.
[email protected] appears to work in dev with MongoDB 5.x but eventually will migrate to >= [email protected] after rechecking sub-dependencies connect-mongo, express-rate-limit and rate-limit-mongo functionality.
https://docs.mongodb.com/manual/tutorial/install-mongodb-on-debian/

Possible dev installation when available for 6.x

KEY=https://www.mongodb.org/static/pgp/server-6.0.asc
KEYRING=/usr/share/keyrings/mongodb-org-6.0-keyring.gpg
REPO=https://repo.mongodb.org/apt/ubuntu
LIST=/etc/apt/sources.list.d/mongodb-org-6.0.list 

curl -fsSL $KEY | sudo gpg --batch --yes --dearmor -o $KEYRING
echo "deb [signed-by=$KEYRING] $REPO $(lsb_release -cs)/mongodb-org/6.0 multiverse" | sudo tee $LIST > /dev/null

apt-cache policy mongodb-org
N: Unable to locate package mongodb-org

Possible pro installation when available for 6.x (2022 09 01 ... missing in buster, present in bullseye)

KEY=https://www.mongodb.org/static/pgp/server-6.0.asc
KEYRING=/usr/share/keyrings/mongodb-org-6.0-keyring.gpg
REPO=https://repo.mongodb.org/apt/debian
LIST=/etc/apt/sources.list.d/mongodb-org-6.0.list 

curl -fsSL $KEY | sudo gpg --batch --yes --dearmor -o $KEYRING
echo "deb [signed-by=$KEYRING] $REPO $(lsb_release -cs)/mongodb-org/6.0 main" | sudo tee $LIST > /dev/null

Jul 18 '21 21:07 Martii

This is starting to get important. mongo is starting to show bitrot as an updated element didn't update but it said it did. Need mongosh soon as it is more actively developed (same with mongoose and mongodb deps). Blehh.... more work for the coming new year as usual.

See also:

https://github.com/OpenUserJS/OpenUserJS.org/pull/1899#issuecomment-1000253046

Dec 23 '21 22:12 Martii

Interesting:

https://docs.mongodb.com/manual/core/index-multikey/

MongoDB 4.x seems to allow one only on production however local dev here (MongoDB 5.x) says the index was created when I did 2. Since pro is a slightly different platform may not work with 2... but now there is an option. Been waiting for this feature for several years now. We can query multiple "parallel arrays" for years however we can't index them is what they are saying in the docs... paraphrased of course.

Jan 07 '22 07:01 Martii

Misc critical error encountered:

MongoDB Script save critical error
iFelix18/Monkey_Utils.js
{
 "version": 0,
 "modifiedPaths": [
  "meta",
  "uses",
  "hash",
  "updated"
 ]
}
0bc2e3b4-803a-11ec-868a-26da9250a6e2 statusError: {
 "message": "Database write error",
 "code": 502
}
    at ~/OpenUserJS.org1/controllers/scriptStorage.js:2178:27
    at ~/OpenUserJS.org1/node_modules/mongoose/lib/model.js:5074:18
    at processTicksAndRejections (node:internal/process/task_queues:78:11) {
  status: { message: 'Database write error', code: 502 },
  code: undefined
}

No further updates to mongoose@5.x and mongodb@3.x at this time.

Could be related to https://stackoverflow.com/questions/70536836/unreachable-mongodb-once-memory-is-being-allocated-on-the-swap since that's what came up on a log investigation.

Author is sending both normal and minified scripts from GH to OUJS on sync.

Jan 28 '22 18:01 Martii

Hoping to get this started next week after the holiday... still have https://repo.mongodb.org/apt/ubuntu/dists/jammy/mongodb-org/5.0/multiverse/binary-amd64/ missing for dev station but 6.0 does exist at this time.

Been a seriously long wait for them to get the completed new versions out. Last weeks check yielded the 6.0 for pro and dev available... currently testing on dev:

$ apt-cache policy mongodb-org
mongodb-org:
  Installed: 6.0.3
  Candidate: 6.0.3
  Version table:
 *** 6.0.3 500
        500 https://repo.mongodb.org/apt/ubuntu jammy/mongodb-org/6.0/multiverse amd64 Packages
        100 /var/lib/dpkg/status

NOTE: At this time neither doc pages show jammy or bullseye support yet the repo has support... wondering if they are expecting real world testing instead of internal.

Nov 22 '22 16:11 Martii

This is now scheduled to happen with the next 96 hours. I'm quite busy off and on but will find some time for this. Will update the topic label on main page when it's "Offline".

Dec 01 '22 06:12 Martii

Thrills: Cc: @sizzlemctwizzle

Callbacks are deprecated in the Node.js driver version 4.10 and will be removed in a future release. We recommend that you use Promises or Async/Await instead.

If your application requires the use of callbacks, you can use the MongoDB Node.js driver with optional callback support , which wraps the Node.js driver and provides optional callback support.

Ref: https://www.mongodb.com/docs/drivers/node/current/fundamentals/promises/#callbacks--deprecated-

Pretty sure they are already eol'd as I'm getting MongoInvalidArgumentError: Argument "options" must not be function when used with sessions in the mongodb@4.x driver. This also leads heavily into #1556

Anyhow... in-place migration from MongoDB 4.x to MongoDB 6.x failed... but luckily purged it as I described above and restored. Seems to be working just like dev station.

Closing this migration issue.

Dec 01 '22 10:12 Martii