Is there any way to improve performance

[ranqiangjun]

I adjusted the script a little bit on my fork. Following is the pseudocode:

Before:

for item in items:
    dump_file()
for item in items:
   update_db()
for item in items:
    remove_blobs()

After:

for item in items:
    dump_file()
    update_db()
    remove_blobs()

the scanned rows would become less and less, I guess it will improve the performance a little

For MongoDB, I am a newbie. As you can see from the screenshot below, about 20s to dump the file, and 20 more seconds to remove blobs. The migration process is pretty slow. So I am wondering if there is a way to improve it, thanks in advance, and thanks for your script!

[ranqiangjun]

btw, creating index saved my day db.rocketchat_uploads.chunks.createIndex( { files_id: 1, n: 1 }, { unique: true } )

[arminfelder]

@jungleran , I am quite supprised that rocketchat does not set in an index files_id, I guess should be fixed in RC as well