FaxServer icon indicating copy to clipboard operation
FaxServer copied to clipboard

Performance issues with 3000+ files

Open ZetaTwo opened this issue 6 years ago • 1 comments

Faxes started to fail due to timeouts. I have about 3000 sent faxes at a total of ~700mb

> db.faxFile.stats()
{
	"ns" : "vapor.faxFile-backup1",
	"size" : 745257940,
	"count" : 3050,
	"avgObjSize" : 244346,
	"storageSize" : 728104960,

Removing all old files by running

> db.faxFile.renameCollection("faxFile_backup1")

Got rid of the issue. In my opinion, 3000 files totalling 700mb isn't that much data and it feels like there is some kind of inefficiency in the code. Unfortunately, I haven't had the time to look closer to try to see what the problem could be or how it could be improved but I post this here in the meanwhile as a start for further investigation.

ZetaTwo avatar Feb 03 '19 13:02 ZetaTwo

What's missing is an index on the faxFile collection, for example, in the Mongo console, running: db.faxFile.createIndex({"faxObjectId":1 }, {"unique": true}) completely solves the performance issue.

I don't know much about MongoKitten but reading their docs would suggest that the following code:

try FaxFile.collection.createIndex(named:"faxObjectId", withParameters: .unique)

somewhere would create the relevant index. I know too little about the codebase however to know where to put this.

ZetaTwo avatar Feb 20 '19 22:02 ZetaTwo