mongolite
mongolite copied to clipboard
count() taking long time
I guess since the new count api was implemented, I'm having an issue where count() takes a long time to complete. My collection has 1739040 documents, of average size ~ 15kB, and runs in a docker setup. Using the mongo API-itself db.collection.count() returns right away, but when using the mongolite-interface it takes a few minutes. It looks like a full query of some kind is executed, instead of using stored metadata. I guess the new setup is more resistant to errors, but I'm often simply using count() as a quick check if I have a working connection to the right database, where I'd like a quick result. Is this a bug, or could we get back the old count?
Details of setup:
- R 3.5.1
- mongolite 2.0
- Mongo version 3.6.5 with Wired Tiger storage engine
- running under docker 18.06.1-ce-mac73 (26764)
- MacOS 10.13.6
- rJava 0.9-10 with jdk 10.0.2
- When the query is run, CPU usage is 100% for one core used by docker-host-process.
- Timing by R itself:
system.time(MonDB$count())
user system elapsed
0.135 0.118 246.766
- It looks like I have the latest commit:
> mongolite:::mongo_collection_count
function (col, query = "{}")
{
.Call(R_mongo_collection_count, col, bson_or_json(query))
}
<bytecode: 0x1081d7698>
<environment: namespace:mongolite>
When the process is busy, db.currentOp() gives the following output:
{
"inprog" : [
{
"host" : "2351dfabc6d4:27017",
"desc" : "conn7",
"connectionId" : 7,
"client" : "127.0.0.1:45744",
"appName" : "MongoDB Shell",
"clientMetadata" : {
"application" : {
"name" : "MongoDB Shell"
},
"driver" : {
"name" : "MongoDB Internal Client",
"version" : "3.6.5"
},
"os" : {
"type" : "Linux",
"name" : "PRETTY_NAME=\"Debian GNU/Linux 8 (jessie)\"",
"architecture" : "x86_64",
"version" : "Kernel 4.9.93-linuxkit-aufs"
}
},
"active" : true,
"currentOpTime" : "2018-09-10T09:03:20.710+0000",
"opid" : 1596,
"secs_running" : NumberLong(0),
"microsecs_running" : NumberLong(84),
"op" : "command",
"ns" : "admin.$cmd.aggregate",
"command" : {
"currentOp" : 1,
"$db" : "admin"
},
"numYields" : 0,
"locks" : {
},
"waitingForLock" : false,
"lockStats" : {
}
},
{
"host" : "2351dfabc6d4:27017",
"desc" : "conn5",
"connectionId" : 5,
"client" : "172.17.0.1:37970",
"appName" : "r/mongolite",
"clientMetadata" : {
"application" : {
"name" : "r/mongolite"
},
"driver" : {
"name" : "mongoc",
"version" : "1.12.0"
},
"os" : {
"type" : "Darwin",
"name" : "macOS",
"version" : "17.7.0",
"architecture" : "x86_64"
},
"platform" : "cfg=0x0216a8e9 posix=200112 stdc=201112 CC=clang 6.0.0 (tags/RELEASE_600/final) CFLAGS=\"\" LDFLAGS=\"\""
},
"active" : true,
"currentOpTime" : "2018-09-10T09:03:20.710+0000",
"opid" : 1509,
"lsid" : {
"id" : UUID("225e3d99-4ac5-4a1d-9488-9d8d02d5c820"),
"uid" : BinData(0,"47DEQpj8HBSa+/TImW+5JCeuQeRkm5NMpJWZG3hSuFU=")
},
"secs_running" : NumberLong(59),
"microsecs_running" : NumberLong(59064213),
"op" : "command",
"ns" : "MyDB.MyCol",
"command" : {
"aggregate" : "MyCol",
"cursor" : {
},
"pipeline" : [
{
"$group" : {
"_id" : null,
"n" : {
"$sum" : 1
}
}
}
],
"$db" : "MyDB",
"$readPreference" : {
"mode" : "primaryPreferred"
},
"lsid" : {
"id" : UUID("225e3d99-4ac5-4a1d-9488-9d8d02d5c820")
}
},
"planSummary" : "COLLSCAN",
"numYields" : 5077,
"locks" : {
"Global" : "r",
"Database" : "r",
"Collection" : "r"
},
"waitingForLock" : false,
"lockStats" : {
"Global" : {
"acquireCount" : {
"r" : NumberLong(10158)
}
},
"Database" : {
"acquireCount" : {
"r" : NumberLong(5079)
}
},
"Collection" : {
"acquireCount" : {
"r" : NumberLong(5079)
}
}
}
}
],
"ok" : 1
}