mongolite icon indicating copy to clipboard operation
mongolite copied to clipboard

count() taking long time

Open EmilBode opened this issue 6 years ago • 8 comments

I guess since the new count api was implemented, I'm having an issue where count() takes a long time to complete. My collection has 1739040 documents, of average size ~ 15kB, and runs in a docker setup. Using the mongo API-itself db.collection.count() returns right away, but when using the mongolite-interface it takes a few minutes. It looks like a full query of some kind is executed, instead of using stored metadata. I guess the new setup is more resistant to errors, but I'm often simply using count() as a quick check if I have a working connection to the right database, where I'd like a quick result. Is this a bug, or could we get back the old count?

Details of setup:

  • R 3.5.1
  • mongolite 2.0
  • Mongo version 3.6.5 with Wired Tiger storage engine
  • running under docker 18.06.1-ce-mac73 (26764)
  • MacOS 10.13.6
  • rJava 0.9-10 with jdk 10.0.2
  • When the query is run, CPU usage is 100% for one core used by docker-host-process.
  • Timing by R itself:
system.time(MonDB$count())
   user  system elapsed 
  0.135   0.118 246.766
  • It looks like I have the latest commit:
> mongolite:::mongo_collection_count
function (col, query = "{}") 
{
    .Call(R_mongo_collection_count, col, bson_or_json(query))
}
<bytecode: 0x1081d7698>
<environment: namespace:mongolite>

When the process is busy, db.currentOp() gives the following output:

{
	"inprog" : [
		{
			"host" : "2351dfabc6d4:27017",
			"desc" : "conn7",
			"connectionId" : 7,
			"client" : "127.0.0.1:45744",
			"appName" : "MongoDB Shell",
			"clientMetadata" : {
				"application" : {
					"name" : "MongoDB Shell"
				},
				"driver" : {
					"name" : "MongoDB Internal Client",
					"version" : "3.6.5"
				},
				"os" : {
					"type" : "Linux",
					"name" : "PRETTY_NAME=\"Debian GNU/Linux 8 (jessie)\"",
					"architecture" : "x86_64",
					"version" : "Kernel 4.9.93-linuxkit-aufs"
				}
			},
			"active" : true,
			"currentOpTime" : "2018-09-10T09:03:20.710+0000",
			"opid" : 1596,
			"secs_running" : NumberLong(0),
			"microsecs_running" : NumberLong(84),
			"op" : "command",
			"ns" : "admin.$cmd.aggregate",
			"command" : {
				"currentOp" : 1,
				"$db" : "admin"
			},
			"numYields" : 0,
			"locks" : {
				
			},
			"waitingForLock" : false,
			"lockStats" : {
				
			}
		},
		{
			"host" : "2351dfabc6d4:27017",
			"desc" : "conn5",
			"connectionId" : 5,
			"client" : "172.17.0.1:37970",
			"appName" : "r/mongolite",
			"clientMetadata" : {
				"application" : {
					"name" : "r/mongolite"
				},
				"driver" : {
					"name" : "mongoc",
					"version" : "1.12.0"
				},
				"os" : {
					"type" : "Darwin",
					"name" : "macOS",
					"version" : "17.7.0",
					"architecture" : "x86_64"
				},
				"platform" : "cfg=0x0216a8e9 posix=200112 stdc=201112 CC=clang 6.0.0 (tags/RELEASE_600/final) CFLAGS=\"\" LDFLAGS=\"\""
			},
			"active" : true,
			"currentOpTime" : "2018-09-10T09:03:20.710+0000",
			"opid" : 1509,
			"lsid" : {
				"id" : UUID("225e3d99-4ac5-4a1d-9488-9d8d02d5c820"),
				"uid" : BinData(0,"47DEQpj8HBSa+/TImW+5JCeuQeRkm5NMpJWZG3hSuFU=")
			},
			"secs_running" : NumberLong(59),
			"microsecs_running" : NumberLong(59064213),
			"op" : "command",
			"ns" : "MyDB.MyCol",
			"command" : {
				"aggregate" : "MyCol",
				"cursor" : {
					
				},
				"pipeline" : [
					{
						"$group" : {
							"_id" : null,
							"n" : {
								"$sum" : 1
							}
						}
					}
				],
				"$db" : "MyDB",
				"$readPreference" : {
					"mode" : "primaryPreferred"
				},
				"lsid" : {
					"id" : UUID("225e3d99-4ac5-4a1d-9488-9d8d02d5c820")
				}
			},
			"planSummary" : "COLLSCAN",
			"numYields" : 5077,
			"locks" : {
				"Global" : "r",
				"Database" : "r",
				"Collection" : "r"
			},
			"waitingForLock" : false,
			"lockStats" : {
				"Global" : {
					"acquireCount" : {
						"r" : NumberLong(10158)
					}
				},
				"Database" : {
					"acquireCount" : {
						"r" : NumberLong(5079)
					}
				},
				"Collection" : {
					"acquireCount" : {
						"r" : NumberLong(5079)
					}
				}
			}
		}
	],
	"ok" : 1
}

EmilBode avatar Sep 10 '18 09:09 EmilBode