mongo-hacker
mongo-hacker copied to clipboard
TylerBrock
Rework and fix index paranoia feature
Hi.
I like this tool a lot. So I invested quite some time in fixing and improving index paranoia feature. Using the magic of virtual machines (vagrant and VirtualBox), m to quickly install/switch between different MongoDB versions and some handwritten scripts and tests, I managed to get this feature working in (almost) every scenario possible.
Now this feature can correctly show index or indexes that was/were used in a query of almost any complexity.
It also shows if a query was covered by index.
And works with shards, showing which indexes were used on each shard.
Compatibility table:
2.4.0 - OK
2.4.14 - OK
2.5.0 - OK
2.5.5 - OK
2.6.0 - OK
2.6.12 - OK
2.7.0 - Bugged, doesn't show names of 2+ indexes used in query
2.7.8 - OK
3.0.0 - OK
3.0.15 - OK
3.1.0 - OK
3.1.9 - OK
3.2.0 - OK
3.2.17 - OK
3.3.0 - OK
3.3.15 - OK
3.4.0 - OK
3.4.9 - OK
3.5.1 - OK
3.5.13 - OK
3.6.0-rc0 - OK
This also closes #159.
Wow, thanks for all the hard work @angly-cat! Any idea if this works with the released version of 3.6.x?
It should, but to tell for sure some testing needs to be done.
Also, this version has a bug. I detect whether a query is covered by checking if there're FETCH stages.
But some time ago I stumbled upon a query, which accessed a collection, but there wasn't a single FETCH stage in execution stats of it. I don't remember the details at all, but I suspect it was related to sharding.
So, this implementation on rare occasions acts false-positively.
Thats most likely a covered query (one that can be serviced by just looking at index keys).
For example a query where you project name only and the index used for the match and/or sort is also on name doesn't need to fetch the actual documents to return results.
No, that query wasn't covered for sure, because it had non-zero docsExamined. But without FETCH stages.
@angly-cat sorry for the delay here, I'd love to get these changes in. Would you still want to work with me to get it over the finish line?
Would you still want to work with me to get it over the finish line?
Sure. What do you want me to do? I need to warn you, though, that my skills with MongoDB became a little rusty, because I haven't touched it for quite a while. :smile_cat:
@TylerBrock I'll test this out with modern versions of MongoDB (3.6+). This has been lingering for a while ;-)
