Count Query construction time into QPS

[zhaih]

Right now we included the parsed query as part of the task and start time counting after those tasks are built, then we're not able to measure the performance of constructing those queries. In most of the services that are using Lucene, I believe the query construction process is a part of the overall performance, particularly for queries like RegexQuery, only constructing it could even take seconds in some extreme cases.

I believe the fix should be simply carrying the raw task text along with the task and parse it only after the benchmark is started?

[mikemccand]

+1. The benchmark is really cheating today by init'ing all the queries up front, and then only measuring how long those take to execute against the index.

Changing it to re-parse the Query and then execute sounds right?

[shubhamvishu]

Right now we included the parsed query as part of the task and start time counting after those tasks are built, then we're not able to measure the performance of constructing those queries. In most of the services that are using Lucene, I believe the query construction process is a part of the overall performance, particularly for queries like RegexQuery, only constructing it could even take seconds in some extreme cases.

I believe the fix should be simply carrying the raw task text along with the task and parse it only after the benchmark is started?

I'll take this up and work on this fix

[mikemccand]

Awesome, thanks @zhaih!