4x+ perf hit to search query on BSUP in sequential runtime

[philrz]

The merge of the changes in #5669 is correlated with the performance of the Search+ query from the super command docs becoming more than 4x worse.

Detail

Repro is with super commit f55d7bd. The run times shown below are on my Intel-based Macbook, but the perf hit was originally observed on an AWS m6idn.2xlarge EC2 instance.

Test data can be downloaded from s3://brim-sampledata/gha/gha.bsup. It was generated as shown in the super command doc by downloading the JSON-format GitHub Archive data and turning it into BSUP, i.e.,

$ super -f bsup -o gha.bsup 2023-02-08-*.json.gz

Baseline performance as of commit 0f8e7db that came right before the changes in #5669 merged:

$ super -version
Version: 0f8e7dbef

$ time super -c "
SELECT count() AS count
FROM 'gha.bsup'
WHERE grep('in case you have any feedback 😊');"

{count:3(uint64)}

real	0m13.063s
user	1m49.575s
sys	0m2.644s

Then at the next commit f55d7bd that's associated with the changes in #5669, the perf of the same query drops to:

$ super -version
Version: f55d7bd4b

$ time super -c "
SELECT count() AS count
FROM 'gha.bsup'
WHERE grep('in case you have any feedback 😊');"

{count:3(uint64)}

real	0m57.893s
user	1m16.188s
sys	0m3.234s

Most of this perf drop is still with us at tip of main, which is commit c8cc05e at the moment.

$ super -version
Version: c8cc05e61

$ time super -c "
SELECT count() AS count
FROM 'gha.bsup'
WHERE grep('in case you have any feedback 😊');"
{count:3(uint64)}

real	0m46.585s
user	1m18.878s
sys	0m2.070s

[philrz]

@nwt took a look at this and reported that the perf delta is due to some new "yields" that occur in the revised approach. This is something we could improve on with some changes to the optimizer, but since sequential runtime performance is not the current priority focus, we can keep this topic on the shelf for now.