Dan Luu

Consider adding higher rank documentActive row

2

It seems like this should be a "large" speedup.

(I think) our test time is dominated by allocator time because we have the same fixed size and number of blocks for tests as for our wikipedia ingestion build.

Non-deterministic row counts after rebuilding TermTable

2

This is the half of #275 that wasn't just a display issue in `TermTableBuilder`. If we re-run the `bitfunnel termtable` on `chunked1` from http://bitfunnel.org/wikipedia-as-test-corpus-for-bitfunnel/, we get varying results from `bitfunnel...

See #314 for description and discussion, including why this bug was introduced.

What has to change to support higher (e.g., rank 13) rows?

1

1. Dedup buffer has to get wider. For example, it would have to go from `2**6` to `2**13`. This seems ok because we only clear the entirety when we start...

Comma-seperated number terms (e.g., "0,1,2,3,4,5,6,7,8,9")

1

This is likely a `Workbench` issue, but until that's confirmed I'm going to leave this here. We have terms like `0,1,2,3,4,5,6,7,8,9`, `1,2,4,8,16`, `1,2,3,4,5`, etc. It's possible that this is "correct"...

valgrind drd errors: "not a condition variable" and "Probably a race condition"

3

Base valgrind fails with memory errors in `BlockingQueue.ShortRandom`. ~~~ ==30829== Thread 2: ==30829== Invalid read of size 8 ==30829== at 0x463138: BitFunnel::ThreadManager::ThreadEntryPoint(void*) (in /home/leah/dev/BitFunnel/build-ninja/src/Common/Utilities/test/UtilitiesTe st) ==30829== by 0x511254E: ??? (in...

Increase in false positives when substituting rank0 rows for rank3 rows

3

This is from 99f99973e385d8f74a9ee26da1c7709a53751ec3. We've talked about this but don't have an issue on this exact thing. With the current `TreatmentPrivateSharedRank0And3`, get the following number of false positives:  As...

 I have some preliminary code I'm using to calculate correlations which I expect to be slow because I'm using some slow `std` data structures. But in addition to the...

Large amount of time spent requesting and destructing tokens

2

This is running 16 threads on a quad-core 8 HT machine. Looks like roughly 30% of query time is token-related here. I'm not familiar with the Instruments profiler so I...