pisa

pisa copied to clipboard

Improve block codecs

3

Hi everyone! I bring up for discussion some possible improvements that could be introduced in the `block codecs`. ### Interpolative Starting here: https://github.com/pisa-engine/pisa/blob/35dd03b73506d2089686a4a211e38898671b425c/include/pisa/codec/block_codecs.hpp#L151 Why not just use `in` instead of...

gustingonzalez

Improving the understanding of what PISA is

13

Given recent feedback from HN, we should look at improving how we explain PISA, and offer some benchmarks to common systems like Lucene and Tantivy (perhaps). We also should document...

JMMackenzie

Slow parse_collection due to I/O synchronization

3

**Describe the bug** Ingesting plaintext records via stdin seems very slow (`12`MiB/sec) even though `60` worker threads are used. I suspect this has to do with https://en.cppreference.com/w/cpp/io/ios_base/sync_with_stdio Before I disabled...

mpetri

Redundant quantize flag?

2

Somehow, we ended up with two quantization options for `queries`. First, we have `--quantized`, and then we have `--scorer quantized`. I think the easier way would be to remove `--quantized`...

elshize

Right now, the scorer is hard-coded to `bm25`.

elshize

Speed up CI for PRs

3

I propose to limit Travis build for PRs to few essential checks. First, we can compile headers since it is quite fast so we make sure all the included are...

amallia

Always check if path exists

1

We have many places where we just assume that a file exists, say, when we create a `mio::mmap_source`, which then fails with simply `No such file exists` or something similar...

elshize

When PISA is used as a library, the exported path for includes is wrong; you need to include headers like so: ```cpp #include ``` #### Should be ```cpp #include ```

elshize

Get rid of `configuration.hpp`

amallia

Tokenization in parse_plaintext_content

2

**Describe the bug** Shouldn't we do proper tokenization in `parse_plaintext_content` too? https://github.com/pisa-engine/pisa/blob/4a739b2ec50d2faa1e3c57336337e4fe219e09ec/include/pisa/forward_index_builder.hpp#L60-L66

amallia