tongrams

tongrams copied to clipboard

Implement building ngrams storage via python

4

Hi. I have written an abstraction layer around multiple libraries doing word splitting (`londonisacapitalofgreatbritain` must become `london is a capital of great britain`). All the libs rely on preprocessed ngrams...

KOLANICH

Move as much of things out of headers as possible and make tongrams a shared library

3

KOLANICH

Currently, `boost` is used: - for the preprocessor's `for_each`; - for memory mapped files; - for iterating through gzipped files.

jermp

Create one master tool `tongrams` with sub-tools, like `tongrams build` and `tongrams query` instead of many separate executables.

jermp

jermp