odict icon indicating copy to clipboard operation
odict copied to clipboard

Feature Requests: A dictionary can store 1 billion of entries and under 100GB.

Open ghost opened this issue 1 year ago • 1 comments

  • A dictionary can store 1 billion of entries and under 100GB

Approximately 100 bytes per line of text, what is the total storage requirement for 1 billion lines of text?

$$10^9 \times 100 \approx 10^{11}$$ bytes

Converted to GB, it is approximately:

$$\frac{10^{11}}{1024 \times 1024 \times 1024} \approx 93.13$$ GB

ghost avatar Oct 12 '24 02:10 ghost

This hasn't officially been tested, but it could probably be very easily (take the size in bytes of a dictionary with 1 entry and multiply it by 1B), minus a few extra bytes for the file header information. ODict is a very efficient format, so it could most likely very easily store a dictionary of this size. However, I'm not sure how fast the lookup times will be (would need testing).

Nickersoft avatar Oct 13 '24 18:10 Nickersoft