Hash-based indexes

[saosebastiao]

I'm not familiar enough with FDWs to know for sure if this could work with the query planner, but I think it would be beneficial to use hash based indexes. In this sense, you could get away with not storing the indexed columns as columns, but store the hash in the table data similar to how you store skiplists. I think this would be extremely beneficial for data size and its implicit performance. I believe this is the strategy that kdb+ uses for its database (which is pretty ubiquitous in HFT).

[saosebastiao]

For reference, here is an article about a hack using a similar idea to get similar storage efficiency and performance using MySQL tables: https://engineering.eventbrite.com/optimizing-a-table-with-composite-primary-keys/

[pykello]

Thanks @saosebastiao. I agree that this can be beneficial for data size. Original ORC format uses something similar. We will evaluate this in our next releases.