Persisting of secondary indexes to disk?

[wgerven]

Hi, very nice database, thank you for your work!

I am currently reviewing implementing your database in an electron project of me. I need secondary indexes, but dataset can be rather large ~100k documents, 100Mb < size < 1Gb.

To rebuild databases every time at app launch is a big performance hit. Is there a way to persist secondary indexes to disk; to prevent building them from scratch each time the app is opened?

Thank you again!

[sergeyksv]

Index persistence is not something impossible but we didn't have that. We target more servers (tiny) and doing full scan on server scan is not a big deal. This is for sure not the same for desktop app. In the same time modern laptops are really powerful and scanning of 1GB doesn't looks a huge task. Keep in mind that scan is delayed until first access to collection.

We actually considered intermediate options, not persist indexes itself but persist indexed values separately fro data. Building index is not a big deal, but full data scan is.

Do you have any expectations how much it will take to load your dataset?

[wgerven]

Dear Sergey, thank you for your quick reply, I appreciate it. I am myself on rather old hard ware, so I do not consider my laptop "state of the art". :) but actually that is good for benchmarking, as I should assume average hardware for my end user.

I wanted to actually perform some simple benchmarks today or tomorrow, using typical data in use by my app. I will report back to you. Perhaps you're indeed right that building the index at app launch is not that big a deal. I'm curious!