RocksDict icon indicating copy to clipboard operation
RocksDict copied to clipboard

Support for large data storage, and transformation

Open umanniyaz opened this issue 9 months ago • 7 comments

Can it support storage for large data as a database.How transformations and CRUD operations impact its latency factor.

umanniyaz avatar May 05 '24 21:05 umanniyaz

I think it can support fairly large data size. It certainly is designed to store a large number of key-value pairs, especially when it cannot fit into the memory. For the specifics, refer to rocksdb, as this package is a thin wrapper around it.

Congyuwang avatar May 09 '24 11:05 Congyuwang

How to allocate maximum memory for storage of large key value pairs?

umanniyaz avatar May 09 '24 11:05 umanniyaz

What do you mean "allocate memory"?

Congyuwang avatar May 09 '24 12:05 Congyuwang

Did you meet some storage limitation or some other issue?

Congyuwang avatar May 09 '24 12:05 Congyuwang

When I am trying store huge data in key value pairs around 50 Million records my system crashes,is there a way to allocate some storage to Rdict to maximum limit

umanniyaz avatar May 09 '24 19:05 umanniyaz

What’s the error message?

Congyuwang avatar May 09 '24 22:05 Congyuwang

How many bytes are the keys and values on average, roughly? And are you sure that you have enough free disk space too?

Congyuwang avatar May 10 '24 00:05 Congyuwang