LiteDB icon indicating copy to clipboard operation
LiteDB copied to clipboard

[BUG] Slow as hell with +2GB db size even on SSD

Open okarpov opened this issue 4 years ago • 4 comments

Version 5.0.11

Describe the bug When file of the DB grows over +2GB then the Insert method becomes almost unresponsive

Code to Reproduce just add +10M entities and try to add more

Expected behavior not sure

Screenshots/Stacktrace NA

Additional context no Indexes were added so far

okarpov avatar Dec 26 '21 11:12 okarpov

It is better to insert BsonDocument entities (without mapping). In my case, 18m rows were inserted in 5 minutes, and actual DB file about 5GB.

alexSatov avatar Feb 09 '22 08:02 alexSatov

Hi @okarpov, can you describe how design your document? How many fields/arrays/subdocuments contains? How many indexes you have in this collection?

mbdavid avatar Feb 09 '22 17:02 mbdavid

I used my class for storing bitcoin keys it had >10M rows/items

`[Table("Search")] public partial class Search { public long ID { get; set; }

    [StringLength(128)]
    public string Priv { get; set; }

    [Required]
    [StringLength(128)]
    public string PubWif { get; set; }

    [Required]
    [StringLength(128)]
    public string PubAddr { get; set; }

    public decimal amt { get; set; }
}`

okarpov avatar Feb 09 '22 18:02 okarpov

Did you have any adicional index? Can you send some data-example of "Priv", "PubWif" e "PubAddr" to get some size-idea per iitem?

mbdavid avatar Feb 09 '22 19:02 mbdavid