Lightning.NET icon indicating copy to clipboard operation
Lightning.NET copied to clipboard

Add documentation about common patterns and pitfalls

Open AlgorithmsAreCool opened this issue 4 years ago • 0 comments

Topics

LMDB Basics

  • Overview of concepts and terms
  • Usage scenarios
  • Common problems and limitations (find some by looking at github issues)
    • Max Key size (511 bytes by default)
  • How to use Async/Await safely (and when not to use it)
  • Picking good config defaults
    • MaxReaders
    • MapSize
    • MaxDatabases
  • DupSort and DupFixed
    • DupSort Values are limited to 511 bytes :/

Getting the best performance

  • Performance expectations and knowing what things cost
  • Picking good keys
    • Integer keys with IntegerKey set on database if possible
  • Understanding write amplification and overhead
    • Small records can use more space than expected
  • Hot to use strings as keys efficiently
  • Importance of ReadOnly vs Normal Transactions
  • Batching Writes
  • Compression?
  • The seductive danger of NO_SYNC
  • Use the cursor API for faster reads (0-copy,
  • Impact of lots of Databases
    • cache your db handles instead of reopening everytime
  • DB Health and maintenance
    • When should you consider reallocating the DB to defragment

Advanced nonsense (might never get around to writing these)

  • How to build your own indexes
  • How to grow/reallocate the DB as quickly as possible
  • Coordinating multiple lmdb environments
  • Sharding to allow incremental reallocations

AlgorithmsAreCool avatar Aug 21 '20 13:08 AlgorithmsAreCool