reth icon indicating copy to clipboard operation
reth copied to clipboard

Tracking: Database abstraction cleanup

Open onbjerg opened this issue 2 years ago • 4 comments

Describe the feature

This is a tracking issue for the database abstraction clean up.

Current abstraction

Currently we have:

  • ShareableDatabase which is essentially just a crude wrapper around Arc<DB> with some helpers on it
  • Transaction (in reth-providers) which is a wrapper around a DbTx or DbTxMut, allowing the transaction to be opened or closed. It sort of works like ShareableDatabase. Also has a set of helpers
  • DbTx/DbTxMut which are the raw transactions from the database abstraction
  • Provider traits for reading different things (AccountProvider etc.)
## Tasks

- [ ] Create "write providers", i.e. a set of provider traits that write data instead of reading it
- [ ] Port providers to be implemented for all `DbTx`
- [ ] Move helpers from `ShareableDatabase` to the provider traits
- [ ] Move helpers from `Transaction` (in `reth-providers`) to the provider traits
- [ ] Replace `ShareableDatabase` with `Arc<DB>` instead
- [ ] Replace `Transaction` (in `reth-providers`) with `DbTx` / `DbTxMut` instead

Additional context

The motivations are:

  • Clearer abstractions
  • Reducing the number of helpers
  • Reducing code duplication for data writing

onbjerg avatar May 29 '23 14:05 onbjerg

@joshieDo Feel free to add more task lists with the different parts you intend to touch if you want to, e.g. a task list for all providers or something

onbjerg avatar May 29 '23 14:05 onbjerg

Looks good to me. Would add a stretch goal for figuring out how to pool transaction connections so we don't need to always open a new one, but we can do separately.

gakonst avatar May 29 '23 18:05 gakonst

  • Transaction which is a wrapper around a DbTx or DbTxMut, allowing the transaction to be opened or closed. It sort of works like ShareableDatabase. Also has a set of helpers
  • DbTx/DbTxMut which are the raw transactions from the database abstraction

This confuses me a lot because I think it's the other way around:

https://github.com/paradigmxyz/reth/blob/3347da091fa6c8e5f14a34cdc71dda09b721a445/crates/storage/db/src/implementation/mdbx/tx.rs#L15-L22

This is the wrapper that implements DbTx which is a trait.

mattsse avatar May 30 '23 15:05 mattsse

@mattsse Sorry, I am referring to Transaction in reth-providers

https://github.com/paradigmxyz/reth/blob/aea35263dad8f8e1e04c33804fa939278c9c362c/crates/storage/provider/src/transaction.rs#L44

Updated the issue to reflect it

onbjerg avatar May 31 '23 22:05 onbjerg

What's the status on this? Some cleanups were done in https://github.com/paradigmxyz/reth/pull/5476, cc @mattsse

DaniPopes avatar Dec 28 '23 14:12 DaniPopes

this is so outdated that I'd rather close this

mattsse avatar Dec 28 '23 14:12 mattsse