ord icon indicating copy to clipboard operation
ord copied to clipboard

Upgrade redb to 0.12.0

Open casey opened this issue 2 years ago • 4 comments

@cberner, just took a stab at upgrading to redb 0.12.0:

  • Some additional type hints were needed. I don't fully understand the tradeoffs here, and we might have discussed this before, but could get/insert/etc be changed to take the the final borrowed form, e.g. &[u8] instead of impl Borrow<[u8]>, in which case you could pass fewer things in, but the typing would be less confusing?

  • I couldn't get inscriptions_on_output working, which previously returned a borrowed iterator over a table, but now since the range start and end are references, it has to return a reference. I don't think this is critical for ord in this particular instance, but could be a problem in other cases where it isn't practical to collect into a vec.

  • I tried to switch to the non-mmap backend (btw did you ever wind up benchmarking it?), but after switching, the new/old_schema_gives_correct_error tests fail with a panic while panicking error, which abort the thread, and sadly don't give a good backtrace. These tests might be the only ones where we create a database, write to it, close it, and then open it again. Strangely though they only fail locally, so it could be an M1 mac/16kb page thing.

casey avatar Jan 22 '23 06:01 casey

  1. it could, although you were the one that asked for passing both refs and owned values ;) https://github.com/cberner/redb/issues/448

  2. hmm, I'll take a look. I thought I had worked out the lifetimes such that the range bounds' lifetime didn't matter

  3. uh oh. That's quite odd because 4k vs 16k page size should no longer be a thing. Maybe I have a bug where it can still default to the native page size though:

cberner avatar Jan 22 '23 15:01 cberner

Ah yes, (3) is this bug: https://github.com/cberner/redb/pull/501 it happens in those tests and also if you reopen a database on MacOS with open() (doesn't happen with create())

cberner avatar Jan 22 '23 17:01 cberner

I think this will fix (2): https://github.com/cberner/redb/pull/502

cberner avatar Jan 22 '23 18:01 cberner

Oh yes, I ran benchmarks. mmap is something like 20-50% faster for writes and 1.2x to 3x faster for reads depending on the number of concurrent threads. The biggest benefit is on highly concurrent read workloads, where mmap avoid lock contention by instead relying on unsafe

cberner avatar Jan 22 '23 18:01 cberner

is there a specific way to upgrade the db or should we just index it again from block 1?

utxo-detective avatar Feb 05 '23 23:02 utxo-detective

is there a specific way to upgrade the db or should we just index it again from block 1?

Yup, just delete the db and re-run ord.

casey avatar Feb 06 '23 00:02 casey