foundationdb-haskell icon indicating copy to clipboard operation
foundationdb-haskell copied to clipboard

Add FoundationDB 7.1 Support

Open xnomagichash opened this issue 3 years ago • 8 comments
trafficstars

FoundationDB 7.1 was released in April. According to the release notes for 7.1 and 7.0:

  • https://apple.github.io/foundationdb/release-notes/release-notes-710.html
  • https://apple.github.io/foundationdb/release-notes/release-notes-700.html

There are several new features, but probably no breaking changes. The new features are:

Get range and flat map

C: Added fdb_transaction_get_range_and_flat_map function to support running queries based on another query in one request. (PR #5609)

Reboot worker

C: Added a function, fdb_database_reboot_worker, to reboot or suspend the specified process. (PR #4094)

Force recovery

C: Added a function, fdb_database_force_recovery_with_data_loss, to force the database to recover into the given datacenter. (PR #4220)

Create snapshot

C: Added a function, fdb_database_create_snapshot, to create a snapshot of the database. (PR #4241)

Get main thread busyness

C: Added fdb_database_get_main_thread_busyness function to report how busy a client’s main thread is. (PR #4504)

I'm not sure how many of these actually are even worth exposing in the high level interface. Maybe only fdb_transaction_get_range_and_flat_map.

xnomagichash avatar Jul 21 '22 12:07 xnomagichash

Wow, I didn't see a 7.1 release announcement on the blog. Thanks for the heads-up!

I will start working on support over the next few weekends.

Since this library doesn't currently expose administrative functions, I will focus on fdb_transaction_get_range_and_flat_map.

crclark avatar Jul 23 '22 16:07 crclark

It looks like there is also new functionality for multi-tenancy. I'll look into adding support for that.

crclark avatar Jul 25 '22 00:07 crclark

fdb_database_get_main_thread_busyness also looks handy.

crclark avatar Jul 25 '22 00:07 crclark

Some notes to myself as I start implementing the secondary index helper...

Can't find any docs for fdb_transaction_get_mapped_range and fdb_future_get_mappedkeyvalue_array. The returned struct is a bit complex.

This PR introduced the current API. Here is a unit test in the PR.

This older PR introduced a first version of this API and has some other details. Here is a unit test in that PR.

crclark avatar Jul 31 '22 21:07 crclark

https://forums.foundationdb.org/t/getmappedrange-and-recordlayer/3263

https://forums.foundationdb.org/t/everything-about-getmappedrange/3280

https://github.com/apple/foundationdb/wiki/Everything-about-GetMappedRange

Rust library's docs for this function, can probably learn more by reading the code. https://docs.rs/foundationdb/latest/foundationdb/struct.Transaction.html#method.get_mapped_range

crclark avatar Jul 31 '22 22:07 crclark

Test from the Rust bindings gives a clear usage example: https://github.com/foundationdb-rs/foundationdb-rs/pull/61/files#diff-973f2b62dd7045ff9b764489e9e32ac3a0a3544a58ff96baeec4ce04896b3979

crclark avatar Jul 31 '22 22:07 crclark

@xnomagichash I just merged the most interesting new feature, mapped range support.

crclark avatar Aug 20 '22 01:08 crclark

Thanks! I can confirm it works and after some (a lot) trial and error was also able to get it working on an M1 macbook with foundationdb 7.1.18.

xnomagichash avatar Aug 25 '22 21:08 xnomagichash