Resolves #1493: Consider replacing single record range deletes with multiple single key deletes

[alecgrieser]

This changes the single-record delete path so that in cases where we know which key(s) the record used (including taking split points into account), we issue a series of single-key deletes rather than one clear range over the whole record. As issue #1493 describes in more detail, this approach should be more efficient for the currently-in-development RocksDB storage engine (as well as any other LSM storage engine that might one day be used) while being more-or-less a wash for the other storage engines.

This resolves #1493.

[foundationdb-ci]

AWS CodeBuild CI Report for Linux CentOS 7

• CodeBuild project: fdb-record-layer-pr-proto3
• Commit ID: 250453135923e18035384c05d96baccad3649fd8
• Result: SUCCEEDED
• Error: N/A
• Build Logs (available for 30 days)

[foundationdb-ci]

AWS CodeBuild CI Report for Linux CentOS 7

• CodeBuild project: fdb-record-layer-pr-proto2
• Commit ID: 250453135923e18035384c05d96baccad3649fd8
• Result: SUCCEEDED
• Error: N/A
• Build Logs (available for 30 days)

[sonarqubecloud[bot]]

Kudos, SonarCloud Quality Gate passed!   

0 Bugs
0 Vulnerabilities
0 Security Hotspots
0 Code Smells

0.0% Coverage
0.0% Duplication

[foundationdb-ci]

AWS CodeBuild CI Report for Linux CentOS 7

• CodeBuild project: fdb-record-layer-pr-proto2
• Commit ID: 67e7a7f5d80b1ecacbbea7932b8f9f8e7fe760d2
• Result: SUCCEEDED
• Error: N/A
• Build Logs (available for 30 days)

[foundationdb-ci]

AWS CodeBuild CI Report for Linux CentOS 7

• CodeBuild project: fdb-record-layer-pr-proto3
• Commit ID: 67e7a7f5d80b1ecacbbea7932b8f9f8e7fe760d2
• Result: SUCCEEDED
• Error: N/A
• Build Logs (available for 30 days)

[alecgrieser]

Closing this for now. We may want to do something like this for the RocksDB storage engine, so we may need to re-open this at a later date.

[foundationdb-ci]

Result of fdb-record-layer-pr-proto2 on Linux CentOS 7

• Commit ID: 67e7a7f5d80b1ecacbbea7932b8f9f8e7fe760d2
• Duration 0:12:49
• Result: :white_check_mark: SUCCEEDED
• Error: N/A
• Build Logs (available for 30 days)
• Build Artifact (available for 30 days)

[foundationdb-ci]

Result of fdb-record-layer-pr-proto3 on Linux CentOS 7

• Commit ID: 67e7a7f5d80b1ecacbbea7932b8f9f8e7fe760d2
• Duration 0:13:20
• Result: :white_check_mark: SUCCEEDED
• Error: N/A
• Build Logs (available for 30 days)
• Build Artifact (available for 30 days)

[MMcM]

I am wondering whether we want a feature flag to allow for more controlled testing of this optimization. The necessary plumbing doesn't look straightforward, though. The best I can come up with is replacing clearBasedOnPreviousSizeInfo with an enum and getting that from the RecordSerializer, which is regularly extended.

[foundationdb-ci]

Result of fdb-record-layer-pr-proto2 on Linux CentOS 7

• Commit ID: 85963e02c10f076a9edfeeb832372bd3a6126fbc
• Duration 0:15:16
• Result: :white_check_mark: SUCCEEDED
• Error: N/A
• Build Logs (available for 30 days)
• Build Artifact (available for 30 days)

[foundationdb-ci]

Result of fdb-record-layer-pr-proto3 on Linux CentOS 7

• Commit ID: 85963e02c10f076a9edfeeb832372bd3a6126fbc
• Duration 0:16:32
• Result: :white_check_mark: SUCCEEDED
• Error: N/A
• Build Logs (available for 30 days)
• Build Artifact (available for 30 days)

[alecgrieser]

Rebase updates branch against latest main

[alecgrieser]

I am wondering whether we want a feature flag to allow for more controlled testing of this optimization

I think it would be straightforward enough to add a RecordLayerPropertyKey to control this if we wanted to control the testing/rollout of the optimization. It should be safe to do this, I think, given that the old way and the new way should be compatible in various mixed modes, and so there shouldn't be an issue if some versions have this, some instances don't, though I guess we could add some mixed mode testing here in the Record Layer. I'm not super in love with adding more properties here, but it shouldn't be that messy.

I'm also looking a bit closer now at clearBasedOnPreviousSizeInfo, and it feels like it should be possible to remove this as an independent parameter here, and it feels like it should be removed entirely. It's usages appear to set it to true except in FDBMetaDataStore, which is a bit of a special use case of SplitHelper anyway, and though I haven't convinced myself of it, seems like it should be able to use the previous split information as well. (But this is probably separate from this PR anyway.)

[alecgrieser]

Added a property in bf3e7b1 to allow this to be more selectively rolled out.

[foundationdb-ci]

Result of fdb-record-layer-pr-proto2 on Linux CentOS 7

• Commit ID: bf3e7b183a201ad1855e3cbad27f756d200bc0ff
• Duration 0:15:25
• Result: :white_check_mark: SUCCEEDED
• Error: N/A
• Build Logs (available for 30 days)
• Build Artifact (available for 30 days)

[foundationdb-ci]

Result of fdb-record-layer-pr-proto3 on Linux CentOS 7

• Commit ID: bf3e7b183a201ad1855e3cbad27f756d200bc0ff
• Duration 0:17:29
• Result: :white_check_mark: SUCCEEDED
• Error: N/A
• Build Logs (available for 30 days)
• Build Artifact (available for 30 days)

[foundationdb-ci]

Result of fdb-record-layer-pr-proto2 on Linux CentOS 7

• Commit ID: b4e435aa191dfdae9f86c4ff35edc49f51e592e9
• Duration 0:15:17
• Result: :white_check_mark: SUCCEEDED
• Error: N/A
• Build Logs (available for 30 days)
• Build Artifact (available for 30 days)

[sonarqubecloud[bot]]

SonarCloud Quality Gate failed.   

0 Bugs
0 Vulnerabilities
0 Security Hotspots
10 Code Smells

0.0% Coverage
0.0% Duplication

[foundationdb-ci]

Result of fdb-record-layer-pr-proto3 on Linux CentOS 7

• Commit ID: b4e435aa191dfdae9f86c4ff35edc49f51e592e9
• Duration 0:16:56
• Result: :white_check_mark: SUCCEEDED
• Error: N/A
• Build Logs (available for 30 days)
• Build Artifact (available for 30 days)