💎 Support digest_function in GrpcStore

[bclark8923]

When using an alternative hash function to the default (sha256), GRPC store doesn't receive which hash function to use.

[MarcusSorealheis]

If you are looking to solve this issue, look into resource_info.rs. It predates our support for Blake3.

[boldpulse]

I will work on this issue. cc: @MarcusSorealheis, @allada, @bclark8923.

[boldpulse]

Hi, @bclark8923.

I looked over details of this issue and I have some things unclear about this issue.

FYI, from this PR, we now detect digest function automatically from the request.

And there is code for GrpcStore receiving digest_function from ActiveOriginContext in grpc_store.rs.

https://github.com/TraceMachina/nativelink/blob/8d957a5d25a3f27051a270c4db24682e55213ee5/nativelink-store/src/grpc_store.rs#L438-L442

https://github.com/TraceMachina/nativelink/blob/8d957a5d25a3f27051a270c4db24682e55213ee5/nativelink-store/src/grpc_store.rs#L494-L498

https://github.com/TraceMachina/nativelink/blob/8d957a5d25a3f27051a270c4db24682e55213ee5/nativelink-store/src/grpc_store.rs#L541-L545

And just fyi, while I was trying to evaluate this issue, I found out that there were no test files for testing GrpcStore feature.

Did we omit this for specific purpose or just missing and need to be added later?

And I hope you to make it clear how you caught this issue and how you tested.

Thanks.

cc: @allada, @MarcusSorealheis

[MarcusSorealheis]

@aleksdmladenovic if there is no test we need to write one.

[MarcusSorealheis]

/bounty $1000

[MarcusSorealheis]

more guidance here: https://gist.github.com/MarcusSorealheis/17307f41b6c632cb751c397280c70ed4

[asr2003]

@MarcusSorealheis This is Sahithi from Algora expert community! I would like to connect and tackle this issue and I have gone through the program guidelines attached above. Well, I need to discuss my approach before I design document what's I am thinking of in Slack. Can we continue discussion in a draft design PR or in Slack. Happy to contribute to nativelink :)

Can I get this issue assigned?

/attempt #1325

Algora profile	Completed bounties	Tech	Active attempts	Options
@asr2003	9 bounties from 4 projects	Scala, Rust, Go & more		Cancel attempt

[MarcusSorealheis]

@asr2003 you need to start with. Design document per the details in the gist above.

[asr2003]

@MarcusSorealheis Here's my draft of design doc for this issue. https://docs.google.com/document/d/1VI3OwfBNgIeF_gJ38yILoH-WgJIrVwOz/

I am happy to incorporate any suggestions and feedback :)

[MarcusSorealheis]

Could you include it in the Slack as well?

Here is the Slack: https://join.slack.com/t/nativelink/shared_invite/zt-281qk1ho0-krT7HfTUIYfQMdwflRuq7A

On Thu, Nov 21, 2024 at 9:17 AM asr2003 @.***> wrote:

@MarcusSorealheis https://github.com/MarcusSorealheis Here's my draft of design doc for this issue. https://docs.google.com/document/d/1VI3OwfBNgIeF_gJ38yILoH-WgJIrVwOz/

I am happy to incorporate any suggestions and feedback :)

— Reply to this email directly, view it on GitHub https://github.com/TraceMachina/nativelink/issues/1325#issuecomment-2491829104, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAR6TSDVHNLP4W2GLRHMGTT2BYITPAVCNFSM6AAAAABNXQBKZWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDIOJRHAZDSMJQGQ . You are receiving this because you were mentioned.Message ID: @.***>

-- Marcus Eagan

[algora-pbc[bot]]

💡 @asr2003 submitted a pull request that claims the bounty. You can visit your bounty board to reward.

[SAIKIRANSURAPALLI]

/attempt #1325

Options

• Cancel my attempt

[germanop]

Hi @MarcusSorealheis, I'd like to take on this issue. Is there a place to discuss things other than github?

[asr2003]

@germanop Thanks for your interest, it's mostly done except we have discussion regarding tests of grpc store and test digest function. But reviewers are on other priority works for now

[Excellencedev]

/attempt #1325

Implementation Plan

I will fix the digest_function support in GrpcStore by:

1. Problem: GrpcStore methods don't extract digest_function from incoming GRPC requests, causing them to use default SHA256 instead of client's requested hash function (like Blake3).

2. Solution: Extract digest_function field from all relevant GRPC requests and set proper context using make_ctx_for_hash_func() before forwarding to upstream services.

3. Methods to fix:

   • find_missing_blobs
   • batch_update_blobs
   • batch_read_blobs
   • get_action_result
   • update_action_result

4. Implementation: Add digest_function extraction and .with_context() calls following the same pattern used in CasServer and AcServer.

5. Testing: Add comprehensive tests to verify digest function handling works correctly.

This will ensure alternative hash functions work properly through GrpcStore.