graph-node icon indicating copy to clipboard operation
graph-node copied to clipboard

Add extensible compression support for RPC requests

Open fubhy opened this issue 7 months ago • 3 comments

Summary

Implements extensible compression support for Graph Node's outgoing JSON-RPC requests to upstream providers, addressing issue #5671. The implementation uses an enum-based design that makes it easy to add new compression methods in the future without breaking existing configurations.

Key Changes

  • Enum-based compression configuration: Replaced boolean compression_enabled field with Compression enum (None, Gzip)
  • Per-provider configuration: Each RPC provider can have independent compression settings
  • Future-extensible design: Clear pattern for adding Brotli, Deflate, and other compression methods
  • Backward compatible: Defaults to no compression, existing configs continue to work
  • Type-safe: Prevents invalid compression method strings in configuration

Configuration Format

Enable gzip compression:

[chains.mainnet]
provider = [
  { 
    label = "mainnet-rpc", 
    details = { 
      type = "web3", 
      url = "http://rpc.example.com", 
      features = ["archive"],
      compression = "gzip"
    }
  }
]

Disable compression (default):

compression = "none"  # or omit field entirely

Implementation Details

  • Configuration: Compression enum in node/src/config.rs with serde support
  • Transport layer: Updated Transport::new_rpc() to handle compression enum with match statement
  • Future ready: Commented placeholders show how to add new compression methods
  • Testing: Unit tests for both gzip and none compression parsing
  • Dependencies: Added gzip feature to reqwest dependency

Future Extensions

Adding new compression methods requires only:

  1. New enum variant: #[serde(rename = "brotli")] Brotli
  2. Match arm in transport: Compression::Brotli => client_builder.brotli(true)
  3. Corresponding reqwest feature in Cargo.toml

Test plan

  • [x] Unit tests pass for compression configuration parsing
  • [x] Configuration examples validate correctly
  • [x] Transport layer handles enum variants properly
  • [ ] Manual testing with real RPC providers that support gzip
  • [ ] Performance testing to verify compression benefits
  • [ ] Integration tests with actual compressed RPC traffic

🤖 Generated with Claude Code

fubhy avatar Jul 22 '25 17:07 fubhy

@lutter This is also building now ...

fubhy avatar Aug 26 '25 11:08 fubhy

Rebased to latest master (and cleaned up a little)

lutter avatar Sep 18 '25 18:09 lutter

I just looked at this, and seeing how we do similar things for other providers, why do we use a separate compression = "gzip" here instead of following the pattern established for firehose where we use features = [ "compression" ]; to keep our options open for other compression methods, we could do something like features = [ "gzip" ] or features = ["compression/gzip"] for RPC providers.

lutter avatar Sep 18 '25 19:09 lutter

This pull request hasn't had any activity for the last 90 days. If there's no more activity over the course of the next 14 days, it will automatically be closed.

github-actions[bot] avatar Dec 20 '25 00:12 github-actions[bot]