brpc icon indicating copy to clipboard operation
brpc copied to clipboard

client 使用 snappy 压缩时,server 能正常解压,但是解压出来的数据和原始数据不一致。

Open gulu-goolu opened this issue 5 months ago • 1 comments

Describe the bug client 侧为了节省带宽,使用了 snappy 压缩,但是测试下来,发现服务端用 snappy 解压的数据和 client 侧输入的数据不一致。

To Reproduce 把测试用的请求 dump 到磁盘,然后用 SampleIterator 遍历,并分别用 brpc 中的 snappy 和 github 上的 snappy 解压。其中 brpc 里面的 snappy 每次解压的结果都不一致,而 github 仓库中的 snappy 每次解压的结果都是一致的。

    brpc::SampleIterator iter(dir_);
    while (true) {
      std::unique_ptr<brpc::SampledRequest> sampled(iter.Next());
      if (sampled == nullptr) {
        break;
      }

      if (sampled->meta.compress_type() != brpc::COMPRESS_TYPE_SNAPPY) {
        continue;
      }

      std::cout << "----------------- " << sampled->request.size() << "--------------" << std::endl;

      butil::IOBuf brpc_decompressed_v1;
      butil::IOBuf brpc_decompressed_v2;
      ABSL_CHECK(brpc::policy::SnappyDecompress(sampled->request, &brpc_decompressed_v1));
      ABSL_CHECK(brpc::policy::SnappyDecompress(sampled->request, &brpc_decompressed_v2));

      if (brpc_decompressed_v1.to_string() == brpc_decompressed_v2.to_string()) {
        std::cout << "brpc decompress, same" << std::endl;
      } else {
        std::cout << "brpc decompress, diff" << std::endl;
      }

      std::string snappy_uncompressed_v1;
      std::string snappy_uncompressed_v2;
      auto compressed = sampled->request.to_string();
      ABSL_CHECK(snappy::Uncompress(compressed.data(), compressed.size(), &snappy_uncompressed_v1));
      ABSL_CHECK(snappy::Uncompress(compressed.data(), compressed.size(), &snappy_uncompressed_v2));
      if (snappy_uncompressed_v1 == snappy_uncompressed_v2) {
        std::cout << "snappy decompress, same" << std::endl;
      } else {
        std::cout << "snappy decompress, diff" << std::endl;
      }
    }

Expected behavior

Versions OS: ubuntu 20.04 Compiler: clang-18 brpc: 1.13.0 protobuf: 29.4

Additional context/screenshots

测试程序输出的结果。可以看到,brpc 里面的 snappy decompress 的结果,会有不一致的情况。而 github 上的 snappy,每次解压的结果都是一致的。

----------------- 105486--------------
brpc decompress, diff
snappy decompress, same
----------------- 47149--------------
brpc decompress, same
snappy decompress, same
----------------- 79794--------------
brpc decompress, diff
snappy decompress, same
----------------- 74139--------------
brpc decompress, same
snappy decompress, same
----------------- 107280--------------
brpc decompress, diff
snappy decompress, same

gulu-goolu avatar Jul 03 '25 13:07 gulu-goolu

试了下用 https://github.com/google/snappy 替换 butil/third_party/snappy 里面的 snappy,发现能修复这个问题。怀疑可能是因为编译器的版本变化导致了 butil 里面的 snappy 无法正确地解压。

gulu-goolu avatar Jul 03 '25 14:07 gulu-goolu