Java agent invalid_argument unexpected EOF

[xiangyuwang-stripe]

Describe the bug

Hi, I'm sending profiles from a ARM64 based java service to a Pyroscope distributor.

My Java agent config

public class PyroscopeProfiler {
  protected final FlagsConfig flags;
  protected final String serviceName;
  protected final String PYROSCOPE_BETA_URL =
      "http://pyroscope-distributor.northwest.service.envoy:10080";

  public PyroscopeProfiler(ServerBaseConfig serverBaseConfig, FlagsConfig flags) {
    this.flags = flags;
    this.serviceName = serverBaseConfig.getServiceName();
  }

  /** Start the profiler */
  public void start() {
    PyroscopeAgent.start(
        new Config.Builder()
            .setApplicationName(serviceName)
            .setProfilingEvent(EventType.ITIMER)
            .setFormat(Format.JFR)
            .setServerAddress(PYROSCOPE_BETA_URL)
            .setProfilingAlloc("512k")
            .setProfilingInterval(Duration.ofMillis(10))
            .setUploadInterval(Duration.ofSeconds(30))
            .build());
  }

  /** Stop the profiler */
  public void stop() {
    PyroscopeAgent.stop();
  }
}

Distributor error

"{\\\"code\\\":\\\"invalid_argument\\\",\\\"message\\\":\\\"unexpected EOF\\\"}\\n\"

Distributor config should all be default.

JDK version: 11 Java agent version: 0.18.0

To Reproduce

Steps to reproduce the behavior:

1. Start Pyroscope
2. Start client
3. Check logs in Pyroscope distributor

Expected behavior

Profiles should correctly distributed

Environment

• Infrastructure: Kubernetes
• Deployment tool: Internal tool

Additional Context

The requests does go through internal envoy proxy, I'm not sure if it affects the request payload.

[korniltsev]

Does it work with no envoy? What does envoy do? Can you share an example of JFR file which is not accepted? Does it work from non-arm64 machines

[kolesnikovae]

We chatted with @xiangyuwang-stripe: they managed to fix it by setting PYROSCOPE_EXPORT_COMPRESSION_LEVEL_JFR to NO_COMPRESSION

I suggest that we move it to https://github.com/grafana/pyroscope-java

[xiangyuwang-stripe]

I'm not super familiar with IO stuff. I'm seeing people said you might need to close the stream before flush to tell readers where to stop. Do we need to close the buffer before flushing here ?

[korniltsev]

This is the reason to use flush instead of close https://github.com/square/okhttp/issues/7692