replace json-iterator/go with reflection-free dependency to reenable dead code elimination

[kruskall]

Component(s)

pdata

What happened?

Describe the bug

any binary that depends on go.opentelemetry.io/collector/pdata has DCE (dead code elimination) disabled due to a dependency on https://github.com/json-iterator/go

Any use of reflect.Value.MethodByName() or reflect.Type.MethodByName() disables DCE. The compiler assumes that these functions will look any method up, and cannot remove methods even if they are never called.

Steps to reproduce

build a binary that depends on go.opentelemetry.io/collector/pdata

What did you expect to see?

a small binary

What did you see instead?

a huge binary

Collector version

anything after https://github.com/open-telemetry/opentelemetry-collector/pull/4986

Environment information

Environment

OS: (e.g., "Ubuntu 20.04") Compiler(if manually compiled): (e.g., "go 14.2")

OpenTelemetry Collector configuration

Log output

Additional context

No response

[github-actions[bot]]

Pinging code owners:

• pdata: @BogdanDrutu @dmitryax

See Adding Labels via Comments if you do not have permissions to add labels yourself.

[debugmiller]

Our binaries have bloated to more than 100 MB in size because of this issue. It would be great to fix.

[simitt]

@kruskall do you happen to have any concrete suggestions for alternatives?

[kruskall]

@kruskall do you happen to have any concrete suggestions for alternatives?

go stdlib is gonna include a json v2 package (currently available at https://github.com/go-json-experiment/json). That would be a good starting point which becomes one less dependency once it's available in the stdlib. Another option is to bump github.com/modern-go/reflect2 (that's the jsoniter dependency causing DCE to be disabled) to at least v1.0.3-0.20250322232337-35a7c28c31ee (that's the fixed version).

[bogdandrutu]

@kruskall how can you check if DCE is disable and what causes that?

[bogdandrutu]

@debugmiller how can you measure that? Also what does your "binaries" include?

[bogdandrutu]

Based on my example in https://github.com/open-telemetry/opentelemetry-collector/pull/13373, there is improvement with the replacement, but I am not near close to 100 MB

[bogdandrutu]

Tried also https://github.com/open-telemetry/opentelemetry-collector/pull/13374 and improvement is very little, 300KiB...

[debugmiller]

I used kruskall's suggestion to bump the version of github.com/modern-go/reflect2 which fixed my problem. FWIW I think this is actually an issue that needs to be addressed upstream in json-iterator directly rather than opentelemetry.

To directly answer your question, at my company our largest binaries compile to approx. 250 MB. This is quite large, primarily because we includes many third party dependencies such as aws-sdk-go-v2. Without dead code elimination, the compiler must include all of the functionality within the pkg instead of just the portions of the library that your code uses.

[bogdandrutu]

@debugmiller

I used kruskall's suggestion to bump the version of github.com/modern-go/reflect2 which fixed my problem. FWIW I think this is actually an issue that needs to be addressed upstream in json-iterator directly rather than opentelemetry.

Did you just replace that in the final binary?

[debugmiller]

I did

go get github.com/modern-go/[email protected]

[kruskall]

@kruskall how can you check if DCE is disable and what causes that?

The quickest way is to add a placeholder func next to the main entrypoint. e.g. a func dcetest() {} placeholder can be added in the main file (https://github.com/open-telemetry/opentelemetry-collector/blob/main/cmd/otelcorecol/main.go), compile the binary and ensures the function is not included (strings <binary> | grep dcetest) Another option is to use https://github.com/aarzilli/whydeadcode

Based on my example in https://github.com/open-telemetry/opentelemetry-collector/pull/13373, there is improvement with the replacement, but I am not near close to 100 MB Tried also https://github.com/open-telemetry/opentelemetry-collector/pull/13374 and improvement is very little, 300KiB...

The PR is building otelcorecol while this issue is about the pdata package. That module includes a lot more dependencies and DCE could still be disabled if there are more problematic dependencies (otel is not exactly the pinnacle of minimal software 😅) The binary size improvement depends on the amount of bloat/deadcode being removed by DCE

[bogdandrutu]

After https://github.com/open-telemetry/opentelemetry-collector/pull/14079 and https://github.com/open-telemetry/opentelemetry-collector/pull/14078 there is only one library that disables DCE which is the use of html/template in the service/internal/zpages as well as in the go.opentelemetry.io/contrib/zpages. Tested (using https://github.com/aarzilli/whydeadcode) without the zpages component which also disables the use of service/internal/zpages there are no more errors. There is https://github.com/golang/go/issues/72895 which can help with this, but does not look to be prioritized.

$ go build -ldflags=-dumpdep -tags "grpcnotrace" ./... |& whydeadcode 
text/template.(*state).evalField reachable from:
         text/template.(*state).evalFieldChain
         text/template.(*state).evalCommand
         text/template.(*state).evalPipeline
         text/template.(*state).walk
         text/template.(*Template).execute
         html/template.(*Template).Execute
         go.opentelemetry.io/contrib/zpages.(*tracezHandler).ServeHTTP
         type:*go.opentelemetry.io/contrib/zpages.tracezHandler
         type:go.opentelemetry.io/contrib/zpages.tracezHandler
         go.opentelemetry.io/collector/extension/zpagesextension.(*zpagesExtension).Start
         type:*go.opentelemetry.io/collector/extension/zpagesextension.zpagesExtension
         type:go.opentelemetry.io/collector/extension/zpagesextension.zpagesExtension
         go.opentelemetry.io/collector/extension/zpagesextension.create
         go.opentelemetry.io/collector/extension/zpagesextension.create·f
         main.components
         main.components·f
         main.main
         runtime.main_main·f
         runtime.main
         runtime.mainPC
         runtime.rt0_go
         _rt0_arm64_darwin
         main
         _

With the 2 PRs binary size decreases to 40MB from 43MB, and without zpages extension binary size decreases to 31MB.