pixie
pixie copied to clipboard
Published
20 hours ago •
pixie-io
pixie-io
Upgrade finagle to version that includes tls tracing compatible netty-tcnative
This PR upgrades finagle to install a version that includes the netty changes I upstreamed (https://github.com/netty/netty/pull/12438, https://github.com/netty/netty-tcnative/pull/731) to support #407. Once this is upgraded, the next step is to refresh #463.
Testing
- [x] Ran the thriftmux container and verified that the logs of the client show that netty tcnative is used (improperly installed netty tcnative will silently succeed)
vagrant@vagrant:/vagrantbazel run -c dbg src/stirling/source_connectors/socket_tracer/testing/containers/thriftmux:server_image
vagrant@vagrant:/vagrant$ docker run -it bazel/src/stirling/source_connectors/socket_tracer/testing/containers/thriftmux:server_image --use-tls true
# Run client inside the server container and see netty-tcnative is used successfully
/ # /usr/bin/java -cp @/app/px/src/stirling/source_connectors/socket_tracer/testing/containers/thriftmux/server_image.classpath Client --use-tls true 2>&1 | grep -i tcnative
01:03:03.586 [finagle/netty4-1-1] DEBUG io.netty.util.internal.NativeLibraryLoader - Successfully loaded the library /tmp/libnetty_tcnative_linux_x86_64685160139595718633.so
01:03:03.587 [finagle/netty4-1-1] DEBUG io.netty.util.internal.NativeLibraryLoader - Loaded library with name 'netty_tcnative_linux_x86_64'
01:03:03.587 [finagle/netty4-1-1] DEBUG io.netty.handler.ssl.OpenSsl - Initialize netty-tcnative using engine: 'default'
Run
make buildifier
To reform the build file. The Jenkins failure is because of BUILD.bazel file's line's ordering are not sorted according to the style guide.
0 / 1,544] [Prepa] BazelWorkspaceStatusAction stable-status.txt ... (3 actions, 0 running)
ERROR: /root/.cache/bazel/_bazel_root/4b2a6651efb8be1e285d05f4edee89cf/external/maven/BUILD:1040:11: Creating compile jar for @maven//:io_netty_netty_codec_http2 failed: (Aborted): AddJarManifestEntry failed: error executing command
(cd /root/.cache/bazel/_bazel_root/4b2a6651efb8be1e285d05f4edee89cf/sandbox/linux-sandbox/1410/execroot/px && \
exec env - \
bazel-out/k8-opt-exec-50AE0418-ST-d2f0ea542b71/bin/external/rules_jvm_external/private/tools/java/rules/jvm/external/jar/AddJarManifestEntry --source external/maven/v1/https/repo1.maven.org/maven2/io/netty/netty-codec-http2/4.1.78.Final/netty-codec-http2-4.1.78.Final.jar --output bazel-out/k8-opt-ST-4a519fd6d3e4/bin/external/maven/v1/https/repo1.maven.org/maven2/io/netty/netty-codec-http2/4.1.78.Final/header_netty-codec-http2-4.1.78.Final.jar --remove-entry Class-Path)
# Configuration: 122cfd117844598832e157b473675f04e637a4ebad9f6e1a4695ba9a512508d8
# Execution platform: @local_execution_config_platform//:platform
Happened when the following was ran:
bazel build //src/stirling/bpf_tools:bcc_symbolizer_bpf_test_lib //src/stirling/bpf_tools:bcc_wrapper_bpf_test_lib //src/stirling/bpf_tools:bpftrace_wrapper_bpf_test_lib //src/stirling/bpf_tools:uprobe_extra_trigger_bpf_test_lib //src/stirling/e2e_tests:bpf_map_leak_bpf_test_lib //src/stirling/e2e_tests:stirling_bpf_test_lib //src/stirling/obj_tools:elf_reader_symbolizer_bpf_test_lib //src/stirling/source_connectors/cpu_stat_bpftrace:cpu_stat_bpftrace_connector_bpf_test_lib //src/stirling/source_connectors/dynamic_bpftrace:dynamic_bpftrace_connector_bpf_test_lib //src/stirling/source_connectors/dynamic_tracer:dynamic_trace_bpf_test_lib //src/stirling/source_connectors/dynamic_tracer:stirling_dt_bpf_test_lib //src/stirling/source_connectors/perf_profiler:perf_profiler_bpf_test_lib //src/stirling/source_connectors/perf_profiler:stringifier_bpf_test_lib //src/stirling/source_connectors/proc_exit:proc_exit_trace_bpf_test_lib //src/stirling/source_connectors/socket_tracer:amqp_trace_bpf_test_lib //src/stirling/source_connectors/socket_tracer:conn_stats_bpf_test_lib //src/stirling/source_connectors/socket_tracer:cql_trace_bpf_test_lib //src/stirling/source_connectors/socket_tracer:dns_trace_bpf_test_lib //src/stirling/source_connectors/socket_tracer:dyn_lib_trace_bpf_test_lib //src/stirling/source_connectors/socket_tracer:go_tls_trace_bpf_test_lib //src/stirling/source_connectors/socket_tracer:grpc_trace_bpf_test_lib //src/stirling/source_connectors/socket_tracer:http2_trace_bpf_test_lib //src/stirling/source_connectors/socket_tracer:http_trace_bpf_test_lib //src/stirling/source_connectors/socket_tracer:kafka_trace_bpf_test_lib //src/stirling/source_connectors/socket_tracer:mux_trace_bpf_test_lib //src/stirling/source_connectors/socket_tracer:mysql_trace_bpf_test_lib //src/stirling/source_connectors/socket_tracer:nats_trace_bpf_test_lib //src/stirling/source_connectors/socket_tracer:openssl_trace_bpf_test_lib //src/stirling/source_connectors/socket_tracer:pgsql_trace_bpf_test_lib //src/stirling/source_connectors/socket_tracer:redis_trace_bpf_test_lib //src/stirling/source_connectors/socket_tracer:socket_trace_bpf_test_lib //src/stirling/source_connectors/stirling_error:stirling_error_bpf_test_lib //src/stirling/utils:detect_application_bpf_test_lib //src/stirling/utils:proc_path_tools_container_bpf_test_lib --compilation_mode=opt --config=bpf --build_metadata=COMMIT_SHA=eec5a47cac5f7619d4e9c86eb79209d726ae4c3a --target_pattern_file=bazel_buildables_bpf --build_metadata=CONFIG=bpf
@philkuz thanks for identifying the issue. I ran the command you shared above (removing the --target_pattern_file flag since that cause an error) and I'm not able to reproduce that problem 🤔.
# Verify that the cli args without modification results in an error
vagrant@vagrant:/vagrant$ bazel build //src/stirling/bpf_tools:bcc_symbolizer_bpf_test_lib //src/stirling/bpf_tools:bcc_wrapper_bpf_test_lib //src/stirling/bpf_tools:bpftrace_wrapper_bpf_test_lib //src/stirling/bpf_tools:uprobe_extra_trigger_bpf_test_lib //src/stirling/e2e_tests:bpf_map_leak_bpf_test_lib //src/stirling/e2e_tests:stirling_bpf_test_lib //src/stirling/obj_tools:elf_reader_symbolizer_bpf_test_lib //src/stirling/source_connectors/cpu_stat_bpftrace:cpu_stat_bpftrace_connector_bpf_test_lib //src/stirling/source_connectors/dynamic_bpftrace:dynamic_bpftrace_connector_bpf_test_lib //src/stirling/source_connectors/dynamic_tracer:dynamic_trace_bpf_test_lib //src/stirling/source_connectors/dynamic_tracer:stirling_dt_bpf_test_lib //src/stirling/source_connectors/perf_profiler:perf_profiler_bpf_test_lib //src/stirling/source_connectors/perf_profiler:stringifier_bpf_test_lib //src/stirling/source_connectors/proc_exit:proc_exit_trace_bpf_test_lib //src/stirling/source_connectors/socket_tracer:amqp_trace_bpf_test_lib //src/stirling/source_connectors/socket_tracer:conn_stats_bpf_test_lib //src/stirling/source_connectors/socket_tracer:cql_trace_bpf_test_lib //src/stirling/source_connectors/socket_tracer:dns_trace_bpf_test_lib //src/stirling/source_connectors/socket_tracer:dyn_lib_trace_bpf_test_lib //src/stirling/source_connectors/socket_tracer:go_tls_trace_bpf_test_lib //src/stirling/source_connectors/socket_tracer:grpc_trace_bpf_test_lib //src/stirling/source_connectors/socket_tracer:http2_trace_bpf_test_lib //src/stirling/source_connectors/socket_tracer:http_trace_bpf_test_lib //src/stirling/source_connectors/socket_tracer:kafka_trace_bpf_test_lib //src/stirling/source_connectors/socket_tracer:mux_trace_bpf_test_lib //src/stirling/source_connectors/socket_tracer:mysql_trace_bpf_test_lib //src/stirling/source_connectors/socket_tracer:nats_trace_bpf_test_lib //src/stirling/source_connectors/socket_tracer:openssl_trace_bpf_test_lib //src/stirling/source_connectors/socket_tracer:pgsql_trace_bpf_test_lib //src/stirling/source_connectors/socket_tracer:redis_trace_bpf_test_lib //src/stirling/source_connectors/socket_tracer:socket_trace_bpf_test_lib //src/stirling/source_connectors/stirling_error:stirling_error_bpf_test_lib //src/stirling/utils:detect_application_bpf_test_lib //src/stirling/utils:proc_path_tools_container_bpf_test_lib --config=bpf --compilation_mode=opt --build_metadata=COMMIT_SHA=eec5a47cac5f7619d4e9c86eb79209d726ae4c3a --target_pattern_file=bazel_buildables_bpf --build_metadata=CONFIG=bpf
ERROR: Command-line target pattern and --target_pattern_file cannot both be specified
vagrant@vagrant:/vagrant$ bazel build //src/stirling/bpf_tools:bcc_symbolizer_bpf_test_lib //src/stirling/bpf_tools:bcc_wrapper_bpf_test_lib //src/stirling/bpf_tools:bpftrace_wrapper_bpf_test_lib //src/stirling/bpf_tools:uprobe_extra_trigger_bpf_test_lib //src/stirling/e2e_tests:bpf_map_leak_bpf_test_lib //src/stirling/e2e_tests:stirling_bpf_test_lib //src/stirling/obj_tools:elf_reader_symbolizer_bpf_test_lib //src/stirling/source_connectors/cpu_stat_bpftrace:cpu_stat_bpftrace_connector_bpf_test_lib //src/stirling/source_connectors/dynamic_bpftrace:dynamic_bpftrace_connector_bpf_test_lib //src/stirling/source_connectors/dynamic_tracer:dynamic_trace_bpf_test_lib //src/stirling/source_connectors/dynamic_tracer:stirling_dt_bpf_test_lib //src/stirling/source_connectors/perf_profiler:perf_profiler_bpf_test_lib //src/stirling/source_connectors/perf_profiler:stringifier_bpf_test_lib //src/stirling/source_connectors/proc_exit:proc_exit_trace_bpf_test_lib //src/stirling/source_connectors/socket_tracer:amqp_trace_bpf_test_lib //src/stirling/source_connectors/socket_tracer:conn_stats_bpf_test_lib //src/stirling/source_connectors/socket_tracer:cql_trace_bpf_test_lib //src/stirling/source_connectors/socket_tracer:dns_trace_bpf_test_lib //src/stirling/source_connectors/socket_tracer:dyn_lib_trace_bpf_test_lib //src/stirling/source_connectors/socket_tracer:go_tls_trace_bpf_test_lib //src/stirling/source_connectors/socket_tracer:grpc_trace_bpf_test_lib //src/stirling/source_connectors/socket_tracer:http2_trace_bpf_test_lib //src/stirling/source_connectors/socket_tracer:http_trace_bpf_test_lib //src/stirling/source_connectors/socket_tracer:kafka_trace_bpf_test_lib //src/stirling/source_connectors/socket_tracer:mux_trace_bpf_test_lib //src/stirling/source_connectors/socket_tracer:mysql_trace_bpf_test_lib //src/stirling/source_connectors/socket_tracer:nats_trace_bpf_test_lib //src/stirling/source_connectors/socket_tracer:openssl_trace_bpf_test_lib //src/stirling/source_connectors/socket_tracer:pgsql_trace_bpf_test_lib //src/stirling/source_connectors/socket_tracer:redis_trace_bpf_test_lib //src/stirling/source_connectors/socket_tracer:socket_trace_bpf_test_lib //src/stirling/source_connectors/stirling_error:stirling_error_bpf_test_lib //src/stirling/utils:detect_application_bpf_test_lib //src/stirling/utils:proc_path_tools_container_bpf_test_lib --config=bpf --compilation_mode=opt --build_metadata=COMMIT_SHA=eec5a47cac5f7619d4e9c86eb79209d726ae4c3a --build_metadata=CONFIG=bpf
INFO: Analyzed 34 targets (702 packages loaded, 89714 targets configured).
INFO: Found 34 targets...
INFO: From Building external/io_bazel_rules_scala/src/java/io/bazel/rulesscala/scalac/scalac.jar (4 source files):
warning: [path] bad path element "/home/vagrant/.cache/bazel/_bazel_vagrant/6ad5fdbcbf2eaa93bd62f92333a2e6e5/execroot/px/bazel-out/k8-opt-exec-50AE0418-ST-d2f0ea542b71/bin/external/io_bazel_rules_scala_scala_compiler/io_bazel_rules_scala_scala_compiler.stamp/scala-reflect.jar": no such file or directory
warning: [path] bad path element "/home/vagrant/.cache/bazel/_bazel_vagrant/6ad5fdbcbf2eaa93bd62f92333a2e6e5/execroot/px/bazel-out/k8-opt-exec-50AE0418-ST-d2f0ea542b71/bin/external/io_bazel_rules_scala_scala_compiler/io_bazel_rules_scala_scala_compiler.stamp/scala-library.jar": no such file or directory
INFO: From scala //src/stirling/source_connectors/socket_tracer/testing/containers/thriftmux:thrift_library:
warning: 3 deprecations (since 2.13.0); re-run with -deprecation for details
1 warning
INFO: From Executing genrule @com_github_antlr_grammars_v4//:mysql_parser_gen:
warning(154): MySQLParser.g4:1347:0: rule blockStatement contains an optional block with at least one alternative that can match an empty string
INFO: Elapsed time: 3242.679s, Critical Path: 358.46s
INFO: 6023 processes: 1152 internal, 4844 linux-sandbox, 27 worker.
INFO: Build completed successfully, 6023 total actions