Simplify the UX for writing OpenTelemetry export scripts

[philkuz]

Is your feature request related to a problem? Please describe. Users have complained that the UX for writing OpenTelemetry export scripts is painful. The configuration is very verbose an highly repetitive which makes it annoying to write these scripts:

df = pxviews.http_graph(px.plugin.start_time, px.plugin.end_time)
df = df[['time_', 'responder_pod', 'requestor_pod', 'responder_node', 'requestor_node',
               'responder_namespace', 'requestor_namespace',
               'responder_ip', 'requestor_ip','responder_service', 'requestor_service',
               'latency_quantiles', 'latency_sum', 'num_requests',
               'window_start_time', 'window_end_time',
               'num_errors', 'req_bytes', 'resp_bytes']]
df.latency_p01 = px.pluck_float64(df.latency_quantiles, 'p01')
df.latency_p10 = px.pluck_float64(df.latency_quantiles, 'p10')
df.latency_p25 = px.pluck_float64(df.latency_quantiles, 'p25')
df.latency_p50 = px.pluck_float64(df.latency_quantiles, 'p50')
df.latency_p75 = px.pluck_float64(df.latency_quantiles, 'p75')
df.latency_p90 = px.pluck_float64(df.latency_quantiles, 'p90')
df.latency_p99 = px.pluck_float64(df.latency_quantiles, 'p99')

df.latency_sum = df.latency_sum * 1.0

df = add_otel_attributes(df)
px.display(df, 'http_graph')
px.export(df, px.otel.Data(
    resource={
        'service.name': df.responder_service,
        'service.instance.id': df.responder_pod,
        'px.http_graph.responder_pod': df.responder_pod,
        'px.http_graph.requestor_pod': df.requestor_pod,
        'px.http_graph.responder_namespace': df.responder_namespace,
        'px.http_graph.requestor_namespace': df.requestor_namespace,
        'px.http_graph.responder_node': df.responder_node,
        'px.http_graph.requestor_node': df.requestor_node,
        'px.http_graph.responder_ip': df.responder_ip,
        'px.http_graph.requestor_ip': df.requestor_ip,
        'px.http_graph.responder_service': df.responder_service,
        'px.http_graph.requestor_service': df.requestor_service,
        'px.http_graph.window_start_time': df.window_start_time,
        'px.http_graph.window_end_time': df.window_end_time,
        'k8s.pod.name': df.responder_pod,
        'k8s.node.name': df.responder_node,
        'k8s.namespace.name': df.responder_namespace,
        'k8s.cluster.name': df.cluster_name,
        'px.cluster.id': df.cluster_id,
        'instrumentation.provider': df.provider,
    },
    data=[
        px.otel.metric.Gauge(
            name='px.http_graph.num_errors',
            description='The number of erroroneous HTTP requests made to the pod. An error occurs response with a status code >= 400',
            value=df.num_errors,
        ),
        px.otel.metric.Gauge(
            name='px.http_graph.num_requests',
            description='The total number of HTTP requests made to the pod, regardless of success or failure',
            value=df.num_requests,
        ),
        px.otel.metric.Gauge(
            name='px.http_graph.req_bytes',
            description='The number of bytes received over HTTP in the pod',
            value=df.req_bytes,
        ),
        px.otel.metric.Gauge(
            name='px.http_graph.resp_bytes',
            description='The number of bytes sent from the pod over HTTP in the pod',
            value=df.resp_bytes,
        ),
        px.otel.metric.Summary(
            name='px.http_graph.latency',
            description='The latency of the pod',
            count=df.num_requests,
            sum=df.latency_sum,
            quantile_values={
                0.01: df.latency_p01,
                0.10: df.latency_p10,
                0.25: df.latency_p25,
                0.50: df.latency_p50,
                0.75: df.latency_p75,
                0.90: df.latency_p90,
                0.99: df.latency_p99,
            },
        )
    ],
))

Describe the solution you'd like Users with a script that returns DataFrame(s) (aka they must call px.display(df)) should be able to tell Pixie to generate the script. We'll have this available in the public API and hook that up to the following UI component:

Users who write a Scratchpad script can then opt to call "Export Script". That'll take the script, generate the OTel export script, and drop them into the "Create Custom Script" flow https://work.withpixie.dev/configure-data-export/create with that script where they can configured the options

Describe alternatives you've considered

1. Hide the OTel export configuration from the user

• We assume that users will want to add/subtract columns from generated scripts, so this isn't a great solution.

2. Create a UI that allows users to select columns to create otel configs for

• not out of the question for the future, but navigating a UI like this might be rather clunky. Users who write these scripts will already be familiar with PxL syntax and used to the text editor interface so they'd probably prefer the text editor.