examples: Add OpenTelemetry example

[zasweq]

This PR adds an OpenTelemetry instrumentation example.

The example is not tested (I have a commit that does have the test, but deleted it, so the diff can be seen). This is due to our examples test expecting the client to exit, which in this case the client cannot exit for the sake of the example, as it is exporting Prometheus metrics on a port. Let me know if you would prefer changing our examples test. Note that a CSM Observability example with exact same behavior expect configure OpenTelemetry through CSM Package will be coming right after this.

RELEASE NOTES: N/A

[easwars]

Given that OTel is a separate submodule, why not have the example inside that module? We take that approach for advancedtls I believe.

[zasweq]

Hmmmm sure.

[zasweq]

There is a CSM Example in flight that we plugged into UG that is similar to this; I think it would be best to wait for our full back and forth on that and then incorporate all changes there into this. I already did some changes such as flags to align with UG and also relaxing the RPC failure on client side to a Info vs. Error to keep the binary running even if RPC's fail. That way, erroring RPC's can also be seen through Prometheus (say if the Server Restarts on Kubernetes for CSM Observability example, or locally for this example), and doesn't require to restart binary (on Kubernetes for CSM Observability example and locally for this example.)

[codecov[bot]]

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 81.28%. Comparing base (bb49a88) to head (7ef4259).

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #7296      +/-   ##
==========================================
- Coverage   81.33%   81.28%   -0.05%     
==========================================
  Files         348      348              
  Lines       26744    26744              
==========================================
- Hits        21752    21739      -13     
- Misses       3795     3804       +9     
- Partials     1197     1201       +4     

see 11 files with indirect coverage changes