Working version for Lambda - AWS Managed Prometheus integration

[pcolazurdo]

This PR shows a fully working Lambda function based on Python sending metrics to AWS Managed Prometheus using an OTel Layer.

A few comments:

• As force_flush is not defined yet to wait for the actual collector to flush the metrics we add a fake wait of 1 second at the end of the function call for it to succeed. This should be removed once the SDK/API team solve the discussions around force_flush functionality for FaaS
• The current version of the Python opentelemetry-api module (1.11.1) has issues with the protobuf version which makes the library unusable (more info). To solve this we are asking the Function to use the python implementation of the protobuf compiler on the fly by adding an Environment variable PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION: python in the template.yaml for the function.
• Also, the current version of the opentelemetry-api module (1.11.1) marks the metrics api as internal, that's why _metrics is used. Once the next stable version is released, I'll update the code with a new PR.

[linux-foundation-easycla[bot]]

• :white_check_mark: login: EddieEldridge / name: Medik (52268bf422ff67757b45258cee87362846005711)
• :white_check_mark: login: vasireddy99 / name: Vasi Vasireddy (a9a4d80c889af954d54624cb25a05f97bc5f0daf, 34e37f28012753e70756e8cedc4c5530f88cdd83, 7208181421cd6da2f7915adc28addb6098c7d2af, c44e382639fb33918827d4450ca63ccdaa0f686c)
• :x: - login: @pcolazurdo / name: Pablo Colazurdo . The commit (5593c75e958a36e5116cc86b093d10cc310316a5, 25d8121cbd2bcbdb3cb773ce4b41fda0ea7dfabf, 85b97a73a0dcc2ef0e1c67b3c76f6c7045a87518, 094b2bed1579702265039b837c3e7bbea9b767e3, add5813076b17be73bf91bbdbf1134c132c25188, d76c827ed62e21aa00c6f1ff682ee76a83d471f5, 33f9a28a257b2cd9ed76c0ea5f7e18b4432299ae, aad88095dee3e0f1de59c93038a8a954f016e383, 28045f73fc7ecb4e0809c145a5580a86b0bcf7d7, 5492ec207ce3925cbd0de48cf98a05885d4d55a3, e5d6194784a9a81b520b99d06d9999fc04b2afe4) is not authorized under a signed CLA. Please click here to be authorized. For further assistance with EasyCLA, please submit a support request ticket.

[NathanielRN]

@pcolazurdo Are you able to fix the unit tests?

I'll defer to @Aneurysm9 for the approval here 🙂

[pcolazurdo]

Updated to match latest commit on main. Some comments:

• go.mod and go.sum are different because I had to include awsprometheuswriter. This is because the current collector doesn't seem to include the sigv4auth extension
• config.yaml is modified on the collector/ path because there isn't a mechanism defined to override configuration. I would have used AOT_CONFIG_CONTENT environment variable as discussed here but this option doesn't seem to be available in the stripped-down version version of the collector in the project.

[erichsueh3]

Updated to match latest commit on main. Some comments:

• go.mod and go.sum are different because I had to include awsprometheuswriter. This is because the current collector doesn't seem to include the sigv4auth extension
• config.yaml is modified on the collector/ path because there isn't a mechanism defined to override configuration. I would have used AOT_CONFIG_CONTENT environment variable as discussed here but this option doesn't seem to be available in the stripped-down version version of the collector in the project.

Hi @pcolazurdo, can you clarify on your first point? The sigv4auth extension has been included since 0.48, and the current collector version is latest, at 0.56.

[pcolazurdo]

@erichsueh3 I've tried initialising the extension and I just get an extension crash on Lambda. As soon as I add:

extensions:
  sigv4auth:
    region: "us-west-2"
...
service:
  extensions: [sigv4auth]

I get:

2022/08/02 11:41:36 Failed to start the extension: unable to start, otelcol state is 3 

Also, looking in the source code in the repo for sigv4authextension, I couldn't find any reference to it, not even in the go.mod.

[vasireddy99]

This example in the PR is really helpful.

• prometheusremotewriteexporter with sigv4 auth extension can now be used as PR is merged.
• I second this comment as it would be the be better case example as in the downstream repo used for integration tests.
• force_flush() is now supported for metrics in v1.12.0, although this issue still exists in opentelemetry-instrumentation-aws-lambda.

[pcolazurdo]

Lot has changed in the project since I built this demo and now. It served its purpose but it is now mostly obsolete. I'll close this for now to avoid people getting confused, and try to create a new one in the coming weeks (no promise)