Unable to import library from layer while using opentelemetry-lambda layer

[guyfrid]

Describe the bug

I have an AWS Lambda function that uses the jwt library. The jwt library is located in a layer, with the following directory structure: python/lib/python3.8/site-packages/jwt. This path is valid according to the AWS Lambda Layers documentation.

The Lambda function is working as expected.

Now, I want to instrument the Lambda function with OpenTelemetry. I added the opentelemetry-lambda layer (arn:aws:lambda:eu-central-1:901920570463:layer:aws-otel-python-amd65-ver-1-16-0:2) to the Lambda function. However, when I try to run the Lambda function, I get the following error message:

{
  "errorMessage": "Unable to import module 'otel_wrapper': No module named 'jwt'",
  "errorType": "Runtime.ImportModuleError",
  "stackTrace": []
}

Steps to reproduce

1. create python3.8 lambda and copy flowing code:

import jwt

def lambda_handler(event, context):
    token = 'some_to'
    try:
        decoded = jwt.decode(token, 'mysecret', algorithms=['HS256'])
    except jwt.InvalidTokenError:
        return {
            'statusCode': 401,
            'body': 'Invalid token'
        }
    
    return {
        'statusCode': 200,
        'body': f'Hello, {decoded.get("name")}!'
    }

2. create layer contains the pyjwt(2.4.0) library under python/lib/python3.8/site-packages
3. add the jwt layer and the opentelemetry-lambda layer to the lambda
4. invoke the lambda by run simple test

What did you expect to see? lambda invoked

What did you see instead? A clear and concise description of what you saw instead. Execution result: failed

{
  "errorMessage": "Unable to import module 'otel_wrapper': No module named 'jwt'",
  "errorType": "Runtime.ImportModuleError",
  "stackTrace": []
}

What version of collector/language SDK version did you use?

1.16.0/0.37b0

What language layer did you use? Python3.8

Additional context It appears that the root cause of this issue lies in the definition of the PYTHONPATH environment variable within the otel-instrument script. Specifically, it appears that the site-packages directory is not included in the PYTHONPATH variable, resulting in an inability to import the jwt library when the Lambda handler is imported.

To resolve this issue, it was necessary to modify the PYTHONPATH environment variable within the otel-instrument script to include the path to the site-packages directory. This modification allowed the jwt library to be successfully imported when the Lambda handler was executed. https://github.com/guyfrid/opentelemetry-lambda/commit/f431c4f9ad87191fa8447e4a97c5e465428454ab

[purple4reina]

I've hit the same issue. Seems it is caused by https://github.com/open-telemetry/opentelemetry-lambda/blob/c6a7138f19999f2c0adb7b5752b263ad76581647/python/src/otel/otel_sdk/otel_wrapper.py#L43-L45

[oestoer]

I hit this issue with the arn:aws:lambda:<region>:184161586896:layer:opentelemetry-python-0_3_0:1 layer that has the versions opentelemetry-python 1.21.0/0.41b0

[antonioazambuja]

I get the same issue with the library dnspython. It happens because jwt or dnspython isn't library auto-instrumentation. There's a list package still developing by auto instrumentation in this repository: https://github.com/open-telemetry/opentelemetry-python-contrib/tree/main/instrumentation. If you want to know what packages to your project supported by auto-instrumentations follow these steps:

• Install packages in your environments;
• Follow these steps to install opentelemetry-bootstrap and run only this command opentelemetry-bootstrap in your environment. You'll see the output like this:

`✗ opentelemetry-bootstrap

opentelemetry-instrumentation-asyncio==0.45b0 opentelemetry-instrumentation-aws-lambda==0.45b0 opentelemetry-instrumentation-dbapi==0.45b0 opentelemetry-instrumentation-logging==0.45b0 opentelemetry-instrumentation-sqlite3==0.45b0 opentelemetry-instrumentation-urllib==0.45b0 opentelemetry-instrumentation-wsgi==0.45b0 opentelemetry-instrumentation-asgi==0.45b0 opentelemetry-instrumentation-boto3sqs==0.45b0 opentelemetry-instrumentation-botocore==0.45b0 opentelemetry-instrumentation-fastapi==0.45b0 opentelemetry-instrumentation-grpc==0.45b0 opentelemetry-instrumentation-requests==0.45b0 opentelemetry-instrumentation-tortoiseorm==0.45b0 opentelemetry-instrumentation-urllib3==0.45b0`

In my case, dnspython library isn't supported.