aws-lambda-redshift-copy icon indicating copy to clipboard operation
aws-lambda-redshift-copy copied to clipboard

Unable to fetch key value

Open Rammmsankar opened this issue 7 years ago • 7 comments

Hi

CloudWatch shows the following error :

An error occurred (NoSuchKey) when calling the GetObject operation: The specified key does not exist.: NoSuchKey Traceback (most recent call last): File "/var/task/copy.py", line 43, in handler raise e NoSuchKey: An error occurred (NoSuchKey) when calling the GetObject operation: The specified key does not exist.

Appreciate if you could help.

Regards Ramu

Rammmsankar avatar Oct 13 '17 12:10 Rammmsankar

Resolved it by replacing s3 = boto3.client('s3') with s3client = boto3.client('s3', endpoint_url='s3-external-1.amazonaws.com')

but now facing another issue as below Unable to import module 'copy': /var/task/psycopg2/_psycopg.so: undefined symbol: _Py_ZeroStruct

Rammmsankar avatar Oct 13 '17 13:10 Rammmsankar

Did you find a work around? I've got the same issue (/var/task/psycopg2/_psycopg.so: undefined symbol: _Py_ZeroStruct

dre2004 avatar Oct 20 '17 05:10 dre2004

@Rammmsankar @dre2004 could you please upload a screenshot of the folder/files structure of the zip file you're uploading to AWS Lambda? Also the configuration, especially for the handler. Did you choose v 2.7 for Python?

chrismld avatar Oct 20 '17 14:10 chrismld

@christianhxc , i get same issue now /var/task/psycopg2/_psycopg.so: undefined symbol: _Py_ZeroStruct .. can you please let me know what was the fix for this?

sureshbabumandava avatar Jun 28 '18 23:06 sureshbabumandava

Hey @christianhxc , I am running into the similar error (using Python 3.6 in AWS Lambda):

Unable to import module 'copy': /var/task/psycopg2/_psycopg.so: undefined symbol: _Py_ZeroStruct

I've commented most of your code and simply want to experiment a select * from the Redshift DB by connecting with it. SS follows:

image

AakashBasu avatar Apr 16 '19 08:04 AakashBasu

I used this repo to solve the issue:

https://github.com/jkehler/awslambda-psycopg2

But now stuck at:

could not translate host name "jdbc:redshift://redshift-cluster-abc.grga543tf.us-west-1.redshift.amazonaws.com" to address: Name or service not known

I think it is an issue with the string I am passing and can be solved. Still if anyone came across this problem, do help.

AakashBasu avatar Apr 16 '19 09:04 AakashBasu

@AakashBasu I'm glad it worked by using the other code you found. I'm planning on updating the code by using lambda layers, stay tuned.

Now, regarding the connection string, it looks like the ":redshift:" is the problem. The way I connect to Redshif is by using the below code where I basically build the connection string:

conn = psycopg2.connect("dbname=" + db_database
                                + " user=" + db_user
                                + " password=" + db_password
                                + " port=" + db_port
                                + " host=" + db_host)

chrismld avatar Apr 17 '19 11:04 chrismld