scrapy-dotpersistence
scrapy-dotpersistence copied to clipboard
add AWS_SESSION_TOKEN to make it works with AWS lambda (need executio…
trafficstars
…n role instead of user role)
i use scrappy-dotpersistence with scrappy-deltafetch to store result on aws S3.
The code run in a lambda. For now i had to provide my personnal credential to make it works. 🥴
I investigated, and lambda use default credentials, and it say, user doesn't exists.
It's because AWS lambda use execution role
When you make a call using temporary security credentials, the call must include a session token, which is returned along with those temporary credentials. AWS uses the session token to validate the temporary security credentials.
see https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_temp_use-resources.html