Convenience function for execute-api calls

[jpbarto]

All,

Can we create a boto3 convenience function that can sign HTTP requests meant for deployed API gateway-hosted customer endpoints protected by IAM? This function would create a Sigv4 signature for the request and enable it to be sent as described in the documentation for IAM action 'execute-api:Invoke':

http://docs.aws.amazon.com/apigateway/latest/developerguide/api-gateway-iam-policy-examples-for-api-execution.html

[jamesls]

I think this would be useful. Maybe it should be extracted out into a separate library so people that just want to make requests to their API gateway endpoint don't have to pull in all of boto3 (the code for signing requests would be much smaller than all of boto3/botocore).

Thoughts?

[aodhan-domhnaill]

Has anyone worked on this? If not, I might do it.

[dobesv]

See also https://stackoverflow.com/a/39357370/399738

[dobesv]

And https://github.com/DavidMuller/aws-requests-auth

[bigunyak]

I think I've spent like an hour trying to figure out how to sign requests with boto3 to call my IAM policy protected API Gateway endpoints. Just couldn't believe this is not available in public interface, especially considering that functionality is there, it's just a matter of exposing it. This is basically the first question which you think about while reading documentation on Resource and IAM policies for API Gateway, how do I sign requests. Please, make it available in boto3 or a separate package, doesn't matter.

[jpbarto]

@bigunyak noting the comment above about the functionality being much smaller than botocore and the note from @dobesv about a 3rd party library that provides this capability what would your feeling be about a convenience library vs a boto3 API Gateway client function? The code you're writing to consume your API Gateway is that using a large amount of Boto3 functionality already or do you think that calling 'Execute-API' is the only functionality needed?

Perhaps a way forward may be to, in the Boto3 API gateway docs, provide note about calling Execute-API with a 3rd party library to include links to some options?

[bigunyak]

The most important thing for me would be that this functionality comes from AWS, it can be a part of boto3 SDK or a separate package but it should be developed and maintained by AWS. This would be much better instead of me searching of available 3rd party packages, which are some years old now and trying to understand what's the difference between them and what I should use. We have boto3 library installed everywhere and we use it in many places, so having it available there would be very practical for us, but a separate package would work as well. Regarding the documentation, yes, it would be very helpful with some links and examples in API Gateway documentation on how build signed requests with which would work with authentication rules based on IAM policies.

[golankopi]

Hi, any update on this topic?

if anyone else encounters this problem, after some digging in the boto3 code I used the following workaround to being able to use IAM authentication for API Gateway calls (in this example for s3):

client = session.client('s3', endpoint_url=<api_gateway_url>)
client._request_signer._signing_name = 'execute-api'
client._request_signer._service_id = ServiceId('execute-api')

[mrpackethead]

Hi, any update on this topic?

if anyone else encounters this problem, after some digging in the boto3 code I used the following workaround to being able to use IAM authentication for API Gateway calls (in this example for s3):

client = session.client('s3', endpoint_url=<api_gateway_url>)
client._request_signer._signing_name = 'execute-api'
client._request_signer._service_id = ServiceId('execute-api')

I have this exact problem now.. I want to be able to make GET/POSTS to a API Gateway.. Do you have any more detail on how you did this.

[amancevice]

I solved this for myself using a custom requests Authorizer instance:

import boto3
import requests
from botocore.auth import SigV4Auth
from botocore.awsrequest import AWSRequest
from botocore.compat import (parse_qsl, urlparse)


class IAMAuth(requests.auth.AuthBase):
    """
    IAM authorizer.

    :param boto3.Session session: Optional boto3 Session object
    :param str service_name: Optional AWS service name

    :Example:

    >>> IAMAuth()
    >>> IAMAuth(boto3.Session(), 'execute-api')
    """
    def __init__(self, boto3_session=None, service_name='execute-api'):
        self.boto3_session = boto3_session or boto3.Session()
        self.sigv4 = SigV4Auth(
            credentials=self.boto3_session.get_credentials(),
            service_name=service_name,
            region_name=self.boto3_session.region_name,
        )

    def __call__(self, request):
        # Parse request URL
        url = urlparse(request.url)

        # Prepare AWS request
        awsrequest = AWSRequest(
            method=request.method,
            url=f'{url.scheme}://{url.netloc}{url.path}',
            data=request.body,
            params=dict(parse_qsl(url.query)),
        )

        # Sign request
        self.sigv4.add_auth(awsrequest)

        # Re-add original headers
        for key, val in request.headers.items():
            if key not in awsrequest.headers:
                awsrequest.headers[key] = val

        # Return prepared request
        return awsrequest.prepare()

Example use:

session = requests.Session()
session.auth = IAMAuth()
session.get('<your-iam-authd-api-url>')

[golankopi]

Hi, any update on this topic? if anyone else encounters this problem, after some digging in the boto3 code I used the following workaround to being able to use IAM authentication for API Gateway calls (in this example for s3):

client = session.client('s3', endpoint_url=<api_gateway_url>)
client._request_signer._signing_name = 'execute-api'
client._request_signer._service_id = ServiceId('execute-api')

I have this exact problem now.. I want to be able to make GET/POSTS to a API Gateway.. Do you have any more detail on how you did this.

@mrpackethead exactly like you see in my example, after monkey patching like that, use the client as you would if you didn't use api gateway (so for my example- with s3 client you can use client.upload_file regularly - the request will be signed for an execute-api request)

[mrpackethead]

Example use:

session = requests.Session()
session.auth = IAMAuth()
session.get('<your-iam-authd-api-url>')

Thanks! thats really awsome.

[amancevice]

you are welcome! I turned this into a pip real quick for anyone's convenience: https://github.com/amancevice/requests-iamauth

[mrpackethead]

thanks, its a good start for what i needed, I have to deal with 2FA and named profiles but this just what i needed for the job!

[juanlucky]

hello there, thanks @amancevice for sharing the IAMAuth code. Unfortunately, I couldn't succeed to run it successfully. Could you please complete your provided sample code with a first Cognito user login (user/pwd) and then call an API using the access token obtained for this user? Or help me to understand what it is wrong with the following code ?

I generated an node js /movies API with Amplify secured with Cognito IAM user. I succeeded to call this API within a Vue.js webapp using the amplify JS client SDK. A user can log in via Cognito (vue.js amplify component) and the call to /movies API returns the defined movies of the logged user. Good, now I want to do the same thing but from a Python app.

I first tried the following code 'copied from your sample) :

session = requests.Session()
session.auth = IAMAuth()
response = session.get( "https://ol5oyvr5rg.execute-api.eu-west-1.amazonaws.com/test/movies")

I received a 200 but with no data as it is a call with no logged user first.

Then I successfully log in as a defined Cognito user in the Python app using module pycognito. But when I tried to call the /movies API using a boto3 session used for with the login, I received a 403 error

from pycognito.aws_srp import AWSSRP
#Init a boto3 session
awsSession = boto3.session.Session(
                    aws_access_key_id = "myAppClientId", 
                    region_name = "eu-west-1")
#login as a user
cognitoClient = awsSession.client('cognito-idp')
auth = AWSSRP(username="user, password="pwd", pool_id="userPoolId", client_id="myAppClientId",  client=cognitoClient)
tokens = auth.authenticate_user()  

session = requests.Session()
#I now use the awsSession assuming it is populated with the login user and its access tokens 
session.auth = IAMAuth(awsSession)
response = session.get( "https://ol5oyvr5rg.execute-api.eu-west-1.amazonaws.com/test/movies")
>> KO with a 403 error

[amancevice]

I think you are confusing Cognito with IAM. This solution is for using IAM (your AWS access key+secret) to authenticate with API Gateway, nothing to do with Cognito.

[juanlucky]

thanks @amancevice for your reply. Sorry for the misunderstanding... Any Idea how to call with boto3 an AWS API within a Cognito user logged in ? Really searching and trying four hours with no success !

[abhijeetbote]

how to get a api keys from a specific api gateway

[nikonyrh]

Amazon's own signing example starts with this note:

If you are using one of the AWS SDKs (including the SDK for C++, SDK for Go, SDK for Java, AWS SDK for JavaScript, AWS SDK for .NET, SDK for PHP, SDK for Python (Boto3), or SDK for Ruby), you do not have to manually perform the steps of deriving a signing key and adding authentication information to a request. The SDKs perform this work for you. You need to manually sign requests only if you are directly making HTTP or HTTPS requests.

I'd like to see an official example regarding how to use Boto3 to do an API Gateway HTTP GET and POST requests. Is one available somewhere? So far I've just found 3rd party libraries and one promising looking gist.