python-sdk icon indicating copy to clipboard operation
python-sdk copied to clipboard

Python SDK error handling fails to report underlying OpenFGA exception

Open aryehklein opened this issue 9 months ago • 6 comments

Checklist

  • [x] I have looked into the README and have not found a suitable solution or answer.
  • [x] I have looked into the documentation and have not found a suitable solution or answer.
  • [x] I have searched the issues and have not found a suitable solution or answer.
  • [x] I have upgraded to the latest version of OpenFGA and the issue still persists.
  • [x] I have searched the Slack community and have not found a suitable solution or answer.
  • [x] I agree to the terms within the OpenFGA Code of Conduct.

Description

Using the sample code:

class OpenFgaFacade:
    """
    Facade class for interacting with OpenFGA authorization service.
    """

    def __init__(self, api_url=None):
        """
        Initialize the OpenFGA facade.

        Args:
            api_url: The OpenFGA API URL. If not provided, it will be read from FGA_API_URL environment variable.
        """
        self.api_url = api_url or os.environ.get("FGA_API_URL", "http://localhost:8085/")
        # Defer client creation until needed
        self.client = None

    async def initialize(self):
        """
        Asynchronously initialize the OpenFGA client with store and authorization model.
        This method must be called before using the facade.
        """
        configuration = openfga_sdk.ClientConfiguration(
            api_url=self.api_url,
            store_id=os.environ.get("FGA_STORE_ID"),
            authorization_model_id=os.environ.get("FGA_AUTHORIZATION_MODEL_ID"),
        )
        # Defer client creation until needed
        self.client = AsyncOpenFgaClient(configuration)
        await self.client.read_authorization_models()

    async def write_tuple(self):
        """
        Write a tuple to the OpenFGA store.
        
        This is a sample method that writes a test tuple.
        In production, this would be replaced with actual authorization logic.
        """

        body = ClientWriteRequest(
            writes=[
                ClientTuple(
                    user="user:anne",
                    relation="viewer",
                    object="document:Z",
                ),
            ],
        )
        response = await self.client.write(body)

I am getting an error:

AttributeError: 'ClientResponse' object has no attribute 'data'

I expect an error to be thrown here as my OpenFGA model doesnt have a document object, but the actual error thrown isnt an OpenFGA one. The error seems to be coming from https://github.com/openfga/python-sdk/blame/172cd2671d6d54c1ed46f86eb1c00229c3704f60/openfga_sdk/exceptions.py#L126. It looks like the aiohttp clientresponse doesnt have a data field, hence the error.

When I use the cli instead of the python SDK I get the expected error message

Expectation

The python SDK should use the correct field from the aiohttp clientresponse so that the underlying OpenFGA error is returned to the caller.

Reproduction

  1. Create a python client
  2. Call write using a tuple that isnt defined in the relationship model

OpenFGA SDK version

0.9.2

OpenFGA version

0.9.2

SDK Configuration

See vode above

Logs

No response

References

No response

aryehklein avatar Mar 30 '25 19:03 aryehklein

Keep in mind that if you pass an empty list to either the writes or deletes operations, the same exception occurs. So, this might also need to be handled.

body = ClientWriteRequest(writes=[], deletes=[])

demetere avatar Apr 06 '25 14:04 demetere

I just encountered a similar issue again. I tried reinserting the same data, and based on my understanding, it should have raised a TupleAlreadyExists exception (or something similar). Instead, it threw the exception that was mentioned earlier, which seems like a general error handling issue.

I noticed that http_resp.content is an Asyncio.StreamReader, but the handler is a synchronous function. That might explain why there's no http_resp.data object and why the exception is being thrown.

I'm happy to help out—whether that's through discussion or contributing code—if we can agree on how this case should be handled.

@ryanpq

demetere avatar Apr 06 '25 20:04 demetere

Hi @demetere, thanks for offering to help! If you have a proposed solution, can you give some description and explanation here. But if you'd like to first discuss the solution further with the team or need guidance, we can have that conversation in the CNCF #openfga Slack thread that you started. Thank you!

dyeam0 avatar Apr 09 '25 20:04 dyeam0

Hello @dyeam0, I've identified the issue in the request function located in the rest.py file, lines 412–467:

async def handle_response_exception(
        self, response: RESTResponse | aiohttp.ClientResponse
    ) -> None:
        """
        Raises exceptions if response status indicates an error.

        :param response: The response to check.
        :raises ValidationException: If status is 400.
        :raises UnauthorizedException: If status is 401.
        :raises ForbiddenException: If status is 403.
        :raises NotFoundException: If status is 404.
        :raises RateLimitExceededError: If status is 429.
        :raises ServiceException: If status is 5xx.
        :raises ApiException: For other non-2xx statuses.
        """
        if 200 <= response.status <= 299:
            return

        match response.status:
            case 400:
                raise ValidationException(http_resp=response)
            case 401:
                raise UnauthorizedException(http_resp=response)
            case 403:
                raise ForbiddenException(http_resp=response)
            case 404:
                raise NotFoundException(http_resp=response)
            case 429:
                raise RateLimitExceededError(http_resp=response)
            case _ if 500 <= response.status <= 599:
                raise ServiceException(http_resp=response)
            case _:
                raise ApiException(http_resp=response)

async def request(
    self,
    method: str,
    url: str,
    query_params: dict | None = None,
    headers: dict | None = None,
    body: Any | None = None,
    post_params: list[tuple[str, Any]] | None = None,
    _preload_content: bool = True,
    _request_timeout: float | None = None,
) -> RESTResponse | aiohttp.ClientResponse:
    """
    Executes a request and returns the response object.

    :param method: The HTTP method.
    :param url: The endpoint URL.
    :param query_params: Query parameters to be appended to the URL.
    :param headers: Optional request headers.
    :param body: A request body for JSON or other content types.
    :param post_params: Form/multipart parameters for the request.
    :param _preload_content: If True, the response body is read immediately.
    :param _request_timeout: An optional request timeout in seconds.
    :return: A RESTResponse if _preload_content is True, otherwise an aiohttp.ClientResponse.
    """

    # Build our request payload
    args = await self.build_request(
        method,
        url,
        query_params=query_params,
        headers=headers,
        body=body,
        post_params=post_params,
        _preload_content=_preload_content,
        _request_timeout=_request_timeout,
    )

    # Send request, collect response handler
    wrapped_response: RESTResponse | None = None
    raw_response: aiohttp.ClientResponse = await self.pool_manager.request(**args)

    # If we want to preload the response, read it
    if _preload_content:
        # Collect response data
        data = await raw_response.read()

        # Transform response JSON data into RESTResponse object
        wrapped_response = RESTResponse(raw_response, data)

        # Log the response body
        logger.debug("response body: %s", data.decode("utf-8"))

    # Handle any errors that may have occurred
    await self.handle_response_exception(raw_response)

    return wrapped_response or raw_response

As seen above, raw_response is an instance of aiohttp.ClientResponse, which requires await to read its content. The team is clearly aware of this because await raw_response.read() is already used on line 456.

However, the issue is that we're passing raw_response to handle_response_exception() without ensuring its content is read and available for any downstream use. That function checks the response status and raises an appropriate exception — but the constructors of those exception classes are synchronous and attempt to access raw_response.data, which doesn't exist. Since the content needs to be awaited, it cannot be accessed directly in a synchronous constructor.

To fix this, we have a couple of options:

  1. Read the content earlier in the request function and attach it to the raw_response, for example:

    raw_response.data = await raw_response.read()
    

    Then, it can be safely used later in the exception constructors.

  2. Alternatively, pass the already-read content (data) as an explicit argument to handle_response_exception.

Since both request and handle_response_exception are async, we have the opportunity to await the response read in either place. Also, we already construct wrapped_response = RESTResponse(raw_response, data), which confirms that the async reading is known and handled elsewhere correctly.

I’m happy to help implement the fix. Let me know if you'd prefer one approach over the other, or if there's another direction you’d like to explore.

demetere avatar Apr 10 '25 06:04 demetere

quck & dirty i added

        if isinstance(response, aiohttp.ClientResponse):
            response.data  = await response.read()
    async def handle_response_exception(
        self, response: RESTResponse | aiohttp.ClientResponse
    ) -> None:
        """
        Raises exceptions if response status indicates an error.

        :param response: The response to check.
        :raises ValidationException: If status is 400.
        :raises UnauthorizedException: If status is 401.
        :raises ForbiddenException: If status is 403.
        :raises NotFoundException: If status is 404.
        :raises RateLimitExceededError: If status is 429.
        :raises ServiceException: If status is 5xx.
        :raises ApiException: For other non-2xx statuses.
        """
        if 200 <= response.status <= 299:
            return
        
        if isinstance(response, aiohttp.ClientResponse):
            response.data  = await response.read()

        match response.status:
            case 400:
                raise ValidationException(http_resp=response)
            case 401:
                raise UnauthorizedException(http_resp=response)
            case 403:
                raise ForbiddenException(http_resp=response)
            case 404:
                raise NotFoundException(http_resp=response)
            case 429:
                raise RateLimitExceededError(http_resp=response)
            case _ if 500 <= response.status <= 599:
                raise ServiceException(http_resp=response)
            case _:
                raise ApiException(http_resp=response)

extreme4all avatar Apr 10 '25 19:04 extreme4all

@extreme4all I think there can be a lot of different types of solutions, quick and dirty ones too. Let's see what the team has to say on that regard.

demetere avatar Apr 13 '25 09:04 demetere

Hi @demetere. The team discussed your proposed fix during our planning meeting. From a quick overview, it looks good. We would like our resident Python SME to take one more deep look and give feedback before proceeding. Thanks for your patience! He should be able to review this week.

dyeam0 avatar May 05 '25 19:05 dyeam0

👋 I think the solution @extreme4all proposed makes perfect sense. If you'd like to put together a Pull Request we'd be happy to move forward with implementing your fix. Thanks!

evansims avatar May 07 '25 19:05 evansims

@extreme4all Should I proceed and create PR or do you have everything ready already?

demetere avatar May 08 '25 08:05 demetere

Hi @extreme4all, just checking in to see if you want to take on the PR.

dyeam0 avatar May 21 '25 15:05 dyeam0

This would be very helpful since handling specific errors produced by the OpenFGA client is difficult or requires workarounds.

cbernard-rm25 avatar May 21 '25 17:05 cbernard-rm25

Since this is the quite problematic issue, I will start working on it as soon as I can. Will try to finalize PRs by end of this week

demetere avatar May 24 '25 21:05 demetere

Since this is the quite problematic issue, I will start working on it as soon as I can. Will try to finalize PRs by end of this week

Thanks @demetere!

dyeam0 avatar May 27 '25 13:05 dyeam0

Any movement on this issue @demetere? Definitely looking to have proper error handling in a python app.

cbernard-rm25 avatar Jun 05 '25 20:06 cbernard-rm25

For anyone wondering this is resolved by https://github.com/openfga/python-sdk/pull/197 as of 2 days ago and can be closed.

cgensheimer avatar Jun 29 '25 01:06 cgensheimer

Yes, that's correct. @cmbernard333 Thanks for getting this over the finish line! @demetere and @extreme4all Thanks for the initial investigations. @evansims for the review. And @aryehklein thanks for the opening the issue.

Thank you all for your valuable time to the project!

dyeam0 avatar Jun 29 '25 13:06 dyeam0

@dyeam0 just curious, what is the upcoming timeline or release schedule? I noticed it's been a couple months since the last release and I would like to be able to pin my project to a version with this fix instead of applying my current patch.

cgensheimer avatar Jun 30 '25 17:06 cgensheimer

Hi @cgensheimer, it might take a few weeks before we cut the release, but we will try to get it out sooner.

dyeam0 avatar Jul 01 '25 21:07 dyeam0

Thanks all, just setting up OpenFGA in my project and ran into this issue, quite a significant one! Glad there's a release coming soon.

josh-mission avatar Jul 03 '25 13:07 josh-mission

Hi @cgensheimer and @josh-mission. Release v0.9.5 is available with the fix!

dyeam0 avatar Jul 10 '25 07:07 dyeam0