Request body JSON serialization not working with httpx

[eidorb]

Description The cookbook says if fmt returns a dict then JSON serialization is performed for you. This doesn't work when used with httpx.

How to Reproduce Adapt the cookbook example to httpx (see snippet below).

Expected behavior Should work like the original aiohttp example. I think the formatter is being applied: https://github.com/pmateusz/meatie/blob/cbb1134a389cdc5d80d545cb0c3986b17be8b1ef/src/meatie/internal/template/request.py#L124-L129

I'll admit I got lost after this point as to how HTTP library-specify marshalling is done.

Code snippet Adapting the cookbook example to httpx:

from typing import Annotated, Any

from httpx import Client as HttpxClient
from meatie import api_ref, endpoint
from meatie_httpx import Client
from pydantic import BaseModel, Field


class Todo(BaseModel):
    user_id: int = Field(alias="userId")
    id: int
    title: str
    completed: bool


class Params:
    @staticmethod
    def todo(value: Todo) -> dict[str, Any]:
        return value.model_dump(by_alias=True)


class JsonPlaceholderClient(Client):
    def __init__(self) -> None:
        super().__init__(HttpxClient(base_url="https://jsonplaceholder.typicode.com"))

    @endpoint("/todos")
    def post_todo(
        self, todo: Annotated[Todo, api_ref("body", fmt=Params.todo)]
    ) -> Todo: ...


JsonPlaceholderClient().post_todo(Todo(userId=123, id=456, title="abc", completed=True))

[pmateusz]

Hi @eidorb,

Thank you for the detailed bug description. It was straightforward to repro the issue and understand where the problem is.

The easiest way to address the problem is to return str or bytes from the fmt function whenever the underlying client is HTTPX. It requires no changes in the library.

I am posting an example below. I am using model_dump_json instead of model_dump.

from http import HTTPStatus
from typing import Annotated, Any

import pytest
from http_test import HTTPTestServer, Handler
from httpx import Client as HttpxClient
from pydantic import BaseModel, Field

from meatie import api_ref, endpoint
from meatie_httpx import Client

pydantic = pytest.importorskip("pydantic", minversion="2.0.0")


class Todo(BaseModel):
    user_id: int = Field(alias="userId")
    id: int
    title: str
    completed: bool


class Params:
    @staticmethod
    def todo(value: Todo) -> str:
        return value.model_dump_json(by_alias=True)


class JsonPlaceholderClient(Client):
    def __init__(self, base_url: str) -> None:
        super().__init__(HttpxClient(base_url=base_url))

    @endpoint("/todos")
    def post_todo(self, todo: Annotated[Todo, api_ref("body", fmt=Params.todo)]) -> Todo: ...


def test_post_request_body_with_fmt(http_server: HTTPTestServer) -> None:
    # GIVEN
    def handler(request: Handler) -> None:
        content_length = request.headers.get("Content-Length", "0")
        raw_body = request.rfile.read(int(content_length))
        try:
            data = Todo.model_validate_json(raw_body)
            request.send_json(HTTPStatus.OK, data.model_dump(mode="json", by_alias=True))
        except pydantic.ValidationError:
            request.send_response(HTTPStatus.BAD_REQUEST)
        request.end_headers()

    http_server.handler = handler

    # WHEN
    with JsonPlaceholderClient(http_server.base_url) as client:
        todo = client.post_todo(Todo(userId=123, id=456, title="abc", completed=True))

    # THEN
    assert todo.user_id == 123

I hope this solution unblocked you. In the meantime, I will keep this bug open to see if there is an elegant way to fix the issue.

[eidorb]

Thank you for looking into this.

Suppose an endpoint expects a simple JSON body:

@dataclass
class RequestBody:
    version: str

I attempted something "tricky" in order to construct a request body from a simple parameter:

@endpoint('/api')
def post_initialise_widget(self, version: Annotated[str, api_ref('body', fmt=lambda version: asdict(RequestBody(version)))]) -> None:
    ...

Instead of this:

@endpoint('/api')
def post_initialise_widget(self, body: RequestBody) -> None:
    ...

If fmt returns bytes, then I think I will also have to manually add a Content-Type header.

It is perhaps better to avoid trickiness, and simply be explicit about request body type.

[pmateusz]

I overlooked handling the Content-Type header in my previous response. Thank you for you remarks! 🙇

I've drafted a fix in PR https://github.com/pmateusz/meatie/pull/160 that ensures the Content-Type header is correctly set to application/json when the fmt method returns a dictionary. This applies to all HTTP clients: aiohttp, requests, and httpx.

I plan to skip developing a custom solution for setting the Content-Type when the fmt function returns a byte array or string. I see this as part of a more generic feature for setting HTTP request headers.

[eidorb]

Thanks Mat. This library is very fun to use!