Adding OrderingSchema for ordering QuerySets

[aryan-curiel]

Problem

The FilterSchema definition is a simple but effective approach to centralize logic around filtering QuerySets. A similar approach could be followed for ordering. Current for ordering and filtering in the same handler we would the following:

@api.get("/books")
def list_books(request, filters: BookFilterSchema = Query(...), order_by: list[str] | None = None):
    books = Book.objects.order_by(**order_by)
    books = filters.filter(books)
    return books

But what if we want to limit the API to allow ordering by just some of the fields. Or if we want to customize ordering based on custom fields and logic. What if we want to have a similar approach for other data sources, for example ElasticSearch.

Proposal

This PR propose to include a helper schema class, similar to FilteringSchema, but for ordering. It';s a simple schema class, with only one field: order_by, that accepts a list of string. The allowed fields can be specified through the Config inner class, and a Pydantic validator will check that the provided query values are part of the allowed fields. The schema then will provide a .sort() method (similar to .filter()) that we can use to pass the query set, and expect it ordered as a returned value. The values can be provided using django standard behavior for descending order.

Example

Using it with out-of-the-box definition, allowing all fields from the model.

from ninja import OrderingSchema

@api.get("/books")
def list_books(request, filters: BookFilterSchema = Query(...), ordering: OrderingSchema = Query(...)):
    books = Book.objects.all()
    books = filters.filter(books)
    books = ordering.sort(books)
    return books

Using it with custom definition of allowed fields

from ninja import OrderingSchema

class BookOrderingSchema(OrderingSchema):
    class Config(OrderingSchema.Config):
        allowed_fields = ['name', 'created_at']  # Leaving out `author` field

@api.get("/books")
def list_books(request, filters: BookFilterSchema = Query(...), ordering: BookOrderingSchema = Query(...)):
    books = Book.objects.all()
    books = filters.filter(books)
    books = ordering.sort(books)
    return books

Other ideas not followed

I also considered to have a default value field in the config, but decided to go with field default definition on custom schema level Another consideration was to create a class method factory in the OrderingSchema, so it can be define inline, but I wasn't sure if it would be used:

@api.get("/books")
def list_books(
    request,
    filters: BookFilterSchema = Query(...),
    ordering: OrderingSchema.with_allowed_fields('name', 'created_at') = Query(...)
):
    ...

Notes

I didn't add field validation with Model definition, to keep the practices followed in the FilterSchema definition. Also, I think is a good idea to keep this helpers class as simple as possible, and give room to personalization for more complex scenarios. However, let me know if validating allowed_fields with Model fields is something we would like to have, and I can update the PR.

This was really useful in a personal project, were we needed to provide different ordering behaviors for a QuerySet and for an OpenSearch query. We had to do similar personalizations for pagination and filtering.

[shijl0925]

How about OrderingSchema.sort support list type sort?

class OrderingSchema(OrderingBaseSchema):
    def sort(self, items: Union[QuerySet, List]) -> Union[QuerySet, List]:
        if self.order_by:
            if isinstance(items, QuerySet):  # type:ignore
                return items.order_by(*self.order_by)
            elif isinstance(items, list) and items:
                def multisort(xs: List, specs: List[Tuple[str, bool]]) -> List:
                    getter = itemgetter if isinstance(xs[0], dict) else attrgetter
                    for key, reverse in specs:
                        xs.sort(key=getter(key), reverse=reverse)
                    return xs

                return multisort(
                    items,
                    [
                        (o[int(o.startswith("-")):], o.startswith("-"))
                        for o in self.order_by
                    ],
                )

        return items

[aryan-curiel]

How about OrderingSchema.sort support list type sort?

Adding support for list type is definitely a good idea. However, not sure adding it to this sort method is the most convenient approach. For example, if we want to support other collections, then we would need to add another type hint, and another elif. Any extension for supporting other collections or types, might be more suitable as a subclass of the base OrderingBaseSchema. However, we might need to make some small arrangements in the base class for a better definition.

[shijl0925]

May I ask if there is any progress on this OrderingSchema? I found same failure in these pull request actions. But I can not see these failed logs. What happened about this pull request?

[aryan-curiel]

May I ask if there is any progress on this OrderingSchema?

Honestly, I'm not sure. I haven't received any review, and find no sense in the failed tests related to schemas in the pipeline, so I haven't been able to fix it. If you need it with some level of urgency, feel free to bring the added code to your code base.

[crbunney]

For consideration on this topic, this library also exists and provides ordering support to django-ninja: https://eadwincode.github.io/django-ninja-extra/tutorial/ordering/ I don't know how the functionality & implementation compare to the PR, but thought it might be useful for anyone looking at this to be aware of it

[aryan-curiel]

@crbunney Wow, thanks! Didn't know about it. I'm also doing a library for supporting filtering, ordering and pagination for Ninja and FastAPI with multiple data sources. I'm not a big fan of using decorators for this :( It feels like it;s obfuscating the logic. In any case, it's amazing that we have already something like ninja extras!

[lthon-sha]

Hi @aryan-curiel, just wanted to say a huge thank you for this PR! The OrderingSchema implementation is incredibly clean, elegant. It's exactly the solution I was looking for. Amazing work!

[lthon-sha]

I noticed that when order_by is empty ([]), calling queryset.order_by() clears the model's default ordering from Meta.ordering.

Maybe we should only call order_by() when there are actual fields to sort by?

[aryan-curiel]

I noticed that when order_by is empty ([]), calling queryset.order_by() clears the model's default ordering from Meta.ordering.

Maybe we should only call order_by() when there are actual fields to sort by?

@lthon-sha Thanks for that amazing feedback! I will update the PR according to your recommendation and add the needed tests

[aryan-curiel]

@vitalik I would love a feedback from any of the more experienced contributors. It has been almost a year since the PR was opened, and I would love to move forward with it if it's something that you all consider it might bring some value.

[vitalik]

Hi @aryan-curiel

Sorry for dragging this for this long - but we finally getting rid of "Config" child class in favour of Meta (as it will be "broken" in future pedantic releases

Could you rework Config -> Meta like https://github.com/vitalik/django-ninja/blob/52d510ac9d50794d1935086127132d917da670c9/ninja/filter_schema.py#L44-L47

[aryan-curiel]

Hi @vitalik

That's awesome! PR updated with the requested changes. Thanks for the feedback.

Let me know if anything else is needed.