django-elasticsearch-dsl-drf icon indicating copy to clipboard operation
django-elasticsearch-dsl-drf copied to clipboard

Implement aggregations backend

Open barseghyanartur opened this issue 6 years ago • 4 comments

Implement aggregations backend (rethink faceted search backend).

barseghyanartur avatar Sep 19 '17 13:09 barseghyanartur

I wrote this backend that provides functionality similar to elasticsearch-dsl's Faceted Search.

class FacetedFilterAggregationFilterBackend(FilteringFilterBackend, FacetedSearchFilterBackend):
    """ This filter supports facets and filtering in a way similar to elasticsearch-dsl's FacetedSearch class.
    It combines the functionality of FilteringFilterBackend and FacetedSearchFilterBackend to take filters into
    account when creating facets.

    When creating a facet, filters for other faceted fields except those for the current facet are applied. Filters
    for faceted fields are then applied as post_filters. Filters on non-faceted fields are applied as normal filters.
    
    To do this, we need to store both the facets and the filters. Because the superclasses use @classmethod,
    we can't simply store state on this object. Instead, we stash helper variables on the queryset object.
    """
    def filter_queryset(self, request, queryset, view):
        # the fact that apply_filter is a classmethod means we can't store state on self,
        # so we hitch it onto queryset
        queryset._facets = self.construct_facets(request, view)
        queryset._filters = defaultdict(list)

        # apply filters
        queryset = FilteringFilterBackend.filter_queryset(self, request, queryset, view)

        # apply aggregations
        return self.aggregate(request, queryset, view)

    @classmethod
    def apply_filter(cls, queryset, options=None, args=None, kwargs=None):
        if args is None:
            args = []
        if kwargs is None:
            kwargs = {}

        facets = queryset._facets
        filters = queryset._filters

        # if this field is faceted, then apply it as a post-filter
        if options['field'] in facets:
            queryset = queryset.post_filter(*args, **kwargs)
        else:
            queryset = queryset.filter(*args, **kwargs)

        filters[options['field']].append(Q(*args, **kwargs))

        # ensure the new queryset object retains the facets
        queryset._facets = facets
        queryset._filters = filters
        return queryset

    def aggregate(self, request, queryset, view):
        facets = queryset._facets
        filters = queryset._filters

        for field, facet in facets.items():
            agg = facet['facet'].get_aggregation()

            if facet['global']:
                queryset.aggs.bucket(
                    '_filter_' + field,
                    'global'
                ).bucket(field, agg)
                continue

            # apply filters for other facets except this one
            agg_filter = Q('match_all')
            for f, _filter in filters.items():
                if field not in facets or field == f:
                    continue
                # combine with or
                q = _filter[0]
                for x in _filter[1:]:
                    q = q | x
                agg_filter &= q

            queryset.aggs.bucket(
                '_filter_' + field,
                'filter',
                filter=agg_filter
            ).bucket(field, agg)

        return queryset

The use of @classmethod in django-elasticsearch-dsl-drf makes implementing this a bit awkward, but I worked around that by stashing state on the queryset object. I'm not sure what value the classmethods add, but it did make re-using the existing classes more difficult.

longhotsummer avatar Jan 07 '22 13:01 longhotsummer

@longhotsummer:

Great! Do you mind submitting a proper PR, docs and tests? :)

barseghyanartur avatar Jan 07 '22 22:01 barseghyanartur

Sure. Do you mind if I remove the classmethods, or do they need to be kept?

longhotsummer avatar Jan 08 '22 08:01 longhotsummer

Please, keep the classmethods.

barseghyanartur avatar Jan 08 '22 08:01 barseghyanartur