django-elasticsearch-dsl-drf icon indicating copy to clipboard operation
django-elasticsearch-dsl-drf copied to clipboard

_type parameter is deprecated in the MoreLikeThis query

Open Prakash2403 opened this issue 3 years ago • 1 comments

Describe the bug _type parameter is deprecated in MoreLikeThis query in the latest version of ES. I get the following deprecation warning after I query at more_like_this endpoint

{
  "type": "deprecation",
  "timestamp": "2021-08-22T18:00:17,324Z",
  "level": "DEPRECATION",
  "component": "o.e.d.i.q.MoreLikeThisQueryBuilder",
  "cluster.name": "docker-cluster",
  "node.name": "2fd6f0e3e02f",
  "message": "[types removal] Types are deprecated in [more_like_this] queries. The type should no longer be specified in the [like] and [unlike] sections.",
  "cluster.uuid": "iDcaco5NROutaFg3svGMrw",
  "node.id": "hdeF_22aSxWUswmsMhYC9A"
}

Environment

Python environment:

  1. pip list
Package                       Version
----------------------------- ---------
amqp                          2.6.1
asgiref                       3.4.1
attrs                         21.2.0
billiard                      3.6.4.0
boto3                         1.17.93
botocore                      1.20.112
celery                        4.4.6
certifi                       2021.5.30
cffi                          1.14.6
chardet                       4.0.0
click                         8.0.1
click-plugins                 1.1.1
cligj                         0.7.2
coreapi                       2.3.3
coreschema                    0.0.4
cryptography                  3.4.7
defusedxml                    0.7.1
diff-match-patch              20200713
Django                        3.2.4
django-celery-results         2.0.1
django-cors-headers           3.7.0
django-elasticsearch-dsl      7.2.0
django-elasticsearch-dsl-drf  0.22.1
django-import-export          2.5.0
django-nine                   0.2.4
django-templated-mail         1.1.1
djangorestframework           3.12.4
djangorestframework-simplejwt 4.8.0
djoser                        2.1.0
elasticsearch                 7.14.0
elasticsearch-dsl             7.4.0
et-xmlfile                    1.1.0
Fiona                         1.8.20
flower                        0.9.7
future                        0.18.2
geopandas                     0.9.0
gunicorn                      20.1.0
humanize                      3.11.0
idna                          2.10
itypes                        1.2.0
Jinja2                        3.0.1
jmespath                      0.10.0
kombu                         4.6.11
lxml                          4.6.3
MarkupPy                      1.14
MarkupSafe                    2.0.1
munch                         2.5.0
numpy                         1.21.2
oauthlib                      3.1.1
odfpy                         1.4.1
pandas                        1.3.2
pathvalidate                  2.4.1
Pillow                        8.2.0
pip                           21.2.4
prometheus-client             0.8.0
psycopg2-binary               2.8.6
pycparser                     2.20
PyJWT                         2.1.0
pyproj                        3.1.0
pytesseract                   0.3.7
python-dateutil               2.8.2
python3-openid                3.2.0
pytz                          2021.1
PyYAML                        5.4.1
redis                         3.5.3
requests                      2.25.1
requests-oauthlib             1.3.0
s3transfer                    0.4.2
setuptools                    57.4.0
Shapely                       1.7.1
six                           1.16.0
social-auth-app-django        4.0.0
social-auth-core              4.1.0
sqlparse                      0.4.1
tablib                        3.0.0
tornado                       6.1
unicodecsv                    0.14.1
uritemplate                   3.0.1
urllib3                       1.26.6
vine                          1.3.0
wheel                         0.37.0
xlwt                          1.3.0
  1. python --version 3.9.6

Which version of Elasticsearch are you using? 7.12.1

To Reproduce Steps to reproduce the behavior:

  1. Create a ViewSet and inherit MoreLikeThisMixin
  2. Run a query against more_like_this endpoint
  3. Monitor ES server logs

Expected behavior Deprecation warnings shouldn't appear

@barseghyanartur I've fixed it on my fork. If you feel that this should be merged here, kindly let me know. I'll make a PR

Prakash2403 avatar Aug 25 '21 06:08 Prakash2403

Leaving this note here for anyone trying to get more_like_this working with Elasticsearch 7 or higher. I could not get it to work in the AWS version of Elasticsearch. The fix proposed by @Prakash2403 works, my solution was to use a local version of the class:-

from rest_framework.decorators import action
from elasticsearch_dsl.query import MoreLikeThis
from rest_framework.response import Response

import copy

"""
The current version of django-elasticsearch-dsl-drf will not work with Elasticsearch 7 or higher more_like_this
reason being that the _type parameter is deprecated. So we copy their code and fix it locally

"""


class MoreLikeThisMixin(object):
    """More-like-this mixin."""

    @action(detail=True)
    def more_like_this(self, request, pk=None, id=None):
        """More-like-this functionality detail view.

        :param request:
        :return:
        """
        if 'view' in request.parser_context:
            view = request.parser_context['view']
            kwargs = copy.copy(getattr(view, 'more_like_this_options', {}))
            id_ = pk if pk else id

            queryset = self.filter_queryset(self.get_queryset())
            fields = kwargs.pop('fields', [])

            if fields:
                queryset = queryset.query(
                    MoreLikeThis(
                        fields=fields,
                        like={
                            '_id': "{}".format(id_),
                            '_index': "{}".format(self.index)
                        },
                        **kwargs
                    )
                ).sort('_score')
            else:
                queryset = queryset.query(
                    MoreLikeThis(
                        like={
                            '_id': "{}".format(id_),
                            '_index': "{}".format(self.index)
                        },
                        **kwargs
                    )
                ).sort('_score')

            # Standard list-view implementation
            page = self.paginate_queryset(queryset)
            if page is not None:
                serializer = self.get_serializer(page, many=True)
                return self.get_paginated_response(serializer.data)

            serializer = self.get_serializer(queryset, many=True)
            return Response(serializer.data)

citizenfish avatar Jun 02 '23 08:06 citizenfish