algoliasearch-django icon indicating copy to clipboard operation
algoliasearch-django copied to clipboard

Mvdb/support record duplication

Open francois-travais opened this issue 2 years ago • 0 comments

Q A
Bug fix? no
New feature? yes
BC breaks? no
Related Issue Fix #271
Need Doc update yes

Describe your change

This is just an updated version of #274 by @m-vdb

Quoting @m-vdb https://github.com/algolia/algoliasearch-django/issues/271#issuecomment-437401895

hey @clemfromspace I finally got the time to finish my work on the PR. I have coded integration tests to validate that everything works. IMO, what's missing:

  • validate my API: is it clear enough for other developers (method names, comments, etc...)?
  • documentation in the README: I can do it once you validate the API
  • real life tests for the AlgoliaIndexBatch: I haven't tested it yet with real data. In my case, I have records that will "duplicate" to thousands of records. It's next on my list.
  • compat code: I had to keep compat code for Django 1.7, I don't know if you'll agree with that. Ideally, it'd be better if algoliasearch-django dropped support for 1.7, but that's up to you.

What problem is this fixing?

Quoting @m-vdb https://github.com/algolia/algoliasearch-django/issues/271#issue-363493890

Documentation: https://www.algolia.com/doc/guides/ranking/distinct/#distinct-for-de-duplication

I am hitting the 10kb limit for records and I need to be able to support de-duplication. Here is my use case:

  • I have one model that contains a list of locations
  • for some models, this list contains a lot of locations (some thousands)
  • when indexing my model, I want to push 1 record for each location in the list: this will tremendously decrease record size while at the same time enhance the search experience

francois-travais avatar May 12 '22 14:05 francois-travais