algoliasearch-django icon indicating copy to clipboard operation
algoliasearch-django copied to clipboard

Published 20 hours ago • verified publisher icon algolia

Feature request: Support distinct option for de-duplication

Open m-vdb opened this issue 6 years ago • 7 comments

Description

Documentation: https://www.algolia.com/doc/guides/ranking/distinct/#distinct-for-de-duplication

I am hitting the 10kb limit for records and I need to be able to support de-duplication. Here is my use case:

  • I have one model that contains a list of locations
  • for some models, this list contains a lot of locations (some thousands)
  • when indexing my model, I want to push 1 record for each location in the list: this will tremendously decrease record size while at the same time enhance the search experience

Are you willing to accept PRs for this feature? I can draft a spec here and get on with it 💪 Cheers!

m-vdb avatar Sep 25 '18 09:09 m-vdb

no one?

m-vdb avatar Oct 15 '18 13:10 m-vdb

Hey @m-vdb,

Sorry for the late answer, we will definitely accept a PR for this case.

Had a quick look at your fork, the global logic looks good so far :)

Happy to review your PR when it's ready!

clemfromspace avatar Oct 17 '18 09:10 clemfromspace

thanks @clemfromspace! I'll be adding tests too. Will let you know when it's ready

m-vdb avatar Oct 17 '18 11:10 m-vdb

hey @clemfromspace I finally got the time to finish my work on the PR. I have coded integration tests to validate that everything works. IMO, what's missing:

  • validate my API: is it clear enough for other developers (method names, comments, etc...)?
  • documentation in the README: I can do it once you validate the API
  • real life tests for the AlgoliaIndexBatch: I haven't tested it yet with real data. In my case, I have records that will "duplicate" to thousands of records. It's next on my list.
  • compat code: I had to keep compat code for Django 1.7, I don't know if you'll agree with that. Ideally, it'd be better if algoliasearch-django dropped support for 1.7, but that's up to you.

You can see my new pull request on this repo here: https://github.com/algolia/algoliasearch-django/pull/274

m-vdb avatar Nov 09 '18 15:11 m-vdb

What's the status on this?

yhoiseth avatar Oct 23 '19 11:10 yhoiseth

I would like to use this functionality. Is there any plan to merge #274?

soulshake avatar Mar 01 '20 13:03 soulshake

I would also need this one. Is there any plan to do something about it?

francois-travais avatar May 12 '22 09:05 francois-travais