django-elasticsearch-dsl icon indicating copy to clipboard operation
django-elasticsearch-dsl copied to clipboard

Support bulk operations

Open kirillpechurin opened this issue 2 years ago • 5 comments

In order to support the actual state of documents in indexes, I propose a solution to support bulk operations via manager implementation

A manager is described for working with the following bulk operations:

  • bulk_create
  • bulk_update
  • update
  • delete

Manager can be connected to the model by calling as_manager() in the objects attribute.

The manager itself is implemented in a similar way to the signal processor. The manager is described using the Mixin approach, allowing you to connect additional functionality to existing managers. The mixin contains functions that call registry. The process of working with registry for massoperations boils down to the following: entities are received by IDs, these entities are normalized and sent to registry

To update related entities, the update occurs with an additional many flag. The many flag is introduced to get target entities from related from a queryset or a list.

The following has been adjusted for registry:

  • Introduction of the many flag for update functions on related entities. With the many flag, the document function get_instances_from_many_related is called. The get_instances_from_many_related function receives the entity class and the selection of related entities itself as input
  • Getting the entity class according to the passed argument. Three types of support have been introduced:
    • Model
    • QuerySet
    • list

A check has been introduced into the existing signal processor for the correct processing of the following deletion signals:

  • handle_pre_delete
  • handle_delete

For these functions, a check of the key parameter origin has been introduced in order to reduce the operation of the signal only when interacting with a specific entity. To process the signal by m2m changes, the transmission of origin as a key parameter has been introduced. The origin parameter is passed as a call to the class of the model from which the signal was received

Added in the get_value_from_instance (DEDField) function to work with

  1. QuerySet as ignored entities for a specific entity (Model-QuerySet)
  2. QuerySet as ignored entities for ignored entities (QuerySet-QuerySet)

Completed and updated testing

Updated readme. Add Support bulk operations.

kirillpechurin avatar May 14 '23 16:05 kirillpechurin