upgrade recipes: missing reindexing of users and other indices

[ppanero]

Package version (if known): v10

Describe the bug

When upgrading the instance, sometimes we need to wipe out indices. Then we recreate. However, we only recreate some (records, communities, vocabularies) but not users and others.

Steps to Reproduce

1. invenio index destroy --yes-i-know
2. invenio index init
3. invenio rdm-records rebuild-index
4. invenio communities rebuild-index

Expected behavior

All indices should be rebuilt

Additional context

Potential solution:

• index init should allow to create only one (or several, but not all) indices. That way we could make use of the same functionality in index destroy, which is already available.
• reubild-indices should accept a name and rebuild only thata
  • encapsulate in invenio-cli
  • vocabularies should be in invenio-vocabularies not in rdm-records, alternatively include also communities. The command should be made parametrizable so we can use the target destroy-recreate-reindex.

---

At the moment we need to run this extra lines in a invenio-cli pyshell

from invenio_access.permissions import system_identity
from invenio_communities.proxies import current_communities
from invenio_records_resources.proxies import current_service_registry
from invenio_requests.proxies import current_events_service, current_requests_service

# reindex users
users_service = current_service_registry.get("users")
users_service.rebuild_index(system_identity)

# reindex groups
groups_service = current_service_registry.get("groups")
groups_service.rebuild_index(system_identity)

# reindex members and archived invitations
members_service = current_communities.service.members
members_service.rebuild_index(system_identity)

# reindex requests
for req_meta in current_requests_service.record_cls.model_cls.query.all():
    request = current_requests_service.record_cls(req_meta.data, model=req_meta)
    if not request.is_deleted:
        current_requests_service.indexer.index(request)

# reindex requests events
for event_meta in current_events_service.record_cls.model_cls.query.all():
    event = current_events_service.record_cls(event_meta.data, model=event_meta)
    current_events_service.indexer.index(event)
        

[github-actions[bot]]

This issue was automatically marked as stale.

[ppanero]

@zzacharo @kpsherva this is kinda important mostly for future releases (v11?)

[github-actions[bot]]

This issue was automatically marked as stale.

[ntarocco]

@ppanero is this still valid for v12?

[ppanero]

@ntarocco I haven't followed in depth the changes in indexer, so for context:

IIRC, when doing the migration v9 to v10 we had to remove indices because there were changes in the mappings. Then there is no complete re-index command (there is one in rdm-records, but it does not take into account subservices and the like). On the other hand this is only needed if there are changes to the mapping. Otherwise, if the records/users/etc. change but the mapping does not a simple re-index would work and there would be no need to remove the indices. In this latest case, the code above is only needed if we have to reindex users, groups, members, requrests, or events.

Idea: Maybe the work in access requests requires this since it adds some properties to the users (and I guess requests)

[github-actions[bot]]

This issue was automatically marked as stale.

[utnapischtim]

documented in https://github.com/inveniosoftware/docs-invenio-rdm/pull/607 and therefore fixed. the line invenio rdm rebuild-all-indices does the job