connectors icon indicating copy to clipboard operation
connectors copied to clipboard

Connector fails to sync to rollover index when alias is used as sync target

Open ramyacp397 opened this issue 5 months ago • 2 comments

When the connector is configured to sync to an index alias, the sync fails during writes (especially after rollover) because the connector-generated API key includes alias-level access only, not write access to the actual backing indices. This is not communicated to users, resulting in unexpected failures. Documentation and the UI do not warn about this permission limitation.

Workaround (Temporary Fix) Manually edit the role associated with the connector’s API key to include index-write privileges on the alias plus all backing indices—e.g.:

{ "names": ["events-alias", "events-alias*"], "privileges": ["index", "write"] } This allows the connector to write successfully through rollover events.

ramyacp397 avatar Aug 04 '25 11:08 ramyacp397

@maxesse , could you please provide error details

ramyacp397 avatar Aug 04 '25 11:08 ramyacp397

Sure, if you don't add permissions to the api key for the connector you'll get this error at the end of a sync run:

[FMWK][09:49:00][ERROR] Exception found for task Task-79 Traceback (most recent call last): File "/app/connectors/sync_job_runner.py", line 192, in execute await self.check_job() File "/app/connectors/sync_job_runner.py", line 504, in check_job raise ConnectorJobCanceledError connectors.sync_job_runner.ConnectorJobCanceledError During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/app/connectors/sync_job_runner.py", line 204, in execute await self._sync_done(sync_status=JobStatus.CANCELED) File "/app/connectors/sync_job_runner.py", line 330, in _sync_done ] = await self.connector.document_count() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/app/connectors/protocol/connectors.py", line 1079, in document_count await self.index.client.indices.refresh( File "/app/.venv/lib/python3.11/site-packages/elasticsearch/_async/client/indices.py", line 3874, in refresh return await self.perform_request( # type: ignore[return-value] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/app/.venv/lib/python3.11/site-packages/elasticsearch/_async/client/_base.py", line 423, in perform_request return await self._client.perform_request( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/app/.venv/lib/python3.11/site-packages/elasticsearch/_async/client/_base.py", line 271, in perform_request response = await self._perform_request( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/app/.venv/lib/python3.11/site-packages/elasticsearch/_async/client/_base.py", line 352, in _perform_request raise HTTP_EXCEPTIONS.get(meta.status, ApiError)( elasticsearch.AuthorizationException: AuthorizationException(403, "{'_shards': {'total': 4, 'successful': 1, 'failed': 3, 'failures': [{'shard': 0, 'index': 'sharepoint-mms-0001', 'status': 'FORBIDDEN', 'reason': {'type': 'security_exception', 'reason': 'action [indices:admin/refresh[s]] is unauthorized for API key id [PB4FZZgBCwWh5Nn2xlmQ] of user [807408337] on indices [sharepoint-mms-0001], this action is granted by the index privileges [maintenance,manage,all]'}}]}}") [FMWK][09:53:13][INFO] [Connector id: IlcFZZgBMc_X3JmlP3Vx, index name: sharepoint-mms, Sync job id: Px4MZZgBCwWh5Nn2t1kr] Executing full sync

maxesse avatar Aug 04 '25 11:08 maxesse