Update index search preference

[ps48]

Description

Update index search preference to _primary

Issues Resolved

https://github.com/opensearch-project/observability/issues/1562

Check List

• [ ] New functionality includes testing.
  • [ ] All tests pass, including unit test, integration test and doctest
• [ ] New functionality has been documented.
  • [ ] New functionality has javadoc added
  • [ ] New functionality has user manual doc added
• [x] Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license. For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[derek-ho]

@ps48 this should be backported to 2.x as well, right?

[derek-ho]

Might help with #1597 https://github.com/opensearch-project/observability/issues/1597