Set preference to _primary when searching control-center index

[gaobinlong]

*Issue *
#833

Description of changes:
This change is to set the query parameter preference to _primary when searching control-center index, because when indexing new documents in this index, we use RefreshPolicy.IMMEDIATE to force a refresh after the document is indexed which aims to make the new document visible immediately, but if segment replication is enabled in the control-center index, read-after-write is not guaranteed, so we need to search the primary shard only to make the document can be seen as soon as possible.

The search load on the control-center index is pretty low, it will not have some performance issue if we only search the primary shard.

CheckList:

• Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[codecov]

Codecov Report

Merging #911 (767fd6e) into main (c2ccf7e) will increase coverage by 0.23%.
The diff coverage is 100.00%.

@@             Coverage Diff              @@
##               main     #911      +/-   ##
============================================
+ Coverage     75.60%   75.84%   +0.23%     
- Complexity     2837     2853      +16     
============================================
  Files           364      364              
  Lines         16185    16186       +1     
  Branches       2329     2329              
============================================
+ Hits          12237    12276      +39     
+ Misses         2610     2570      -40     
- Partials       1338     1340       +2     

Files Changed	Coverage Δ
...ication/action/get/TransportGetLRONConfigAction.kt	94.44% <100.00%> (+0.10%)	⬆️

... and 13 files with indirect coverage changes

[dreamer-89]

Thanks @gaobinlong for this change.

Please note that using _primary preference may result in failure if primary shard is not avaialble thus there is availability hit. If availability is important than consistency, then you can use _primary_first parameter, which routes the request to primary first (as name suggest). If primary is unavailable the request is routed to replica shards, though data can be stale when served from replicas. The choice you have to make is b/w consistency (_primary) vs availablility (_primary_first). I will let you take this decision based on your use case.

CC @getsaurabh02 @bowenlan-amzn

[bowenlan-amzn]

No harm adding this. But may not be necessary IMO.
The reasoning is the time between LRON config indexed, and needed by sending notification would be much longer than the time for segRep to sync.

But if the consideration is to rule out any possible race condition, and adding this introduce small overhead, then I think we can do this for all search request in this repo