Extensible design to add new query and field type support for Star Tree

[expani]

Description

This PR introduces constructs to help convert user queries into the equivalent star tree traversal by generating the appropriate DimensionFilter based on the type of query.

DimensionFilterMapper acts as an equivalent mapper for Star Tree supported fields and queries.

StarTreeFilterProvider is responsible for converting the user queries ( represented by QueryBuilder )

I have changed all the older tests to point towards the new classes to ensure that we get coverage for at least numeric term queries.

We have tested term, terms and range queries on numeric and keyword over http_logs for sanity of the code and fixed some tricky bugs on different cases. Thanks to @bharath-techie for help with creating end to end query test with randomness.

I will be adding Unit tests for Range and other new classes that are not covered by existing tests in the coming days.

Most of the TODO left around the code will be picked up in future when supporting Filter Aggregations and Boolean queries. If that's not the case, please comment on the same.

Related Issues

#16537
#16538
#16539

NOTE

I had to close the older PR due to it becoming very hard to recover after a rebase gone wrong. Apologies to the reviewers for the inconvenience caused.

[expani]

@msfroh @bharath-techie Kindly help with the review.

[github-actions]

✅ Gradle check result for 7153ccd: SUCCESS

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 0.00%. Comparing base (3c55a31) to head (c177a3f).
Report is 1 commits behind head on main.

Additional details and impacted files

@@              Coverage Diff              @@
##               main   #17137       +/-   ##
=============================================
- Coverage     72.35%        0   -72.36%     
=============================================
  Files          5298        0     -5298     
  Lines        304603        0   -304603     
  Branches      44187        0    -44187     
=============================================
- Hits         220397        0   -220397     
+ Misses        66095        0    -66095     
+ Partials      18111        0    -18111     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[msfroh]

Overall, there's a lot of really nice refactoring in this PR. Of course, whenever you clean up some stuff, it highlights some of the mess in what's left. A lot of my comments are around things that could be even cleaner.

Also, for your two providers (StarTreeFilterProvider and DimensionFilterMapper), I believe that String keys are more consistent with existing pluggable interfaces.

[github-actions]

❌ Gradle check result for 775330e: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions]

❌ Gradle check result for 204a107: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions]

❌ Gradle check result for c177a3f: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[expani]

./gradlew ':server:internalClusterTest' --tests "org.opensearch.cluster.SpecificClusterManagerNodesIT.testElectOnlyBetweenClusterManagerNodes" -Dtests.seed=51E8DF2F5E1DE33C -Dtests.security.manager=true -Dtests.jvm.argline="-XX:TieredStopAtLevel=1 -XX:ReservedCodeCacheSize=64m" -Dtests.locale=fr-CI -Dtests.timezone=America/Montevideo -Druntime.java=21

Flaky UNRELATED test passing locally trying with closing and reopening PR again.

[github-actions]

✅ Gradle check result for c177a3f: SUCCESS

[opensearch-trigger-bot]

The backport to 2.x failed:

The process '/usr/bin/git' failed with exit code 128

To backport manually, run these commands in your terminal:

# Navigate to the root of your repository
cd $(git rev-parse --show-toplevel)
# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add ../.worktrees/OpenSearch/backport-2.x 2.x
# Navigate to the new working tree
pushd ../.worktrees/OpenSearch/backport-2.x
# Create a new branch
git switch --create backport/backport-17137-to-2.x
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 b9ddef9c9749e02deeccf95946b60b2303bbfd9c
# Push it to GitHub
git push --set-upstream origin backport/backport-17137-to-2.x
# Go back to the original working tree
popd
# Delete the working tree
git worktree remove ../.worktrees/OpenSearch/backport-2.x

Then, create a pull request where the base branch is 2.x and the compare/head branch is backport/backport-17137-to-2.x.