Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BUG] GeoHashGridIT.testGeoShapes failure #6222

Closed
andrross opened this issue Feb 7, 2023 · 5 comments
Closed

[BUG] GeoHashGridIT.testGeoShapes failure #6222

andrross opened this issue Feb 7, 2023 · 5 comments
Labels
bug Something isn't working flaky-test Random test failure that succeeds on second run Geospatial >test-failure Test failure from CI, local build, etc.

Comments

@andrross
Copy link
Member

andrross commented Feb 7, 2023

See https://build.ci.opensearch.org/job/gradle-check/10743/consoleFull

REPRODUCE WITH: ./gradlew ':modules:geo:internalClusterTest' --tests "org.opensearch.geo.search.aggregations.bucket.GeoHashGridIT.testGeoShapes" -Dtests.seed=58EEE2632DEE6DD -Dtests.security.manager=true -Dtests.jvm.argline="-XX:TieredStopAtLevel=1 -XX:ReservedCodeCacheSize=64m" -Dtests.locale=ar-OM -Dtests.timezone=Europe/Madrid -Druntime.java=19

org.opensearch.geo.search.aggregations.bucket.GeoHashGridIT > testGeoShapes FAILED
    Failed to execute phase [query], all shards failed; shardFailures {[C7ePtMCAQWamX99P3Fj6MA][geoshape_index][0]: RemoteTransportException[[node_s0][127.0.0.1:37239][indices:data/read/search[phase/query]]]; nested: QueryPhaseExecutionException[Query Failed [Failed to execute main query]]; nested: ArrayIndexOutOfBoundsException[Index 17680 out of bounds for length 17680]; }
        at __randomizedtesting.SeedInfo.seed([58EEE2632DEE6DD:EF3356E9BDA2B5C0]:0)
        at app//org.opensearch.action.search.AbstractSearchAsyncAction.onPhaseFailure(AbstractSearchAsyncAction.java:663)
        at app//org.opensearch.action.search.AbstractSearchAsyncAction.executeNextPhase(AbstractSearchAsyncAction.java:372)
        at app//org.opensearch.action.search.AbstractSearchAsyncAction.onPhaseDone(AbstractSearchAsyncAction.java:698)
        at app//org.opensearch.action.search.AbstractSearchAsyncAction.onShardFailure(AbstractSearchAsyncAction.java:471)
        at app//org.opensearch.action.search.AbstractSearchAsyncAction$1.onFailure(AbstractSearchAsyncAction.java:294)
        at app//org.opensearch.action.search.SearchExecutionStatsCollector.onFailure(SearchExecutionStatsCollector.java:104)
        at app//org.opensearch.action.ActionListenerResponseHandler.handleException(ActionListenerResponseHandler.java:74)
        at app//org.opensearch.action.search.SearchTransportService$ConnectionCountingHandler.handleException(SearchTransportService.java:747)
        at app//org.opensearch.transport.TransportService$6.handleException(TransportService.java:783)
        at app//org.opensearch.transport.TransportService$ContextRestoreResponseHandler.handleException(TransportService.java:1391)
        at app//org.opensearch.transport.TransportService$DirectResponseChannel.processException(TransportService.java:1505)
        at app//org.opensearch.transport.TransportService$DirectResponseChannel.sendResponse(TransportService.java:1479)
        at app//org.opensearch.transport.TaskTransportChannel.sendResponse(TaskTransportChannel.java:79)
        at app//org.opensearch.transport.TransportChannel.sendErrorResponse(TransportChannel.java:71)
        at app//org.opensearch.action.support.ChannelActionListener.onFailure(ChannelActionListener.java:70)
        at app//org.opensearch.action.ActionRunnable.onFailure(ActionRunnable.java:103)
        at app//org.opensearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:54)
        at app//org.opensearch.threadpool.TaskAwareRunnable.doRun(TaskAwareRunnable.java:78)
        at app//org.opensearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:52)
        at app//org.opensearch.common.util.concurrent.TimedRunnable.doRun(TimedRunnable.java:59)
        at app//org.opensearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:806)
        at app//org.opensearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:52)
        at java.****@19.0.2/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
        at java.****@19.0.2/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
        at java.****@19.0.2/java.lang.Thread.run(Thread.java:1589)

        Caused by:
        [Index 17680 out of bounds for length 17680]; nested: ArrayIndexOutOfBoundsException[Index 17680 out of bounds for length 17680];
            at app//org.opensearch.OpenSearchException.guessRootCauses(OpenSearchException.java:678)
            at app//org.opensearch.action.search.AbstractSearchAsyncAction.executeNextPhase(AbstractSearchAsyncAction.java:370)
            ... 23 more

            Caused by:
            java.lang.ArrayIndexOutOfBoundsException: Index 17680 out of bounds for length 17680
                at org.opensearch.index.fielddata.SortingNumericDocValues.nextValue(SortingNumericDocValues.java:123)
                at org.opensearch.geo.search.aggregations.bucket.geogrid.GeoGridAggregator$1.collect(GeoGridAggregator.java:104)
                at org.opensearch.search.aggregations.LeafBucketCollector.collect(LeafBucketCollector.java:123)
                at org.apache.lucene.search.MultiCollector$MultiLeafCollector.collect(MultiCollector.java:222)
                at org.apache.lucene.search.MatchAllDocsQuery$1$1.score(MatchAllDocsQuery.java:63)
                at org.opensearch.search.internal.CancellableBulkScorer.score(CancellableBulkScorer.java:71)
                at org.apache.lucene.search.BulkScorer.score(BulkScorer.java:38)
                at org.opensearch.search.internal.ContextIndexSearcher.searchLeaf(ContextIndexSearcher.java:279)
                at org.opensearch.search.internal.ContextIndexSearcher.search(ContextIndexSearcher.java:250)
                at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:551)
                at org.opensearch.search.query.QueryPhase.searchWithCollector(QueryPhase.java:349)
                at org.opensearch.search.query.QueryPhase$DefaultQueryPhaseSearcher.searchWithCollector(QueryPhase.java:443)
                at org.opensearch.search.query.QueryPhase$DefaultQueryPhaseSearcher.searchWith(QueryPhase.java:432)
                at org.opensearch.search.query.QueryPhase.executeInternal(QueryPhase.java:275)
                at org.opensearch.search.query.QueryPhase.execute(QueryPhase.java:152)
                at org.opensearch.search.SearchService.loadOrExecuteQueryPhase(SearchService.java:525)
                at org.opensearch.search.SearchService.executeQueryPhase(SearchService.java:589)
                at org.opensearch.search.SearchService$2.lambda$onResponse$0(SearchService.java:558)
                at org.opensearch.action.ActionRunnable.lambda$supply$0(ActionRunnable.java:73)
                at org.opensearch.action.ActionRunnable$2.doRun(ActionRunnable.java:88)
                at org.opensearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:52)
                ... 8 more

/cc @navneet1v
@andrross andrross added bug Something isn't working >test-failure Test failure from CI, local build, etc. untriaged flaky-test Random test failure that succeeds on second run Geospatial labels Feb 7, 2023
@andrross
Copy link
Member Author

andrross commented Feb 7, 2023

Also just want to note that this test appears to take about 3 minutes to run when it passes. These things can really add up (as evidenced by the fact that gradle check currently takes about half an hour) so if possible it would be great to see if we can get the same coverage with a faster test runtime.

@andrross andrross mentioned this issue Feb 7, 2023
Closed
@navneet1v
Copy link
Contributor

navneet1v commented Feb 7, 2023 •
edited
Loading

I will look into that. I am aware of the test runtime. but the longer runtime doesn't happen everytime. As the test data is generated randomly this sometime leads to test running long time.

Will look into this failure

@navneet1v
Copy link
Contributor

I am able to figure out the Root Cause of the Exception. I am currently working on it. Will raise PR soon.

navneet1v added a commit to navneet1v/OpenSearch that referenced this issue Feb 8, 2023
@navneet1v
[opensearch-project#6187, opensearch-project#6222] Fixing the GeoShap…
77a1f0e 
…es GeoHash and GeoTile Aggregations Integration tests to make them more robust.

Signed-off-by: Navneet Verma <[email protected]>
@navneet1v navneet1v mentioned this issue Feb 8, 2023
Merged
6 tasks
navneet1v added a commit to navneet1v/OpenSearch that referenced this issue Feb 8, 2023
@navneet1v
[opensearch-project#6187, opensearch-project#6222] Fixing the GeoShap…
facb25a 
…es GeoHash and GeoTile Aggregations Integration tests to make them more robust.

Signed-off-by: Navneet Verma <[email protected]>
@navneet1v
Copy link
Contributor

@andrross PR is raised #6242

@kotwanikunal kotwanikunal mentioned this issue Feb 8, 2023
Closed
navneet1v added a commit to navneet1v/OpenSearch that referenced this issue Feb 9, 2023
@navneet1v
[opensearch-project#6187, opensearch-project#6222] Fixing the GeoShap…
0c011cf 
…es GeoHash and GeoTile Aggregations Integration tests.

Changes done:
* Fixed the ArrayIndexOutOfBoundsException.
* Reduced the precision for GeoShapes Aggregation IT testing.

Signed-off-by: Navneet Verma <[email protected]>
@navneet1v navneet1v mentioned this issue Feb 9, 2023
Closed
navneet1v added a commit to navneet1v/OpenSearch that referenced this issue Feb 9, 2023
@navneet1v
[opensearch-project#6187, opensearch-project#6222] Fixing the GeoShap…
f139318 
…es GeoHash and GeoTile Aggregations Integration tests.

Changes done:
* Fixed the ArrayIndexOutOfBoundsException.
* Reduced the precision for GeoShapes Aggregation IT testing.

Signed-off-by: Navneet Verma <[email protected]>
nknize pushed a commit that referenced this issue Feb 10, 2023
@navneet1v
[#6187, #6222] Fixing the GeoShapes GeoHash and GeoTile Aggregations …
9386cde 
…Integration tests. (#6242)

Changes done:
* Fixed the ArrayIndexOutOfBoundsException.
* Reduced the precision for GeoShapes Aggregation IT testing.

Signed-off-by: Navneet Verma <[email protected]>
@navneet1v
Copy link
Contributor

The PR is merged. Resolving the issue.

heemin32 pushed a commit to heemin32/OpenSearch that referenced this issue Jun 28, 2023
@navneet1v @heemin32
[opensearch-project#6187, opensearch-project#6222] Fixing the GeoShap…
9105f0c 
…es GeoHash and GeoTile Aggregations Integration tests. (opensearch-project#6242)

Changes done:
* Fixed the ArrayIndexOutOfBoundsException.
* Reduced the precision for GeoShapes Aggregation IT testing.

Signed-off-by: Navneet Verma <[email protected]>
andrross pushed a commit that referenced this issue Jun 30, 2023
@heemin32 @navneet1v
[Backport 2.x] GeoTile, and GeoGrid aggregation (#8295)
0dbafea 
* Add GeoTile and GeoHash Grid aggregations on GeoShapes. (#5589)

Src files for GeoTile and GeoHash Aggregations on GeoShape with integration
tests.

Signed-off-by: Navneet Verma <[email protected]>

* [opensearch-project/geospatial#212] Fixing the IT for GeoTilesAggrega… (#6120)

Fixing the IT for GeoTilesAggregation.

Signed-off-by: Navneet Verma <[email protected]>

* [#6187, #6222] Fixing the GeoShapes GeoHash and GeoTile Aggregations Integration tests. (#6242)

Changes done:
* Fixed the ArrayIndexOutOfBoundsException.
* Reduced the precision for GeoShapes Aggregation IT testing.

Signed-off-by: Navneet Verma <[email protected]>

* [#7101] Fixing the GeoTileIT#testMultivaluedGeoPointsAggregation test case. (#7166)

The issue was happening because we encode the GeoPoint as long and error comes in the precision due to that encoding. The error was not taken care while generating the exepected tiles count for execpected output.

Signed-off-by: Navneet Verma <[email protected]>

---------

Signed-off-by: Navneet Verma <[email protected]>
Signed-off-by: Heemin Kim <[email protected]>
Co-authored-by: Navneet Verma <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working flaky-test Random test failure that succeeds on second run Geospatial >test-failure Test failure from CI, local build, etc.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@navneet1v @andrross