Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Use the determinant formula for calculating the orientation of a polygon #27967

Merged
merged 9 commits into from
Jul 31, 2018
Merged

Use the determinant formula for calculating the orientation of a polygon #27967

merged 9 commits into from
Jul 31, 2018

Conversation

DaveCTurner
Copy link
Contributor

The method for working out whether a polygon is clockwise or anticlockwise is mostly correct but doesn't work in some rare cases such as the included test case. This PR changes it to determine the orientation from the direction of the cross product of two of the edges of the polygon at a point on its convex hull, which should work more reliably.

@DaveCTurner DaveCTurner added :Analytics/Geo Indexing, search aggregations of geo points and shapes >bug v6.1.2 v6.2.0 v7.0.0 labels Dec 22, 2017
@DaveCTurner DaveCTurner requested a review from nknize December 22, 2017 19:24
@jasontedor jasontedor added v6.1.3 and removed v6.1.2 labels Jan 16, 2018
@colings86 colings86 added v6.3.0 and removed v6.2.0 labels Jan 22, 2018
@bleskes bleskes added v6.1.4 and removed v6.1.3 labels Jan 29, 2018
@DaveCTurner
Copy link
Contributor Author

@nknize could you have a look at this please?

@s1monw s1monw added v6.1.5 and removed >docs General docs changes v6.1.4 labels Mar 20, 2018
nknize
nknize reviewed Mar 26, 2018
View reviewed changes
Copy link
Contributor

@nknize nknize left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry for the delay. I'm good with this. As a side note I can make the orient method (https://github.com/apache/lucene-solr/blob/df9b88443430a85c4bb00af63ff1da911584e6f5/lucene/core/src/java/org/apache/lucene/geo/Polygon2D.java#L466) public in the Lucene GeoAPI. Might make a note to replace this method once that's done so we don't forget.

@DaveCTurner
Copy link
Contributor Author

Ok, I added a TODO note as requested. You said "I'm good with this" but didn't give an "approved" review - could you?

@DaveCTurner
Copy link
Contributor Author

Pinging @elastic/es-search-aggs (and @nknize 🙂)

@DaveCTurner
Copy link
Contributor Author

I think this is good to merge but needs a final LGTM - could someone from the Geo team take a look?

@DaveCTurner
Copy link
Contributor Author

Could someone from @elastic/es-search-aggs take a look at this please?

@DaveCTurner
Copy link
Contributor Author

@nknize @elastic/es-search-aggs could you take a last look at this please?

jtibshirani
jtibshirani reviewed Jul 25, 2018
View reviewed changes
Copy link
Contributor

@jtibshirani jtibshirani left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @DaveCTurner, I'm sorry you've been waiting so long for a review. I’m new to the geo code, but did my best to take a look and left a question.

It looks like in the latest version of lucene, the orient method is now public: https://github.com/apache/lucene-solr/blob/master/lucene/core/src/java/org/apache/lucene/geo/GeoUtils.java#L185. We're upgrading to the latest lucene snapshot now (#32390), so it may be nice to cut over to that when the upgrade goes in. We could also do it in a follow-up if you (understandably) don't want to wait.

server/src/main/java/org/elasticsearch/common/geo/builders/PolygonBuilder.java Outdated
final double determinant
= (points[offset + next].x - points[offset + top].x) * (points[offset + prev].y - points[offset + top].y)
- (points[offset + prev].x - points[offset + top].x) * (points[offset + next].y - points[offset + top].y);
assert determinant != 0.0;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have a couple questions about this assertion:

  • I think the determinant could be 0 if the three points lie in a straight line. Is my understanding correct that this won't happen because of the way we choose top? It might be nice to add a quick note to explain.
  • Are we concerned about numeric underflow here? I'm curious as to how we handle numeric issues in other parts of the geo code (in particular, whether we should throw a runtime exception when an issue is detected).

@DaveCTurner
Copy link
Contributor Author

Thanks for spotting that this has landed in Lucene @jtibshirani. #32390 is merged so I've updated this PR to defer to that.

I have a couple questions about this assertion:

  • I think the determinant could be 0 if the three points lie in a straight line. Is my understanding correct that this won't happen because of the way we choose top? It might be nice to add a quick note to explain.

On closer inspection the points could in fact be collinear at this point, but only if top is the last of them, in which case the edges intersect so the shape gets thrown out later on in its construction. I think it's better to reject that here as the message can be made clearer, so I've done that.

  • Are we concerned about numeric underflow here? I'm curious as to how we handle numeric issues in other parts of the geo code (in particular, whether we should throw a runtime exception when an issue is detected).

It seems not - the Lucene implementation is the same as the one written here, although at least it has a link to a more robust implementation:

  // see the "Orient2D" method described here:
  // http://www.cs.berkeley.edu/~jrs/meshpapers/robnotes.pdf
  // https://www.cs.cmu.edu/~quake/robust.html
  // Note that this one does not yet have the floating point tricks to be exact!

(the linked Orient2D method is also not 100% exact, I think, because it assumes support for unbounded exponents, but it's the best one I have come across). I also ran into a similar issue in JTS when investigating your collinearity question. I guess the argument is that for geospatial data this kind of error happens at the scale of µm, and some continents are moving further than that every hour, so it doesn't make sense to work with shapes that are so close to degeneracy anyway. Not a very satisfying answer.

jtibshirani
jtibshirani approved these changes Jul 30, 2018
View reviewed changes
Copy link
Contributor

@jtibshirani jtibshirani left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for looking into the collinearity + underflow issues! It looks good to me.

server/src/main/java/org/elasticsearch/common/geo/builders/PolygonBuilder.java
points[offset + next].x, points[offset + next].y);

if (determinantSign == 0) {
// Points are collinear, but `top` is not in the middle if so, so the edges either side of `top` are intersecting.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think there's an extra 'if so' here.

@DaveCTurner DaveCTurner merged commit 8b57e2e into elastic:master Jul 31, 2018
@DaveCTurner DaveCTurner deleted the 2017-12-22-orientation-bug branch July 31, 2018 07:30
DaveCTurner added a commit that referenced this pull request Jul 31, 2018
@DaveCTurner
Fix calculation of orientation of polygons (#27967)
3ae8bd8 
The method for working out whether a polygon is clockwise or anticlockwise is
mostly correct but doesn't work in some rare cases such as the included test
case. This commit fixes that.
@DaveCTurner DaveCTurner added v6.5.0 and removed v6.4.0 labels Jul 31, 2018
dnhatn added a commit that referenced this pull request Jul 31, 2018
@dnhatn
Merge branch 'master' into ccr
036cb3f 
* master:
  Logging: Make node name consistent in logger (#31588)
  Mute SSLTrustRestrictionsTests on JDK 11
  Increase max chunk size to 256Mb for repo-azure (#32101)
  Docs: Fix README upgrade mention (#32313)
  Changed ReindexRequest to use Writeable.Reader (#32401)
  Mute KerberosAuthenticationIT
  Fix AutoIntervalDateHistogram.testReduce random failures (#32301)
  fix no=>not typo (#32463)
  Mute QueryProfilerIT#testProfileMatchesRegular()
  HLRC: Add delete watch action (#32337)
  High-level client: fix clusterAlias parsing in SearchHit (#32465)
  Fix calculation of orientation of polygons (#27967)
  [Kerberos] Add missing javadocs (#32469)
  [Kerberos] Remove Kerberos bootstrap checks (#32451)
  Make get all app privs requires "*" permission (#32460)
  Switch security to new style Requests (#32290)
  Switch security spi example to new style Requests (#32341)
  Painless: Add PainlessConstructor (#32447)
  update rollover to leverage write-alias semantics (#32216)
  Update Fuzzy Query docs to clarify default behavior re max_expansions (#30819)
  INGEST: Clean up Java8 Stream Usage (#32059)
  Ensure KeyStoreWrapper decryption exceptions are handled (#32464)
dnhatn added a commit that referenced this pull request Aug 2, 2018
@dnhatn
Merge branch '6.x' into ccr-6.x
3016654 
* 6.x:
  Fix scriptdocvalues tests with dates
  Correct minor typo in explain.asciidoc for HLRC
  Fix painless whitelist and warnings from backporting #31441
  Build: Add elastic maven to repos used by BuildPlugin (#32549)
  Scripting: Conditionally use java time api in scripting (#31441)
  [ML] Improve error when no available field exists for rule scope (#32550)
  [ML] Improve error for functions with limited rule condition support (#32548)
  [ML] Remove multiple_bucket_spans
  [ML] Fix thread leak when waiting for job flush (#32196) (#32541)
  Painless: Clean Up PainlessField (#32525)
  Add @AwaitsFix for #32554
  Remove broken @link in Javadoc
  Add AwaitsFix to failing test - see #32546
  SQL: Added support for string manipulating functions with more than one parameter (#32356)
  [DOCS] Reloadable Secure Settings (#31713)
  Fix compilation error introduced by #32339
  [Rollup] Remove builders from TermsGroupConfig (#32507)
  Use hostname instead of IP with SPNEGO test (#32514)
  Switch x-pack rolling restart to new style Requests (#32339)
  [DOCS] Small fixes in rule configuration page (#32516)
  Painless: Clean up PainlessMethod (#32476)
  SQL: Add test for handling of partial results (#32474)
  Docs: Add missing migration doc for logging change
  Build: Remove shadowing from benchmarks (#32475)
  Docs: Add all JDKs to CONTRIBUTING.md
  Logging: Make node name consistent in logger (#31588)
  High-level client: fix clusterAlias parsing in SearchHit (#32465)
  REST high-level client: parse back _ignored meta field (#32362)
  backport fix of reduceRandom fix (#32508)
  Add licensing enforcement for FIPS mode (#32437)
  INGEST: Clean up Java8 Stream Usage (#32059) (#32485)
  Improve the error message when an index is incompatible with field aliases. (#32482)
  Mute testFilterCacheStats
  Scripting: Fix painless compiler loader to know about context classes (#32385)
  [ML][DOCS] Fix typo applied_to => applies_to
  Mute SSLTrustRestrictionsTests on JDK 11
  Changed ReindexRequest to use Writeable.Reader (#32401)
  Increase max chunk size to 256Mb for repo-azure (#32101)
  Mute KerberosAuthenticationIT
  fix no=>not typo (#32463)
  HLRC: Add delete watch action (#32337)
  Fix calculation of orientation of polygons (#27967)
  [Kerberos] Add missing javadocs (#32469)
  Fix missing JavaDoc for @throws in several places in KerberosTicketValidator.
  Make get all app privs requires "*" permission (#32460)
  Ensure KeyStoreWrapper decryption exceptions are handled (#32472)
  update rollover to leverage write-alias semantics (#32216)
  [Kerberos] Remove Kerberos bootstrap checks (#32451)
  Switch security to new style Requests (#32290)
  Switch security spi example to new style Requests (#32341)
  Painless: Add PainlessConstructor (#32447)
  Update Fuzzy Query docs to clarify default behavior re max_expansions (#30819)
  Remove > from Javadoc (fatal with Java 11)
  Tests: Fix convert error tests to use fixed value (#32415)
  IndicesClusterStateService should replace an init. replica with an init. primary with the same aId (#32374)
  auto-interval date histogram - 6.x backport (#32107)
  [CI] Mute DocumentSubsetReaderTests testSearch
  [TEST] Mute failing InternalEngineTests#testSeqNoAndCheckpoints
  TEST: testDocStats should always use forceMerge (#32450)
  TEST: Avoid deletion in FlushIT
  AwaitsFix IndexShardTests#testDocStats
  Painless: Add method type to method. (#32441)
  Remove reference to non-existent store type (#32418)
  [TEST] Mute failing FlushIT test
  Fix ordering of bootstrap checks in docs (#32417)
  Wrong discovery.type for azure in breaking changes (#32432)
  Mute ConvertProcessorTests failing tests
  TESTS: Move netty leak detection to paranoid level (#32354) (#32425)
  Upgrade to Lucene-7.5.0-snapshot-608f0277b0 (#32390)
  [Kerberos] Avoid vagrant update on precommit (#32416)
  TEST: Avoid triggering merges in FlushIT
  [DOCS] Fixes formatting of scope object in job resource
  Switch x-pack/plugin to new style Requests (#32327)
  Release requests in cors handle (#32410)
  Remove BouncyCastle dependency from runtime (#32402)
  Copy missing segment attributes in getSegmentInfo (#32396)
  Rest HL client: Add put license action (#32214)
  Docs: Correcting a typo in tophits (#32359)
  Build: Stop double generating buildSrc pom (#32408)
  Switch x-pack full restart to new style Requests (#32294)
  Painless: Clean Up PainlessClass Variables (#32380)
  [ML] Consistent pattern for strict/lenient parser names (#32399)
  Add Restore Snapshot High Level REST API
  Update update-settings.asciidoc (#31378)
  Introduce index store plugins (#32375)
  Rank-Eval: Reduce scope of an unchecked supression
  Make sure _forcemerge respects `max_num_segments`. (#32291)
jasontedor added a commit to jasontedor/elasticsearch that referenced this pull request Aug 3, 2018
@jasontedor
Merge branch 'ccr' into ccr-stats
a956a9b 
* ccr: (24 commits)
  Remove _xpack from CCR APIs (elastic#32563)
  TEST: Avoid merges in testRecoveryWithOutOfOrderDelete
  Logging: Make node name consistent in logger (elastic#31588)
  Mute SSLTrustRestrictionsTests on JDK 11
  Increase max chunk size to 256Mb for repo-azure (elastic#32101)
  Docs: Fix README upgrade mention (elastic#32313)
  Changed ReindexRequest to use Writeable.Reader (elastic#32401)
  Mute KerberosAuthenticationIT
  Fix AutoIntervalDateHistogram.testReduce random failures (elastic#32301)
  fix no=>not typo (elastic#32463)
  Mute QueryProfilerIT#testProfileMatchesRegular()
  HLRC: Add delete watch action (elastic#32337)
  High-level client: fix clusterAlias parsing in SearchHit (elastic#32465)
  Fix calculation of orientation of polygons (elastic#27967)
  [Kerberos] Add missing javadocs (elastic#32469)
  [Kerberos] Remove Kerberos bootstrap checks (elastic#32451)
  Make get all app privs requires "*" permission (elastic#32460)
  Switch security to new style Requests (elastic#32290)
  Switch security spi example to new style Requests (elastic#32341)
  Painless: Add PainlessConstructor (elastic#32447)
  ...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@nknize nknize nknize left review comments

@jtibshirani jtibshirani jtibshirani approved these changes

Assignees
No one assigned
Labels
:Analytics/Geo Indexing, search aggregations of geo points and shapes >bug v6.5.0 v7.0.0-beta1
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
7 participants
@DaveCTurner @nknize @jtibshirani @colings86 @s1monw @bleskes @jasontedor