Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Flag to reduce precision of GeometryField #161

Merged
merged 1 commit into from
Apr 27, 2018
Merged

Flag to reduce precision of GeometryField #161

merged 1 commit into from
Apr 27, 2018

Conversation

alukach
Copy link
Contributor

@alukach alukach commented Mar 13, 2018

This is a continuation of #158 (I rebased the branch, making the PR ineligible for re-opening).

This is a feature proposal for adding flags to the GeometryField to reduce the precision of a geometry. Perhaps the geometry is stored in high precision in the database, but for the sake of reducing response-size it may make sense to round the coordinates to a certain level.

This features has two portions:

  1. Rounding all coordinates to a specified precision.
  2. Removing redundant coordinates. When coordinates get rounded, it's more likely that there will be redundant values. Consider rounding [ (1.1001, 1.2302), (1.1201, 1.2203), (1.1103, 1.240) ] to a precision value of 1, the resultant array would be [ (1.1, 1.2), (1.1, 1.2), (1.1, 1.2)]. For the sake of size, it may as well be represented as [ (1.1, 1.2) ] if this were a MultiPoint geometry or [ (1.1, 1.2), (1.1, 1.2) ] if this were a LineString geometry.

Tests and documentation are included within this PR. I have not upped the version of the library.

README.rst Outdated
<https://docs.python.org/3/library/functions.html#round>`_), rounding values to
the provided level of precision. E.g. A Point with lat/lng of
``[51.0486, -114.0708]`` passed through a ``GeometryField(precision=2)``
would return a Point with a lat/lng of ``[51.04, -114.07]``.
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't this be 51.05?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You would think, however as I learned when writing this PR, 5 rounds down in Python 3. I know, right?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, this is probably a better reference (see the "note") section: https://docs.python.org/dev/library/functions.html#round

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That is correct if the original value is 51.0456. This indeed rounds down to 51.04. But the digit in question is '8' so 51.0486 definitely rounds up to 51.05.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, yeah, right you are...

In [1]: round(51.0486, 2)
Out[1]: 51.05

Will fix that, thanks.

@alukach
Copy link
Contributor Author

alukach commented Mar 16, 2018

@auvipy @nemesisdesign Any input or feedback would be well appreciated.

@RamizSami
Copy link

Hey @alukach! Thanks for the update. This comment is related to the issue I pointed out on #158. The problem still exists after this update. Thanks

@alukach
Copy link
Contributor Author

alukach commented Mar 19, 2018

@RamizSami, could you please provide the WKT of the feature on question? I'll take a look.

@alukach
Copy link
Contributor Author

alukach commented Mar 26, 2018

@RamizSami Have you been able to put together a test case illustrating the problem that you're having? Without seeing the data you're using and how you're using this library, I am unable to debug.

@alukach
Copy link
Contributor Author

alukach commented Mar 29, 2018

@RamizSami Thanks for the data. We really only needed one geometry. In efforts to keep this PR tidy, could you do me a favour by deleting your past two comments? I uploaded 3 of your geometries here: https://gist.github.com/alukach/acfa950450d4c0a17026786941684a00

I made a test-case out of a simplified version of the first geometry and was able to use that to discover my error. It should be fixed now. Please pull the newest commit and test with your data again.

@auvipy @nemesisdesign Feel free to provide any input. I believe this is good to go.

@RamizSami
Copy link

@alukach deleted the comments. Also tested with the newest pull. It seems all good now

@alukach
Copy link
Contributor Author

alukach commented Apr 16, 2018

@auvipy @nemesisdesign Ping, any thoughts or barriers to getting this merged?

Copy link
Collaborator

@auvipy auvipy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good to me

@auvipy auvipy requested a review from nemesifier April 16, 2018 17:26
Copy link
Member

@nemesifier nemesifier left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great work @alukach 👍, see below.

Why so many commented lines like:

# [-105.0215, 39.5771],  # Dupe

Are these necessary?

def __init__(self, **kwargs):
def __init__(self, precision=None, remove_duplicates=False, **kwargs):
self.precision = precision
self.rm_dupes = remove_duplicates
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would really prefer if we could use readable instance attributes, method names and test method names as well, for consistency with the rest of the codebase

@@ -48,6 +62,39 @@ def validate_empty_values(self, data):
self.fail('required')
return super(GeometryField, self).validate_empty_values(data)

def _recursive_round(self, val, precision):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could you find a better name for val?

return tuple(self._recursive_round(v, precision) for v in val)
return round(val, precision)

def _rm_redundant_points(self, geom, geotype):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we use geometry and geo_type?

@nemesifier
Copy link
Member

PS: once done cleaning up, please clean up the commit messages too, would be great to have 1 or a few commits at max

@alukach
Copy link
Contributor Author

alukach commented Apr 18, 2018

@nemesisdesign

Why so many commented lines like:

# [-105.0215, 39.5771],  # Dupe

Are these necessary?

These are to identify to a developer that those values are the duplicated values from the source data that are removed from the response. I believe they will help keep those tests understandable and maintainable as time goes on.

PS: once done cleaning up, please clean up the commit messages too, would be great to have 1 or a few commits at max

Done, although I would recommend that in the future we use Github's Squash & Merge feature so that this is an automated step once the PR is approved.

Copy link
Member

@nemesifier nemesifier left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great, thanks 👍

@nemesifier nemesifier merged commit 46acc15 into openwisp:master Apr 27, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants