Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add helper methods for RFC 3339 datetime handling #760

Merged

Conversation

philvarner
Copy link
Collaborator

Related Issue(s): n/a

Description:

There are a number of places in the STAC ecosystem where datetimes are converted two and from RFC 3339 strings. Many of these places are doing this incorrectly. These changes aim to add a correct implementations to:

  • rfc3339_str_to_datetime: parse ONLY a valid RFC 3339 string to a datetime
  • str_to_interval: parse an interval string into a datetime tuple
  • now_in_utc: create a datetime with UTC set as the datetime (since utcnow() inexplicably does not do this)
  • now_to_rfc3339_str: create an RFC3339 string representing now

My primary motivation is to replace the numerous uses of the stac_pydantic constant DATETIME_RFC339(sic) with uses of these methods. Other libraries that depend on pystac will also use them, and there are likely several places even within pystac that these could be used instead of the current code.

One downside to this is that the tests need the pytest parameterized decorator to run, and this doesn't work in the class-based test modules that unittest requires. I ran the existing tests with pytest and it seems to run them all fine, so we could change to using pytest to run the test without needing to convert all the tests from unittest to pytest.

PR Checklist:

  • Code is formatted (run pre-commit run --all-files)
  • Tests pass (run scripts/test)
  • Documentation has been updated to reflect changes, if applicable
  • This PR maintains or improves overall codebase code coverage.
  • Changes are added to the CHANGELOG. See the docs for information about adding to the changelog.

@codecov-commenter
Copy link

codecov-commenter commented Mar 3, 2022

Codecov Report

Base: 90.21% // Head: 90.21% // Increases project coverage by +0.00% 🎉

Coverage data is based on head (e7a4d0d) compared to base (2d01225).
Patch coverage: 100.00% of modified lines in pull request are covered.

📣 This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #760   +/-   ##
=======================================
  Coverage   90.21%   90.21%           
=======================================
  Files          47       47           
  Lines        6090     6094    +4     
  Branches      915      915           
=======================================
+ Hits         5494     5498    +4     
  Misses        414      414           
  Partials      182      182           
Impacted Files Coverage Δ
pystac/utils.py 95.55% <100.00%> (+0.13%) ⬆️

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

@philvarner
Copy link
Collaborator Author

It may be more appropriate to put str_to_interval in stac-fastapi, since the datetime interval format is only used by query parameters.

@philvarner
Copy link
Collaborator Author

looks like i need to write some stubs for ciso8601

@duckontheweb
Copy link
Contributor

looks like i need to write some stubs for ciso8601

@philvarner I think it would also be fine to just ignore these in the mypy.ini file as we've done for other libraries without stubs.

@philvarner
Copy link
Collaborator Author

looks like i need to write some stubs for ciso8601

@philvarner I think it would also be fine to just ignore these in the mypy.ini file as we've done for other libraries without stubs.

Okay, I'll do that. Weird thing is that it looks like ciso8601 has stubs defined. I haven't used mypy that much, so I don't really know how the stubs are supposed to be resolved, etc.

setup.py Outdated Show resolved Hide resolved
@duckontheweb
Copy link
Contributor

looks like i need to write some stubs for ciso8601

@philvarner I think it would also be fine to just ignore these in the mypy.ini file as we've done for other libraries without stubs.

Okay, I'll do that. Weird thing is that it looks like ciso8601 has stubs defined. I haven't used mypy that much, so I don't really know how the stubs are supposed to be resolved, etc.

Yeah, that is peculiar. I'll look into this a bit more and see if there is a better way to solve this besides ignoring the import.

@philvarner
Copy link
Collaborator Author

looks like i need to write some stubs for ciso8601

@philvarner I think it would also be fine to just ignore these in the mypy.ini file as we've done for other libraries without stubs.

Okay, I'll do that. Weird thing is that it looks like ciso8601 has stubs defined. I haven't used mypy that much, so I don't really know how the stubs are supposed to be resolved, etc.

Yeah, that is peculiar. I'll look into this a bit more and see if there is a better way to solve this besides ignoring the import.

I needed to add it to the requirements-test.txt and setup.py validation requirements.

setup.py Outdated Show resolved Hide resolved
setup.py Outdated Show resolved Hide resolved
tests/test_utils2.py Outdated Show resolved Hide resolved
requirements-test.txt Outdated Show resolved Hide resolved
@lossyrob
Copy link
Member

lossyrob commented Mar 9, 2022

One downside to this is that the tests need the pytest parameterized decorator to run, and this doesn't work in the class-based test modules that unittest requires. I ran the existing tests with pytest and it seems to run them all fine, so we could change to using pytest to run the test without needing to convert all the tests from unittest to pytest.

I'd be happy with a change to pytest personally; however I wonder if that should be a larger PR that does a wholesale conversion rather than have multiple testing frameworks in the test. You could use a unittest.TestCase with self.subTest to test over each of the cases; for example here:

for example in TestCases.get_examples_info():
with self.subTest(example.path):

As it stands there's not a change to scripts/test to use pytest, so these unit tests wouldn't be run as part of CI.

pystac/utils.py Outdated Show resolved Hide resolved
@gadomski gadomski changed the title add helper methods for RFC 3339 datetime handling Add helper methods for RFC 3339 datetime handling Feb 1, 2023
@gadomski gadomski assigned gadomski and unassigned gadomski Feb 7, 2023
@gadomski gadomski dismissed their stale review February 10, 2023 14:44

I made some changes, so I need someone else to review.

@gadomski gadomski requested a review from pjhartzell February 10, 2023 14:56
@gadomski gadomski self-assigned this Feb 10, 2023
@gadomski gadomski enabled auto-merge February 10, 2023 14:57
@gadomski gadomski force-pushed the pv/add_datetime_handling_functions branch from e7a4d0d to bab2f10 Compare February 10, 2023 15:38
Includes some extensive tests.

Co-authored-by: Pete Gadomski <[email protected]>
@gadomski gadomski force-pushed the pv/add_datetime_handling_functions branch from bab2f10 to 8382193 Compare February 10, 2023 15:39
Copy link
Collaborator

@pjhartzell pjhartzell left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me.

@gadomski gadomski added this pull request to the merge queue Feb 13, 2023
Merged via the queue into stac-utils:main with commit 9865374 Feb 13, 2023
@philvarner
Copy link
Collaborator Author

Got this merged in just under a year!

@philvarner philvarner deleted the pv/add_datetime_handling_functions branch February 13, 2023 17:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants