CI polish to target 0.0.7 release #58

psilentp · 2020-06-05T00:18:40Z

Cleans up some of the CI by moving all the tests to tox, also enables pypi deployment by pushing a tag (haven't tested this yet) and moves control of all versioning to vcs.

consolidated github action and .tox workflow steps closes Test on python 3.8 #56
add another workflow to run black and flake8. At the moment these are set to always return a pass but we can read the error reports in the action output console like this one Perhaps we can set these as mandatory later.
Added some badges to report on the test results and pypi version.
moves versioning to setuptools_scm
workflow code to push to test-pypi on all pushes pypi on tag push.

clbarnes · 2020-06-05T07:34:16Z

Is a separate workflow for doc publishing strictly necessary? Readthedocs automatically generates and hosts pages generated with sphinx.

psilentp · 2020-06-05T08:00:25Z

Is a separate workflow for doc publishing strictly necessary? Readthedocs automatically generates and hosts pages generated with sphinx.

Probably not - though we've already got some published to github pages. Mostly I was hoping to fix some errors in the documentation (like the one in #57) and push them out with a tagged 0.0.7 release - so at least we have a starting point.

I haven't used readthedocs - I'm game for whatever is easiest.

clbarnes · 2020-06-05T08:07:20Z

Ah, having refreshed my memory, there are 2 separate documentation pages - gh-pages as a landing page/ quickstart tutorial, readthedocs for sphinx-generated API docs. That seems like a reasonable distinction to have - whatever simplicity would be gained from having a single host may be wiped out by the hoops you'd have to jump through for sphinx to deal with the landing page.

psilentp · 2020-06-05T19:35:04Z

Ah, having refreshed my memory, there are 2 separate documentation pages - gh-pages as a landing page/ quickstart tutorial, readthedocs for sphinx-generated API docs. That seems like a reasonable distinction to have - whatever simplicity would be gained from having a single host may be wiped out by the hoops you'd have to jump through for sphinx to deal with the landing page.

Just read up on readthedocs and it looks like it handles tags nicely, so as long as we are reasonably committed to semver there shouldn't be a problem.

This reverts commit 0e1e702.

clbarnes · 2020-06-06T08:50:56Z

figurefirst/svg_to_axes.py

@@ -7,6 +7,9 @@
 import time
 import sys
 import copy
+import sys
+if sys.version_info < (3, 6):


IIRC dicts being ordered is an implementation detail of CPython 3.6, but doesn't become part of the language spec until 3.7. I'm sure the overwhelming use of fifi is from CPython but it's not an expensive switch to make.

Implementing the shim caused some of the tests to break in challenging ways - and potentially related to #17 . So I thought it might be best to try and push the fix for #53 into another PR.

clbarnes · 2020-06-06T08:53:54Z

.github/workflows/publish-workflow.yml

@@ -0,0 +1,26 @@
+name: Publish to PyPI and TestPyPI


Am I right in thinking that it's totally personal preference whether you split into multiple workflows or just have one workflow with multiple jobs? There's little benefit either way? I'm pretty new to them.

In this particular case, can you have inter-workflow dependencies? It may be worthwhile to prove that the tests pass before pushing to pypi.

Excellent point: it is simpler to implement the dependency on the tests by keeping them in the same workflow. As far as I know the on: clause is workflow specific - so there might be some reasons for having separate workflows; my original thoughts were to have the build and deploy run on a release event, but I think having all the CI together is cleaner in this case.

This reverts commit 50e9fce.

clbarnes

Looking good!

The unused imports lints from __init__ can be fixed by listing the "exported" items' names like this: __all__ = ["svg_to_axes", "mpl_functions", ...].

I like the usage of the scm version in setup, but you need to be careful about build-time dependencies. setup_requires doesn't actually install that dependency, it just checks whether it's there - I think the preferred way to do it is to add a pyproject.toml which specifies the build system (setuptools) and the build time dependencies (setuptools and setuptool-scm with hard versions).

The flake8 config needs to be compatible with black (from a blog post)

[flake8]
ignore = E203, E266, E501, W503, F403, F401
max-line-length = 89
max-complexity = 18
select = B,C,E,F,W,T4,B9

This reverts commit 478c726.

clbarnes · 2020-06-20T16:22:48Z

examples/example_hello_world.py

 layout.make_mplfigures()

-d = np.array([[144, 57], [138, 57], [138, 59], [141, 61], [141, 82], [138, 84], [138, 85], [142, 85], [147, 85], [147, 84], [144, 82], [144, 57], [144, 57], [155, 57], [149, 57], [149, 59], [152, 61], [152, 82], [149, 84], [149, 85], [153, 85], [158, 85], [158, 84], [155, 82], [155, 57], [155, 57], [273, 57], [267, 57], [267, 59], [270, 61], [270, 82], [267, 84], [267, 85], [271, 85], [276, 85], [276, 84], [273, 82], [273, 57], [273, 57], [295, 57], [289, 57], [289, 59], [292, 61], [292, 70], [287, 67], [278, 76], [287, 85], [292, 83], [292, 85], [298, 85], [298, 84], [295, 81], [295, 57], [295, 57], [90, 57], [90, 59], [91, 59], [94, 61], [94, 82], [91, 84], [90, 84], [90, 85], [96, 85], [102, 85], [102, 84], [101, 84], [98, 82], [98, 71], [110, 71], [110, 82], [107, 84], [106, 84], [106, 85], [112, 85], [118, 85], [118, 84], [117, 84], [113, 82], [113, 61], [117, 59], [118, 59], [118, 57], [112, 58], [106, 57], [106, 59], [107, 59], [110, 61], [110, 70], [98, 70], [98, 61], [101, 59], [102, 59], [102, 57], [96, 58], [90, 57], [90, 57], [193, 57], [193, 59], [197, 60], [205, 85], [205, 86], [206, 85], [213, 65], [219, 85], [220, 86], [221, 85], [229, 61], [233, 59], [233, 57], [229, 58], [224, 57], [224, 59], [228, 61], [227, 62], [221, 80], [215, 60], [215, 60], [218, 59], [218, 57], [213, 58], [208, 57], [208, 59], [211, 60], [212, 63], [207, 80], [200, 60], [200, 60], [203, 59], [203, 57], [198, 58], [193, 57], [193, 57], [128, 67], [120, 76], [129, 85], [135, 80], [135, 80], [134, 80], [129, 84], [125, 82], [123, 76], [134, 76], [135, 75], [128, 67], [128, 67], [169, 67], [160, 76], [169, 85], [178, 76], [169, 67], [169, 67], [240, 67], [231, 76], [240, 85], [249, 76], [240, 67], [240, 67], [257, 67], [251, 68], [251, 69], [254, 71], [254, 82], [251, 84], [251, 85], [256, 85], [261, 85], [261, 84], [260, 84], [257, 82], [257, 75], [262, 68], [262, 68], [261, 70], [263, 71], [265, 70], [262, 67], [257, 71], [257, 67], [257, 67], [128, 68], [133, 75], [123, 75], [128, 68], [128, 68], [169, 68], [173, 70], [174, 76], [173, 81], [169, 84], [164, 82], [163, 76], [164, 70], [169, 68], [169, 68], [240, 68], [244, 70], [246, 76], [245, 81], [240, 84], [235, 82], [234, 76], [235, 70], [240, 68], [240, 68], [287, 68], [292, 70], [292, 72], [292, 80], [292, 82], [287, 84], [283, 82], [281, 76], [283, 71], [287, 68], [287, 68]])
+d = np.array(


Could this long literal be factored out into a CSV?

Maybe, but it's sort of nice to have everything in a single file for an "example". On a side note, it would be great to have an actual test suite rather than using the examples as a smoke test.

clbarnes · 2020-06-20T16:24:19Z

.github/workflows/continuous-integration-workflow.yml

+        run: |
+          python setup.py sdist
+      - name: Publish distribution 📦 to Test PyPI


If this runs every time, will it fail a build if the same version is pushed repeatedly?

This should push something like figurefirst-0.0.6.dev{distance}. Of course given this workflow where we are hitting test-pipy with every push on any branch, there is a potential collision if two branches get pushed at the same distance.

My original idea was to implement something like this:
https://github.com/scikit-hep/pyhf/tree/master/.github/workflows
Which has a separate build and publish workflow for merged branches. But as we discussed above, this might just add unnecessary complexity to the project given its size.

I've enabled continue on error for this step so that the failures in the rare cases of these collisions shouldn't keep us from pushing a release. I'm not sure it is actually necessary to set continue on error to true, but it does get rid of the annoying email updates, and we can still double check the console output before merging.

clbarnes · 2020-06-20T16:27:56Z

inkscape_extensions/0.x/tag_linespec.py

-#import xml.etree.ElementTree as ET
-#ET.register_namespace('figurefirst', 'http://www.figurefirst.com')
+
+sys.path.append("/usr/share/inkscape/extensions")  # or another path, as necessary


These could probably be behind if statements to detect the platform.

Yeah I agree - but since this is mostly some CI improvements I would rather handle this in another PR. I've opened up an issue #60.

psilentp added 17 commits May 29, 2020 15:47

refractor: remove redundancy in ci workflow

db82a05

feat: dev and doc requirments

d13be0c

doc: project badges

19a1fa0

fix: licence

8b24959

chore: update test workflow name

334a510

chore: workflow name

ce0394b

chore: test badge on master

cba8184

chore: add black check to tox

0684f00

fix: gh-actions syntax

84da09b

fix: gh-actions syntax

f9bbbcd

fix: gh-actions syntax

f13917f

chore: add security check workflow

fa43cfd

chore: fix missing reference in security check

fb082b3

chore: using marketplace for soft failing checks

46ff340

chore: remove artifact upload since there is no github ui to access

bb2d3d9

chore: remove line comment

7d494e6

chore: publish workflow

c982ece

psilentp added 4 commits June 5, 2020 13:01

fix:added ordered dict shim to svg_to_axes

0e1e702

chore:remove test-pypy step

50e9fce

version bump: patch

4acd4b8

Revert "fix:added ordered dict shim to svg_to_axes"

ed31fc6

This reverts commit 0e1e702.

clbarnes reviewed Jun 6, 2020

View reviewed changes

psilentp added 4 commits June 10, 2020 16:53

Revert "chore:remove test-pypy step"

e6011b1

This reverts commit 50e9fce.

chore: moved deploy to ci workflow

7fc7217

chore: switched to setuptools_scm for versionning

79a9623

style: black

e1e3ab3

chore: updated local version for test-pypi

771b717

psilentp requested review from clbarnes and florisvb June 11, 2020 18:22

psilentp changed the title ~~[WIP] CI polish to target 0.0.7 release~~ CI polish to target 0.0.7 release Jun 11, 2020

clbarnes reviewed Jun 12, 2020

View reviewed changes

psilentp added 9 commits June 19, 2020 23:16

style: fixed unused import warning

d2e35f2

style: ran black on non-package python files

42d4f59

chore: added flake8 config

d69a8d9

chore: added pyprojec.toml

6549767

fix: FigureLayout import in __init__

ac532ba

style:black

7e93851

doc:readme

1132c85

fix:removed wheel

478c726

Revert "fix:removed wheel"

5058da3

This reverts commit 478c726.

clbarnes reviewed Jun 20, 2020

View reviewed changes

psilentp added 3 commits June 20, 2020 21:29

chore: format dev versions for test pypi

6858565

chore: return to empty dev version string

fa6c2fc

chore: continue on error true for test-pypi

2cc87a5

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

CI polish to target 0.0.7 release #58

CI polish to target 0.0.7 release #58

psilentp commented Jun 5, 2020 •

edited

Loading

clbarnes commented Jun 5, 2020

psilentp commented Jun 5, 2020

clbarnes commented Jun 5, 2020

psilentp commented Jun 5, 2020

clbarnes Jun 6, 2020

psilentp Jun 11, 2020

clbarnes Jun 6, 2020

psilentp Jun 11, 2020

clbarnes left a comment

clbarnes Jun 20, 2020

psilentp Jun 21, 2020

clbarnes Jun 20, 2020

psilentp Jun 21, 2020 •

edited

Loading

clbarnes Jun 20, 2020

psilentp Jun 21, 2020 •

edited

Loading

CI polish to target 0.0.7 release #58

Are you sure you want to change the base?

CI polish to target 0.0.7 release #58

Conversation

psilentp commented Jun 5, 2020 • edited Loading

clbarnes commented Jun 5, 2020

psilentp commented Jun 5, 2020

clbarnes commented Jun 5, 2020

psilentp commented Jun 5, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

clbarnes left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

psilentp Jun 21, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

psilentp Jun 21, 2020 • edited Loading

Choose a reason for hiding this comment

psilentp commented Jun 5, 2020 •

edited

Loading

psilentp Jun 21, 2020 •

edited

Loading

psilentp Jun 21, 2020 •

edited

Loading