ENH: Switch packaging infra to Poetry

[andersbogsnes]

Is your feature request related to a problem? Please describe.
The setup.py model is not as clear as I would like - Poetry is based on pyproject.toml and is a much simpler, declarative solution. Poetry also support dev dependencies

Describe the solution you'd like
Remove setup.py and use poetry for package management instead

Describe alternatives you've considered
Pipenv was discarded due to several issues stemming from lack of maintenance. Flit is another option, but poetry is more full-featured

Additional context
Add any other context or screenshots about the feature request here.