Update version of PyYAML to 6.x.x, dropping support for >=4.2, <6.0

[datajoely]

Description

Is your feature request related to a problem? A clear and concise description of what the problem is: "I'm always frustrated when ..."

Some of you may have seen the very exciting news that the Anaconda folks have a whole Python stack running through the browser via PyScript. This works via another super important library called Pyodide which is essentially a CPython implementation on Web Assembly.

This is what makes JupyterLite possible if you click here you can run a full Jupyter Notebook and Pyodide kernel in the browser, with no server required.

Context

Why is this change important to you? How would you use it? How can it benefit other users?

The value of us making Kedro compatible is obvious, I'm actually most excited about being able to do tutorials and demos with zero infra or cloud requirements. Things like binder are cool, but take ages to spin up the containers on the free plan.

Implementation

So you can install packages into pyodide using a library called micropip. It also includes most of the libraries Kedro needs to do a pythonic (pandas + sklearn) pipeline.

When I try to install kedro today I get this error because the PyYAML requirement we have today still supports Python 2.7 which was dropped as part of the 6.0.0 release in October 2021.

I've actually created a branch to see if Kedro can move to this new version of PyYAML and it looks like the unite tests all pass ✅ , but e2e fail ❌ because kedro-telemetry has the same PyYAML requirement (kedro-docker has something similar in test requirements).

---

As far as I can tell that bumping the PyYAML version has no impact on Kedro, it drops 2.7 support and is thus a breaking change on their side but that's long been dropped on Kedro's side. The only annoying piece is that this requires a whole release for kedro-telemetry.

[antonymilne]

This is really cool, thanks for looking into it! Will there be other bumps needed as well to get the micropip install to work or is it just pyyaml?

I've looked through all the uses of pyyaml in kedro and agree that pyyaml 6.0 shouldn't affect anything. The only one that looked like it might need attention is this change, but since we're using yaml.safe_load everywhere already I don't think we need to do anything.

I don't see why we should drop support for >=4.2, <6.0 though? Why not just extend the range to >=4.2, <7.0? Otherwise we're pinning quite a recent version of pyyaml which isn't necessarily compatible with other packages.

As far as I'm concerned, I'm happy to accept a PR that changes to pyyaml>=4.2, <7.0 everywhere we have it as a requirement. We might as well just remove the requirement from kedro-telemetry (and any other plugins) entirely since it's already a core kedro requirement.

[datajoely]

Awesome - I'll submit PRs to bump the upper bound like you suggest.

Also - I'm 99% sure PyYAML is the only dependency holding 0.18.x back since if I try and install 0.17.7 I get a longer list of incompatibilities:

[datajoely]

Will close this when 0.18.2 releases

[datajoely]

I can confirm this now works! We can do a full demo of Kedro without a server 💪

[datajoely]

Running on https://jupyterlite.github.io/demo/lab/index.html

[stichbury]

@datajoely Would you be able to walk me through this with a basic example please ? I'd like to maybe record it and use it to create some new docs and/or edit spaceflights accordingly. Please LMK and I'll book a short slot if I may.

[datajoely]

Absolutely - I'll message you directly