ModuleNotFoundError: No module named 'markupsafe' error at build time

[DnPlas]

Bug Description

Charms that list jinja2 as a requirement could result in a build failure with the following message:

::    :: Collecting jinja2==3.1.2
::    ::   Using cached Jinja2-3.1.2.tar.gz (268 kB)
::    ::     ERROR: Command errored out with exit status 1:
...
::    ::       File "/tmp/pip-install-jf2rwxln/jinja2/src/jinja2/environment.py", line 14, in <module>
::    ::         from markupsafe import Markup
::    ::     ModuleNotFoundError: No module named 'markupsafe'
::    ::     ----------------------------------------

The issue is due to pallets/jinja#1496 and canonical/charmcraft#1664.

Potential fixes

1. Use ubuntu 22.04 as base for build-on since this OS version will have a newer version of setuptools
2. List the packages that depend on jinja2 in the charm-binary-python-packages to avoid building them from source. For example charm-binary-python-packages: [charmed-kubeflow-chisme]
3. If jinja2 is a direct dependency of a charm, use the above method

To Reproduce

Try building any of the packages that list jinja2 in the requirements file:

1. Any of the kfp-operators
2. istio-pilot

Environment

• Ubuntu 20.04
• Charmcraft latest/candidate (2.6.0)

Relevant Log Output

Logs: https://pastebin.canonical.com/p/DhpBRNn7pz/

Workaround

Install jinja2 from binary as we do in canonical/kfp-operators#443

[syncronize-issues-to-jira]

Thank you for reporting us your feedback!

The internal ticket has been created: https://warthogs.atlassian.net/browse/KF-5602.

This message was autogenerated

[ca-scribner]

I see istio-pilot listed here, but I don't see the CI that is breaking. Is istio-pilot blocked by this too?

[DnPlas]

I have seen it intermittently in istio-operator's CI: here is an example of this.

[ca-scribner]

of course it has to be intermittent ... :/

[orfeas-k]

I see that in katib-controller as well, although the charm is packed after all and running.

[misohu]

The resolution is to bump markupsafe from version 2.1.3 to version 2.1.5. This issue is happening in multiple charms. You can update the lib:

1. Update jinja2 if you have it in all requirements

for pattern in "requirements.in" "requirements-fmt.in" "requirements*.in"; do find . -type f -name "$pattern" -exec bash -c "cd \$(dirname "{}") && pip-compile \$(basename "{}") -P jinja2 --upgrade" \;; done

2. Update charmed-kubeflow-chisme if you don't have jinja2 it in all requirements

for pattern in "requirements.in" "requirements-fmt.in" "requirements*.in"; do find . -type f -name "$pattern" -exec bash -c "cd \$(dirname "{}") && pip-compile \$(basename "{}") -P charmed-kubeflow-chisme --upgrade" \;; done

[DnPlas]

We need to backport this to 1.8 related track branches.

All 1.8 related github branches have been updated with the proposed changes. Feel free to re-open if anything is missing or if the issue is still present.