Add documentation to the website about the Rust dependency

[kevinburke1]

Reading through the docs on the website, there's an explanation that the cryptography package depends on Rust, and how to install Rust, but there's not really an explanation for how or why Rust is used.

I know that there are very good reasons to use Rust, and to try to avoid memory-unsafe languages for cryptographic code, and I support its widespread adoption. But it might help to have a better explanation on the website of why that's happening.

[reaperhulk]

This is a good idea!

Right now we don't use Rust for anything, since we wanted it to be capable of being disabled via env var for the first release for people who needed time to update their build environments. However, we'll be moving our padding code to it first (#5668) followed by an increasing surface of ASN.1 parsing. There is no current intent to move away from OpenSSL for cryptographic operations. This is for several reasons, but among the foremost is OpenSSL's status as the "cryptographic library of record" across the industry. Moving away would cause problems in many places with regard to compliance, and multiple distributions rely on our usage of OpenSSL to assert things like Common Criteria or FIPS. So we should figure out a way to document our use of (and general future intent for) Rust, while also making it clear that we do not intend to abandon OpenSSL.

As an aside, we have a concept of "backends" in cryptography that has been so underutilized that we began deprecation of it. Pure rust cryptographic providers could, however, come in through that door while keeping OpenSSL as an available backend for the consumers who need what it provides. That is strictly hypothetical at this point though and thus far in the 7 years cryptography has existed no one has ever written an alternate backend other than me.

[jephthai]

For those of us who don't particularly want to bring another entire programming language ecosystem into our projects, is there some workable strategy going forward to avoid this Rust dependency?

Is there an alternative implementation in the python world that you would recommend that isn't going this direction? All my program wants to do is some RSA signatures, and I don't want to make all my users install Rust just for that.

[alex]

You're asking the wrong folks :-) We think this library is pretty good (well, we basically have to, we built it!), if we thought some other Python crypto library was better, we'd probably just stop maintaining this. We also think memory safety is critically important, and shouldn't be optional.

We've hard to make this easy to install via our wheels on all major platforms (except M1 macOS, due to lack of CI). If you still don't want to use it, we're sorry to see you go, but you'll have to find your own library to replace us with.

[jephthai]

Thanks for your reply, I totally get your goals in this. I'm not strictly speaking opposed to the idea, there are some compelling advantages with Rust. I only discovered this issue because pyopenssl specifies pyca/cryptography as a dependency, and now have to work around it.

[tiran]

Is there an alternative implementation in the python world that you would recommend that isn't going this direction? All my program wants to do is some RSA signatures, and I don't want to make all my users install Rust just for that.

Rust is only needed at build time. It's not a runtime dependency. If you distribute binaries or use our binary wheels, your users don't need any Rust components at all.

To give you an additional data point for your decision process.
I'm a SE in Red Hat's security engineering department. We have studied many Python crypto libraries. PyCA cryptography is the recommend Python crypto library in Fedora. In RHEL we neither support nor permit any other Python crypto library other than PyCA cryptography and Python's builtin ssl, hashlib, and hmac modules.

[alex]

If you'd like to please contribute constructively you may. Otherwise you'll be banned. Consider this your only warning.

(This and the reply below were in response to a post that has since been deleted, not to any comment remaining in this thread)

[kevinburke1]

That language is completely unacceptable and you should wash your mouth out with soap.

Further for anyone else reading this is not a catchall ticket to complain about the decision to adopt Rust. Open a separate ticket with constructive feedback if you are upset about that.

[edrozenberg]

Rust is only needed at build time. It's not a runtime dependency. If you distribute binaries or use our binary wheels, your users don't need any Rust components at all.

That's good to hear. I was also rudely surprised by now having to maintain yet more packages (setuptools-rust and python-semanticversion). The dozens and dozens of python pre-reqs are mushrooming ever onward. I don't use wheels - packaging each and every req.

At least the rust req doesn't have to be present and the rust pig doesn't have to load and run on any target servers. If it provides some additional mem safety that's a +.

The python ecosystem is at least still better than the node one where devs blindly install hundreds of reqs (at least 10% of which are guaranteed malware it seems).

[hangingman]

@tiran

Rust is only needed at build time. It's not a runtime dependency. If you distribute binaries or use our binary wheels, your users don't need any Rust components at all.

If it's true, I think it's better to move setuptools_rust dependency into extras_require section.
I think this prevent users to requires to install Rust dependencies.

cryptography/setup.py

Line 46 in dd09d50

setup_requirements = install_requirements + [setuptools_rust]

I'm sorry if I misunderstand.

[alex]

extra_requires would cause it to only be installed when the user specified that extra, which is not the same thing as being a build dependency. It's listed in setup_requires, which has the correct effect.

…

On Wed, Feb 24, 2021 at 2:44 AM hangedman ***@***.***> wrote: @tiran <https://github.com/tiran> Rust is only needed at build time. It's not a runtime dependency. If you distribute binaries or use our binary wheels, your users don't need any Rust components at all. If it's true, I think it's better to move setuptools_rust dependency into extras_require section. I think this prevent users to requires to install Rust dependencies. https://github.com/pyca/cryptography/blob/dd09d500f9187df9f620aa7c984d01738cb60b11/setup.py#L46 I'm sorry if I misunderstand. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#5810 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAAGBAL57H4E76ZBUVHDQ3TASU7PANCNFSM4XNWKZDA> .

-- All that is necessary for evil to succeed is for good people to do nothing.

[tiran]

The terms can be confusing. install_requires are run-time dependencies. setup_requires are build-time dependencies. Since commit 048f7c6, setuptools_rust is only installed when your build cryptography from sources. Binary wheel installtion only pulls in cffi.

[theLastOfCats]

@reaperhulk didn't you consider using ring and/or rustls

[jamshid]

I wish cryptography would document that an easy way to avoid the rust requirement, while hopefully not introducing any security concerns, is simply to pin to cryptography<3.4 -- the 3.3.x versions did not require the rust compiler. You can pip3 install cryptography<3.4 before installing the package that requires cryptography and avoid the rust requirements.

I wish cryptography maintainers understood that a lot of users are on centos:7 with yum install python3-pip installing v9.0.3. That is python 3.6 btw, afaik python 3.8 is not available for centos:7 (yet?).
That old version of pip3 will not download the pre-built binaries needed by cryptography. Users could pip3 install --upgrade pip so that it's able to download the binaries, avoiding needing rustc, but that is a bit dangerous since it modifies the pip3 system package.

[reaperhulk]

As always, it is strongly advised that users should pin their dependencies because that is best practice in the entire ecosystem. Users may choose their own risk profiles with regard to not upgrading dependencies, but this project only provides support for the latest versions and we're not willing to recommend things that (depending upon usage) be unsafe.

Mixing OS provided packages and PyPI packages is a choice users make. If they do not wish to upgrade pip (or simply install a newer one in a virtualenv and run their application out of that), then they have made choices and those choices have consequences (e.g. they have to compile the project themselves). Alternately, they could simply stick with the OS's packages entirely rather than attempting to mix and match, which brings its own dangers.

The software ecosystem is imperfect and there are finite resources available. I understand that you feel that lashing out at volunteers on a public issue tracker is necessary because it is likely your belief that we simply do not understand the challenges faced by you and others like you. However, it isn't productive, and we really do understand the challenges. We have provided immense amounts of support in this migration, including methods to disable the rust requirement in 3.4.x releases, new versioning schemes (next release), and helping a variety of packagers find support for bootstrapping Rust in their worlds. But the next release will require Rust, and we aren't going to turn back from that path.

I am going to lock this issue to collaborators due to it being an attractive nuisance, but the documentation improvements we discussed early in the issue are still needed so it will remain open.