-
Notifications
You must be signed in to change notification settings - Fork 28
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Port OpenSSL backend to go1.19 #576
Conversation
42cbfd6
to
4a19d3a
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Changes LGTM, just a couple comments about the patch arrangement.
--- a/src/crypto/internal/boring/cache.go | ||
+++ /dev/null |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hmm, when I see this, my first thought is that it might belong in the first patch so it's clearer that this is part of a file move. I think Git would also count that as a rename so it would also make the diff clearer (the package name change).
But I remember the idea here is that first patch is add-only (avoiding conflicts), making this deletion part of integration.
Maintenance-wise, I think this might cause a problem down the line. If upstream adds a fix to this file, we'll hit a conflict in this patch, and whoever's resolving it needs to know to not only resolve the conflict in this patch, but also go back to the first patch and redo copying src/crypto/internal/boring/cache.go
to src/crypto/internal/backend/bcache/cache.go
. It seems like it might be easy to miss and end up not getting the upstream fix in our build.
I think a reasonable middle ground is to split the deletion into a "cleanup" patch file (before or after the integration patch) with a clear description that conflicts need to ported back to the first patch. We don't have a mechanism for it yet, but we could also add a patch test in the outer repo that makes sure the copy is always the same (other than package name).
But actually, I think it might be best to put both the add and remove into the same patch file before even adding the OpenSSL crypto module. Basically, design the patch so that we could even submit it to upstream if they're willing to help us out with https://groups.google.com/g/golang-dev/c/h6iFPvOnPkg. (Maybe it's not quite that simple--the dir structure we want might be different than what upstream would take?) Then, we can simply zap the patch file if upstream merges the change. In the meantime, merge conflicts should be easy to resolve.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I like the idea of putting this change into its own file. It will also be clearer which refinements we are doing to existing boring package for the sake of reusability.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done! Git does count it as a rename, so we saved almost 500 lines of diffs and future headaches if the original files we changed 😄
Subject: [PATCH] Vendor OpenSSL crypto library | ||
|
||
--- |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should the command from the old summary get copied over here?
go/patches/0102-Vendor-OpenSSL-crypto-library.patch
Lines 4 to 7 in 669b2bd
Subject: [PATCH] Vendor OpenSSL crypto library | |
To reproduce, run 'go mod vendor' in 'go/src'. | |
--- |
More generally about these summaries: I don't think we need to be concerned about including big descriptions for each patch in the initial PR. But, I think we should probably plan to include more in the future, especially as we start to collaborate with other Go distro maintainers. It seems to me like the natural place for people to look to start understanding what we've done.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done! Agree that we should have a small description in each patch, but let's do it in another PR :)
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001 | ||
From: Quim Muntal <[email protected]> | ||
Date: Mon, 23 May 2022 12:59:54 +0000 | ||
Subject: [PATCH] Adjust Go tests to work with crypto module |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I noticed some changes from https://github.com/microsoft/go/blob/microsoft/dev.boringcrypto.go1.18/patches/0103-Adjust-Go-tests-to-work-with-crypto-module.patch aren't in here. If they're not necessary anymore, that's great, but in case the issues come up in CI, figured I should point it out. 😄
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good catch, I wanted to run the CI without those changes just in case they are not needed anymore, but I'll probably end up adding them 😋
4a19d3a
to
3ae175a
Compare
3ae175a
to
77c268a
Compare
This PR ports the OpenSSL backend from
dev.boringbranch
intomain
following the same generic backend strategy as in previous Go versions.Upstream Go did several refactors to how BoringCrypto was being integrated into
crypto
. Our port can leverage almost all work done upstream, except for the following issues:crypto/internal/boring
, and we can't depend on it, so I've copy-pasted the relevant files intocrypto/internal/backend/bcache
and updated the runtime package to back this new package. See https://groups.google.com/g/golang-dev/c/h6iFPvOnPkg for more context.crypto/internal/boring
API is now free ofbig.Int
. It insteadcrypto/internal/boring.BigInt
and a helper package to cast frombig.Int
toboring.BigInt
located atcrypto/internal/boring/bbig
. As we can't depend oncrypto/internal/boring.BigInt
, I've created a bidge package atcrypto/internal/backend/bbig
which implements the casts to the rightBigInt
type:goexperiment.boringcrypto
is set, ituses crypto/internal/boring/bbig
.goexperiment.opensslcrypto
is set, itjackfan.us.kg/microsoft/go-crypto-openssl/openssl/bbig
.See Remove math/big and encoding/asn1 deps from openssl package go-crypto-openssl#26 for more context.
TestAllocations
test fromcrypto/ed25519
has been disabled whengoexperiment.opensslcrypto
is set.boring.Enabled
is not known at compile time when using OpenSSL backend, as it can be encabled/disabled at runtime, so there are some optimizations that the compiler skip, which make us allocate more even ifboring.Enabled==false