[question] usability of linux binaries on conan center

[rconde01]

I was using the xerces/3.2.2 package from conan center on redhat 7 using devtoolset-8 and getting linker errors related to ABI/GLIBC etc. After I rebuilt the binaries locally, it worked. I understand the issue here...but i'm wondering at the conan center strategy. Are the usability of conan center linux binaries a dice roll? If so, is there any plan to handle this better? If not, is there a way to set a policy to always build locally for a given platform?

• I've read the CONTRIBUTING guide.

[jgsogo]

Hi, @rconde01!

Yes, the glibc version is a problem when trying to distribute already compiled binaries. Right now, in Conan Center we are using different Linux distributions to generate the binaries depending on compiler version, but we will try to improve it. We are going to generate all the binaries using the same ubuntu distribution and, at least, all the binaries will use the same glibc (no dice roll). We already have the docker images prepared (thanks to @uilianries ) and now we have to modify the infrastructure in conan-center-index and migrate all the binaries (it will take some time). We will document it, and it should be easy to access information.

This will alleviate some problems we have right now, but I know this is not fixing the issue. We could add glibc version to the Conan model, and then packages will have different package-ID, but it will increase dramatically the number of binaries we are already generating and it is something we can't afford (as a user, you can add it to the settings.yml model, but it cannot be the default).

Your suggestion could work, but we cannot identify the platform/glibc and decide that binaries coming from a remote have to be compiled (even though binaries with exactly the same package-ID are available). A remote is just an URL for Conan, the user can add many remotes to the client, and we don't want to take special actions based on the URL, it'd require to hardcode things that shouldn't be hardcoded in a generic client.

...but we could add a trait for the remotes, so the user can decide if everything coming from a remote should be compiled from sources. It would work, it'd probably be easy to implement, maybe it is a bit overkill, but maybe it is the only solution so far.

[jgsogo]

Related to this one #7117

[rconde01]

Thanks for the background. I definitely get that you can't host binaries for all glibc versions...but why can't add it to settings and just host for one version (i guess whatever is in that ubuntu distro)?

[jgsogo]

We could add glibc to the settings in a way that it is taken into account for the packageID, but then people starting to use Conan won't have binaries available in ConanCenter (they will likely use a different glibc, their Conan client will require a different packageID) and this is something we really want to provide: getting started should be as smooth as possible.

So the plan is to use the oldest supported Ubuntu image to generate the binaries, so it will use a relatively old glibc and it will work for most of the users.

[rconde01]

Gotcha - I guess the success will depend on the Ubuntu LTS strategy which i'm not very familiar with...I would assume the commercial distos are more conservative but I don't know for sure.

[jgsogo]

Yes. We will use Xenial: conan-io/conan-docker-tools#204 @uilianries is preparing a blogpost to explain about this migration, maybe it makes sense to talk about what the user can do to take into account glibc version.

[memsharded]

Thanks for the background. I definitely get that you can't host binaries for all glibc versions...but why can't add it to settings and just host for one version (i guess whatever is in that ubuntu distro)?

The problem with modeling the glibc is that the glibc doesn't need to have an exact match (like most of the other settings), but older glibc are compatible. We could implement this as a compatible_package fall back for all glibc versions, in descending order, in all recipes in ConanCenter, but that will make dependency resolution like one order of magnitude slower.

Building with a relatively older distro will be a pragmatic approach that will probably be good for most of the ConanCenter users.

[ohanar]

No matter what choice of distro/glibc version is chosen for conan center, there will always be someone who has different needs.

If there is no desire to fix the underlying issue of glibc compatibility (due to package resolution performance concerns), how about adding an allow list and/or deny list for binaries? That way, if someone knows that they can't use the binaries on conan center, for whatever reason, they can simply block it. Put another way, rather than simply having a remote being enabled or disabled, it is enabled/source only/disabled.

[jgsogo]

I like that enabled/disabled/source-only flag for the remotes. wdyt, @memsharded ?

[memsharded]

This kind of usage of ConanCenter, specially in production environments is documented in https://docs.conan.io/2/devops/using_conancenter.html, and recommends using recipes from conan-center-index Github repo sources, so binaries from ConanCenter shouldn't be an issue.