Skip to content

Commit

Permalink
Update third-party dependencies to remove CentOS 7
Browse files Browse the repository at this point in the history
Summary:
CentOS 7 has reached its EOL and has been removed from the third-party archive build matrix. Use Amazon Linux 2 instead of CentOS 7 as the "default" OS for third-party dependencies during a YugabyteDB build. With the removal of CentOS 7, Amazon Linux 2 currently has the oldest supported glibc version (2.26) of any Linux distribution that we use in our build matrix, so third-party pre-built archives built with Amazon Linux 2 can be used for a non-sanitizer YugabyteDB build with any supported Linux distribution.

Also the new third-party version enables Clang 18 (not yet supported in the YugabyteDB build itself).

Test Plan: Jenkins

Reviewers: steve.varnau

Reviewed By: steve.varnau

Subscribers: ybase

Differential Revision: https://phorge.dev.yugabyte.com/D36673
  • Loading branch information
mbautin committed Jul 18, 2024
1 parent 9f82c01 commit 1522522
Show file tree
Hide file tree
Showing 2 changed files with 50 additions and 36 deletions.
71 changes: 41 additions & 30 deletions build-support/thirdparty_archives.yml
Original file line number Diff line number Diff line change
@@ -1,90 +1,101 @@
sha: 31776c4936a67fe6f1fc218fb64a6a9909c77311
sha: b6b07342fdfd4a65ee2608d75dd31e4b0ecc0737
archives:

- os_type: almalinux8
architecture: x86_64
compiler_type: clang17
tag: v20240620163555-31776c4936-almalinux8-x86_64-clang17
tag: v20240713003527-b6b07342fd-almalinux8-x86_64-clang17

- os_type: almalinux8
architecture: x86_64
compiler_type: clang18
tag: v20240713003521-b6b07342fd-almalinux8-x86_64-clang18

- os_type: almalinux8
architecture: x86_64
compiler_type: gcc11
tag: v20240620163541-31776c4936-almalinux8-x86_64-gcc11
tag: v20240713003520-b6b07342fd-almalinux8-x86_64-gcc11

- os_type: almalinux9
architecture: x86_64
compiler_type: clang17
tag: v20240620163555-31776c4936-almalinux9-x86_64-clang17
tag: v20240713003540-b6b07342fd-almalinux9-x86_64-clang17

- os_type: almalinux9
architecture: x86_64
compiler_type: gcc12
tag: v20240620163623-31776c4936-almalinux9-x86_64-gcc12
tag: v20240713003537-b6b07342fd-almalinux9-x86_64-gcc12

- os_type: centos7
- os_type: amzn2
architecture: aarch64
compiler_type: clang16
tag: v20240620163739-31776c4936-centos7-aarch64-clang16
compiler_type: clang17
tag: v20240713003725-b6b07342fd-amzn2-aarch64-clang17

- os_type: centos7
- os_type: amzn2
architecture: aarch64
compiler_type: clang16
compiler_type: clang17
lto_type: full
tag: v20240620163735-31776c4936-centos7-aarch64-clang16-full-lto
tag: v20240713003827-b6b07342fd-amzn2-aarch64-clang17-full-lto

- os_type: centos7
- os_type: amzn2
architecture: aarch64
compiler_type: clang17
tag: v20240620163738-31776c4936-centos7-aarch64-clang17
compiler_type: clang18
tag: v20240713003831-b6b07342fd-amzn2-aarch64-clang18

- os_type: centos7
- os_type: amzn2
architecture: aarch64
compiler_type: clang17
compiler_type: clang18
lto_type: full
tag: v20240620163737-31776c4936-centos7-aarch64-clang17-full-lto
tag: v20240713003853-b6b07342fd-amzn2-aarch64-clang18-full-lto

- os_type: centos7
- os_type: amzn2
architecture: x86_64
compiler_type: clang17
tag: v20240620163551-31776c4936-centos7-x86_64-clang17
tag: v20240713003538-b6b07342fd-amzn2-x86_64-clang17

- os_type: centos7
- os_type: amzn2
architecture: x86_64
compiler_type: clang17
lto_type: full
tag: v20240620163542-31776c4936-centos7-x86_64-clang17-full-lto
tag: v20240713003542-b6b07342fd-amzn2-x86_64-clang17-full-lto

- os_type: centos7
- os_type: amzn2
architecture: x86_64
compiler_type: gcc11
tag: v20240620163544-31776c4936-centos7-x86_64-gcc11
compiler_type: clang18
tag: v20240713003544-b6b07342fd-amzn2-x86_64-clang18

- os_type: amzn2
architecture: x86_64
compiler_type: clang18
lto_type: full
tag: v20240713003540-b6b07342fd-amzn2-x86_64-clang18-full-lto

- os_type: macos
architecture: arm64
compiler_type: clang
tag: v20240620173126-31776c4936-macos-arm64
tag: v20240713011052-b6b07342fd-macos-arm64

- os_type: macos
architecture: x86_64
compiler_type: clang
tag: v20240620163640-31776c4936-macos-x86_64
tag: v20240713003540-b6b07342fd-macos-x86_64

- os_type: ubuntu20.04
architecture: x86_64
compiler_type: clang16
tag: v20240620163547-31776c4936-ubuntu2004-x86_64-clang16
tag: v20240713003520-b6b07342fd-ubuntu2004-x86_64-clang16

- os_type: ubuntu22.04
architecture: x86_64
compiler_type: clang17
tag: v20240620163538-31776c4936-ubuntu2204-x86_64-clang17
tag: v20240713003517-b6b07342fd-ubuntu2204-x86_64-clang17

- os_type: ubuntu22.04
architecture: x86_64
compiler_type: gcc11
tag: v20240620163535-31776c4936-ubuntu2204-x86_64-gcc11
tag: v20240713003527-b6b07342fd-ubuntu2204-x86_64-gcc11

- os_type: ubuntu23.04
architecture: x86_64
compiler_type: gcc13
tag: v20240620163558-31776c4936-ubuntu2304-x86_64-gcc13
tag: v20240713003516-b6b07342fd-ubuntu2304-x86_64-gcc13
15 changes: 9 additions & 6 deletions python/yugabyte/thirdparty_tool.py
Original file line number Diff line number Diff line change
Expand Up @@ -69,7 +69,9 @@
# These were incorrectly used without the "clang" prefix to indicate various versions of Clang.
NUMBER_ONLY_VERSIONS_OF_CLANG = [str(i) for i in [12, 13, 14]]

PREFERRED_OS_TYPE = 'centos7'
# Linux distribution with the oldest glibc available to us. Third-party archives built on this
# OS can be used on newer Linux distributions, unless we need ASAN/TSAN.
PREFERRED_OS_TYPE = 'amzn2'


ThirdPartyArchivesYAML = Dict[str, Union[str, List[Dict[str, str]]]]
Expand Down Expand Up @@ -272,13 +274,14 @@ def is_consistent_with_yb_version(self, yb_version: str) -> bool:

def should_skip_as_too_os_specific(self) -> bool:
"""
Certain build types of specific OSes could be skipped because we can use the CentOS 7 build
instead. We can do that in cases we know we don't need to run ASAN/TSAN. We know that we
don't use ASAN/TSAN on aarch64 or for LTO builds as of 11/07/2022. Also we don't skip
Linuxbrew builds or GCC builds.
Certain build types of specific OSes could be skipped because we can use our "preferred OS
type", the supported Linux distribution with the oldest glibc version, instead. We can do
that in cases we know we don't need to run ASAN/TSAN. We know that we don't use ASAN/TSAN
on aarch64 or for LTO builds as of 11/07/2022. Also we don't skip Linuxbrew builds or GCC
builds.
"""
return (
self.os_type != 'centos7' and
self.os_type != PREFERRED_OS_TYPE and
self.compiler_type.startswith('clang') and
# We handle Linuxbrew builds in a special way, e.g. they could be built on AlmaLinux 8.
not self.is_linuxbrew and
Expand Down

0 comments on commit 1522522

Please sign in to comment.