Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Docker packaging tests break the matrix. #47639

Closed
alpar-t opened this issue Oct 7, 2019 · 5 comments · Fixed by #47640
Closed

Docker packaging tests break the matrix. #47639

alpar-t opened this issue Oct 7, 2019 · 5 comments · Fixed by #47640
Assignees
@pugnascotia
Labels
:Delivery/Packaging RPM and deb packaging, tar and zip archives, shell and batch scripts Team:Delivery Meta label for Delivery team >test-failure Triaged test failures from CI v8.0.0-alpha1

Comments

@alpar-t
Copy link
Contributor

alpar-t commented Oct 7, 2019

The docker tests added in #46599 fail packaging tests:

https://elasticsearch-ci.elastic.co/job/elastic+elasticsearch+master+multijob+packaging-tests-unix-next/os=ubuntu-16.04-packaging/13/console

The docker networking issue is one that we can easily deal with but these tests also have to account for the fact that some platforms ( like centos 6 ) don't have docker installed.
@alpar-t alpar-t added :Delivery/Packaging RPM and deb packaging, tar and zip archives, shell and batch scripts >test-failure Triaged test failures from CI v8.0.0 labels Oct 7, 2019
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-core-infra (:Core/Infra/Packaging)

@alpar-t
Copy link
Contributor Author

alpar-t commented Oct 7, 2019

Here's a failure for missing docker: https://elasticsearch-ci.elastic.co/job/elastic+elasticsearch+master+multijob+packaging-tests-unix-next/os=oraclelinux-6-packaging/13/console

@alpar-t
Copy link
Contributor Author

alpar-t commented Oct 7, 2019

The method used to install docker in Vagrant doesn't work on the GCP images:

[alpar@alpar-ci-centos-6-1007-0936 ~]$     # Add repository
[alpar@alpar-ci-centos-6-1007-0936 ~]$     sudo yum-config-manager -y --add-repo https://download.docker.com/linux/centos/docker-ce.repo
Loaded plugins: fastestmirror
adding repo from: https://download.docker.com/linux/centos/docker-ce.repo
grabbing file https://download.docker.com/linux/centos/docker-ce.repo to /etc/yum.repos.d/docker-ce.repo
docker-ce.repo                                                                                                                                                                                                                                           | 2.4 kB     00:00     
repo saved to /etc/yum.repos.d/docker-ce.repo
[alpar@alpar-ci-centos-6-1007-0936 ~]$ 
[alpar@alpar-ci-centos-6-1007-0936 ~]$     # Install Docker
[alpar@alpar-ci-centos-6-1007-0936 ~]$     sudo yum install -y docker-ce docker-ce-cli containerd.io
Loaded plugins: fastestmirror, security
Setting up Install Process
Loading mirror speeds from cached hostfile
 * base: mirror.mobap.edu
 * centos-sclo-rh: mirror.fileplanet.com
 * centos-sclo-sclo: bay.uchicago.edu
 * epel: fedora-epel.mirror.lstn.net
 * extras: mirror.rackspace.com
 * updates: mirror.genesisadaptive.com
docker-ce-stable                                                                                                                                                                                                                                         | 3.5 kB     00:00     
docker-ce-stable/primary_db                                                                                                                                                                                                                              |  34 kB     00:00     
Resolving Dependencies
--> Running transaction check
---> Package containerd.io.x86_64 0:1.2.6-3.3.el7 will be installed
--> Processing Dependency: container-selinux >= 2:2.74 for package: containerd.io-1.2.6-3.3.el7.x86_64
--> Processing Dependency: systemd for package: containerd.io-1.2.6-3.3.el7.x86_64
--> Processing Dependency: systemd for package: containerd.io-1.2.6-3.3.el7.x86_64
---> Package docker-ce.x86_64 3:19.03.2-3.el7 will be installed
--> Processing Dependency: container-selinux >= 2:2.74 for package: 3:docker-ce-19.03.2-3.el7.x86_64
--> Processing Dependency: libseccomp >= 2.3 for package: 3:docker-ce-19.03.2-3.el7.x86_64
--> Processing Dependency: libsystemd.so.0(LIBSYSTEMD_209)(64bit) for package: 3:docker-ce-19.03.2-3.el7.x86_64
--> Processing Dependency: systemd for package: 3:docker-ce-19.03.2-3.el7.x86_64
--> Processing Dependency: libseccomp.so.2()(64bit) for package: 3:docker-ce-19.03.2-3.el7.x86_64
--> Processing Dependency: libsystemd.so.0()(64bit) for package: 3:docker-ce-19.03.2-3.el7.x86_64
---> Package docker-ce-cli.x86_64 1:19.03.2-3.el7 will be installed
--> Finished Dependency Resolution
Error: Package: containerd.io-1.2.6-3.3.el7.x86_64 (docker-ce-stable)
           Requires: container-selinux >= 2:2.74
Error: Package: 3:docker-ce-19.03.2-3.el7.x86_64 (docker-ce-stable)
           Requires: libsystemd.so.0()(64bit)
Error: Package: 3:docker-ce-19.03.2-3.el7.x86_64 (docker-ce-stable)
           Requires: libsystemd.so.0(LIBSYSTEMD_209)(64bit)
Error: Package: 3:docker-ce-19.03.2-3.el7.x86_64 (docker-ce-stable)
           Requires: systemd
Error: Package: containerd.io-1.2.6-3.3.el7.x86_64 (docker-ce-stable)
           Requires: systemd
Error: Package: 3:docker-ce-19.03.2-3.el7.x86_64 (docker-ce-stable)
           Requires: libseccomp >= 2.3
Error: Package: 3:docker-ce-19.03.2-3.el7.x86_64 (docker-ce-stable)
           Requires: libseccomp.so.2()(64bit)
Error: Package: 3:docker-ce-19.03.2-3.el7.x86_64 (docker-ce-stable)
           Requires: container-selinux >= 2:2.74
 You could try using --skip-broken to work around the problem
 You could try running: rpm -Va --nofiles --nodigest

It's strange that it work son Vagrant at all given that centos 6 doesn't have systemd and we have sudo systemctl enable --now docker in there.

alpar-t added a commit that referenced this issue Oct 7, 2019
@alpar-t
Mute docker packaging tests (DockerTests)
b958467 
Tracked in #47639
@alpar-t
Copy link
Contributor Author

alpar-t commented Oct 7, 2019 •
edited
Loading

I also tried enabling ip forwarding on centos 7 ( which does have docker ).
The docker image build works but the test still fails:
https://gradle-enterprise.elastic.co/s/drfn2zihbbsfg

I'm muted the tests in b958467

@pugnascotia
Copy link
Contributor

I'm on it.

@pugnascotia pugnascotia mentioned this issue Oct 7, 2019
Merged
@alpar-t alpar-t mentioned this issue Oct 7, 2019
Merged
@pugnascotia pugnascotia mentioned this issue Oct 9, 2019
Merged
pugnascotia added a commit that referenced this issue Nov 4, 2019
@pugnascotia
Only define Docker pkg tests if Docker is available (#47640)
1f3d101 
Closes #47639, and unmutes tests that were muted in b958467.

The Docker packaging tests were being defined irrespective of whether
Docker was actually available in the current environment. Instead,
implement exclude lists so that in environments where Docker is not
available, no Docker packaging tests are defined. For CI hosts, the build
checks `.ci/dockerOnLinuxExclusions`. The Vagrant VMs can defined the 
extension property `shouldTestDocker` property to opt-in to packaging
tests.

As part of this, define a seperate utility class for checking Docker,
and call that instead of defining checks in-line in BuildPlugin.groovy
pugnascotia added a commit to pugnascotia/elasticsearch that referenced this issue Nov 5, 2019
@pugnascotia
Only define Docker pkg tests if Docker is available (elastic#47640)
1921bdc 
Closes elastic#47639, and unmutes tests that were muted in b958467.

The Docker packaging tests were being defined irrespective of whether
Docker was actually available in the current environment. Instead,
implement exclude lists so that in environments where Docker is not
available, no Docker packaging tests are defined. For CI hosts, the build
checks `.ci/dockerOnLinuxExclusions`. The Vagrant VMs can defined the
extension property `shouldTestDocker` property to opt-in to packaging
tests.

As part of this, define a seperate utility class for checking Docker,
and call that instead of defining checks in-line in BuildPlugin.groovy
pugnascotia added a commit that referenced this issue Nov 5, 2019
@pugnascotia
Add Docker packaging tests on 7.x (#48857)
24f7d4e 
Backport of #46599 and #47640. Add packaging tests for Docker.

* Introduce packaging tests for Docker (#46599)

Closes #37617. Add packaging tests for our Docker images, similar to what
we have for RPMs or Debian packages. This works by running a container and
probing it e.g. via `docker exec`. Test can also be run in Vagrant, by
exporting the Docker images to disk and loading them again in VMs. Docker
is installed via `Vagrantfile` in a selection of boxes.

* Only define Docker pkg tests if Docker is available (#47640)

Closes #47639, and unmutes tests that were muted in b958467.

The Docker packaging tests were being defined irrespective of whether
Docker was actually available in the current environment. Instead,
implement exclude lists so that in environments where Docker is not
available, no Docker packaging tests are defined. For CI hosts, the build
checks `.ci/dockerOnLinuxExclusions`. The Vagrant VMs can defined the
extension property `shouldTestDocker` property to opt-in to packaging
tests.

As part of this, define a seperate utility class for checking Docker,
and call that instead of defining checks in-line in BuildPlugin.groovy
@mark-vieira mark-vieira added the Team:Delivery Meta label for Delivery team label Nov 11, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@pugnascotia pugnascotia

Labels
:Delivery/Packaging RPM and deb packaging, tar and zip archives, shell and batch scripts Team:Delivery Meta label for Delivery team >test-failure Triaged test failures from CI v8.0.0-alpha1
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Only define Docker pkg tests if Docker is available pugnascotia/elasticsearch
5 participants
@jakelandis @alpar-t @mark-vieira @pugnascotia @elasticmachine