CI failing due to hot plug CPUs

[jcvenegas]

Summarizing 4 Failures:

[Fail] Hot plug CPUs container with CPU period and quota [It] should have 3 CPUs
/home/jenkins/jenkins_slave/workspace/kata-containers-tests-ubuntu-17-10-PR/go/src/github.com/kata-containers/tests/integration/docker/cpu_test.go:91

[Fail] Hot plug CPUs container with CPU period and quota [It] should have 4 CPUs
/home/jenkins/jenkins_slave/workspace/kata-containers-tests-ubuntu-17-10-PR/go/src/github.com/kata-containers/tests/integration/docker/cpu_test.go:91

[Fail] Hot plug CPUs container with CPU constraint [It] should have 3 CPUs
/home/jenkins/jenkins_slave/workspace/kata-containers-tests-ubuntu-17-10-PR/go/src/github.com/kata-containers/tests/integration/docker/cpu_test.go:108

[Fail] Hot plug CPUs container with CPU constraint [It] should have 3 CPUs
/home/jenkins/jenkins_slave/workspace/kata-containers-tests-ubuntu-17-10-PR/go/src/github.com/kata-containers/tests/integration/docker/cpu_test.go:108

Ran 98 of 159 Specs in 543.348 seconds
FAIL! -- 94 Passed | 4 Failed | 0 Pending | 61 Skipped --- FAIL: TestIntegration (567.07s)
FAIL

[devimc]

Seems like this issue is more complex than I thought, now that our kernel supports cpuSet libcontainer places the container inside a CPU cgroup with the number of vCPUs available when it is created. VCPUs are hotplugged and connected asynchronously, not all vCPUs are available when the container is created causing this issue

[bergwolf]

I think we should revisit the user experience to calculate CPU numbers. E.g., the tests uses

docker run --runtime kata --rm --cpu-quota 30000 --cpu-period 10000 -it busybox

which seems to require 3 vCPUs to me but we are actually plugging 4 vCPUs to the guest whereas libcontainer only represents 3 of them to the container process.

[devimc]

@bergwolf we always start the containers with 1 vCPU (unless something different is specified in the configuration file), if --cpu-quota+--cpu-period or --cpus are used, then those vCPUs are hot added.

take a look https://github.com/clearcontainers/runtime/blob/master/docs/cpu-constraints.md

Before running containers without CPU constraint, consider that your containers are not running alone. Since your containers run inside a virtual machine other processes try to use the vCPUs as well (e.g. systemd and the Clear Container agent).