Adjust invoker playbook to pull docker images when a prefix and tag is specified.

[rabbah]

Description

The invoker playbook ignores an image prefix when pre-fetching the blackbox images. This patch splits the blackbox tasks so that ones that do not have a prefix may be skipped (as today), but those that specify a prefix are always pulled, using the specified prefix in the image name.

Fixes #3679.

Related issue and scope

• I opened an issue to propose and discuss this change (invoker playbook does not properly pull blackbox images #3679)

My changes affect the following components

• API
• Controller
• Message Bus (e.g., Kafka)
• Loadbalancer
• Invoker
• Intrinsic actions (e.g., sequences, conductors)
• Data stores (e.g., CouchDB)
• Tests
• Deployment
• CLI
• General tooling
• Documentation

Types of changes

• Bug fix (generally a non-breaking change which closes an issue).
• Enhancement or new feature (adds new functionality).
• Breaking change (a bug fix or enhancement which changes existing behavior).

Checklist:

• I signed an Apache CLA.
• I reviewed the style guides and followed the recommendations (Travis CI will check :).
• I added tests to cover my changes.
• My changes require further changes to the documentation.
• I updated the documentation where necessary.

[codecov-io]

Codecov Report

Merging #3680 into master will increase coverage by 0.06%.
The diff coverage is 76.92%.

@@            Coverage Diff             @@
##           master    #3680      +/-   ##
==========================================
+ Coverage   74.92%   74.99%   +0.06%     
==========================================
  Files         132      128       -4     
  Lines        6186     6075     -111     
  Branches      392      388       -4     
==========================================
- Hits         4635     4556      -79     
+ Misses       1551     1519      -32

Impacted Files	Coverage Δ
...er/src/main/scala/whisk/core/invoker/Invoker.scala	0% <0%> (ø)	⬆️
...erpool/kubernetes/KubernetesContainerFactory.scala	0% <0%> (ø)	⬆️
.../scala/src/main/scala/whisk/core/WhiskConfig.scala	92% <100%> (-0.25%)	⬇️
.../containerpool/docker/DockerContainerFactory.scala	26.66% <100%> (ø)	⬆️
...rc/main/scala/whisk/core/entity/ExecManifest.scala	96.29% <100%> (-0.05%)	⬇️
...scala/whisk/core/mesos/MesosContainerFactory.scala	61.81% <100%> (ø)	⬆️
.../scala/src/main/scala/whisk/core/entity/UUID.scala	81.81% <0%> (-9.1%)	⬇️
...whisk/connector/kafka/KafkaConsumerConnector.scala	58.33% <0%> (-3.94%)	⬇️
...ala/whisk/core/database/CouchDbStoreProvider.scala	80% <0%> (-1.82%)	⬇️
...hisk/core/controller/actions/SequenceActions.scala	30.27% <0%> (-1.16%)	⬇️
... and 29 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 9e5cec1...c600668. Read the comment docs.

[dubee]

PG4 1715 ⏳

[dubee]

PG2 3168 ⏳

[rabbah]

@csantanapr made the changes we discussed on Slack to disentangle the image pulls from the docker registry.

[rabbah]

@dgrove-oss I think you'll need to review this now as its scope as widened a little. I did checked the kube deployment and I think the change is compatible but will appreciate your input.

With the latest changes, runtime images are pulled from the given docker registry for runtimes (default is dockerhub) which is now allowed to be different from the docker.registry; the latter is now strictly used for the system components.

It is possible to skip the pulling of images entirely:

    ansible-playbook -i environments/local invoker.yml -e 'skip_pull_runtimes=true'

or with redo

    redo invoker -e"skip_pull_runtimes=true" 

[rabbah]

@ddragosd this may also be relevant to docker-compose although I haven't checked that deployment yet.

[rabbah]

Some caveats with this change: building new runtime images for local testing means:

• locally tagged image used in the runtimes manifest may be used without pushing to docker hub, provided the skip_pull_runtimes is enabled
• if skip_pull_runtimes is enabled but the runtimes manifest includes images that are not already pulled, they may fail in the invoker, or be pulled on the first run.

FYI @remore @akrabat @sciabarracom since you will be affected by this change if we accept it.

[dgrove-oss]

Looks like this would be straightforward for kube. We'd need to update the copies of runtime.json and tweak values.yaml and the various deployment templates for the pods to reflect the change in environment variables.

[csantanapr]

LGTM