feat: use Dockerhub API to poll for digest change

[qbiqing]

• Use Dockerhub API (docs) instead of checking manifest from Docker registry (docs)
  so each poll does not count as an image pull. This is needed as we got warning from
  docker support of exceeding docker rate limits.
• Mock Dockerhub server in tests.

[qbiqing]

Also tried to get integration test to run in CI. The main difficulty is to run Docker-in-Docker. I tried the following:

• First, naively install Docker in the test image. I do feel like that defeats the point of using alpine because we have to install docker, and openrc. Getting docker to start is also pretty unergonomic (can't use service or rc-service gliderlabs/docker-alpine#183 (comment)). Plus this blog post introduces a lot of other issues with using Docker in Docker. So I decided to follow his advice and try mounting the Docker socket as a volume.
• I found this Codefresh guide which seems pretty useful. However I ran into quite some trouble following it. I tried to mount the volumes directly in the freestyle step but it didn't work. Also tried the lpipeline way (https://github.com/dsaidgovsg/lpipeline/blob/cceb826a9bb3a03ee5bcdc0c3833cbec121ef85e/codefresh-integration.yml#L122) yet /codefresh/volume/var/run/docker.sock was not found. I think it might be because as the guide says, the Docker daemon is not exposed in freestyle step on SaaS environments for security reasons.
• Then I tried launching docker under a composition step as suggested. However, now Codefresh composition does not support volumes. Found this thread which explains the error (thanks @xtrntr for starting it!) and also posted a follow-up comment.

In conclusion I think the Docker-to-Docker setup may not be very compatible with Codefresh since they don't seem to expose the Docker daemon without some kind of customization of the runtime environment.

[xtrntr]

In conclusion I think the Docker-to-Docker setup may not be very compatible with Codefresh since they don't seem to expose the Docker daemon without some kind of customization of the runtime environment.

Thanks @qbiqing for documenting why the docker-in-docker approach failed 👍

[qbiqing]

Wrt to running docker-in-docker in CI, Codefresh support has replied the thread, so I think I will continue experimenting in another PR.

[xtrntr]

LGTM, thanks for this!

[qbiqing]

Updated note on using DinD for integration test: https://community.codefresh.io/t/mounting-docker-socket-in-composition-step/615/2

Short answer: not possible with SaaS codefresh