Merge pull request #68 from shizunge/readme

Readme

README.md

@@ -88,7 +88,7 @@ You can configure the most behaviors of *Gantry* via environment variables.
 | GANTRY_UPDATE_JOBS            | false | Set to `true` to update replicated-job or global-job. Set to `false` to disable updating jobs. *Gantry* adds additional options to `docker service update` when there is [no running tasks](docs/faq.md#how-to-update-services-with-no-running-tasks). You can apply a different value to a particular service via [labels](#labels). |
 | GANTRY_UPDATE_NUM_WORKERS     | 1     | The maximum number of updates that can run in parallel. |
 | GANTRY_UPDATE_OPTIONS         |       | [Options](https://docs.docker.com/engine/reference/commandline/service_update/#options) added to the `docker service update` command for all services. You can apply a different value to a particular service via [labels](#labels). |
-| GANTRY_UPDATE_TIMEOUT_SECONDS | 300   | Error out if updating of a single service takes longer than the given time. You can apply a different value to a particular service via [labels](#labels). |
+| GANTRY_UPDATE_TIMEOUT_SECONDS | 0     | Error out if updating of a single service takes longer than the given time. Set to `0` to disable timeout. You can apply a different value to a particular service via [labels](#labels). |
 
 ### After updating
 
@@ -108,18 +108,18 @@ If the images of services are hosted on multiple registries that are required au
 
 * Each line should contain 4 columns, which are either `<TAB>` or `<SPACE>` separated. The columns are 
 ```
-<config name> <host> <user> <password>
+<configuration> <host> <user> <password>
 ```
-> * config name: an identifier for the account.
+> * configuration: an identifier for the account. This is used as a path to [Docker configuration files](https://docs.docker.com/engine/reference/commandline/cli/#configuration-files), which could be either a relative path or an absolute path.
 > * host: the registry to authenticate against, e.g. docker.io.
 > * user: the user name to authenticate as.
 > * password: the password to authenticate with.
 * Lines starting with  `#` are comments.
 * Empty lines, comment lines and invalid lines are ignored.
 
-You need to tell *Gantry* to use a named config rather than the default one when updating a particular service. The named configurations are set via either `GANTRY_REGISTRY_CONFIG`, `GANTRY_REGISTRY_CONFIG_FILE` or `GANTRY_REGISTRY_CONFIGS_FILE`. This can be done by adding the following label to the service `gantry.auth.config=<config-name>`. *Gantry* creates [Docker configuration files](https://docs.docker.com/engine/reference/commandline/cli/#configuration-files) and adds `--config <config-name>` to the Docker command line for the corresponding services.
+You need to tell *Gantry* to use a named configuration rather than the default one when updating a particular service. The named configurations are set via either `GANTRY_REGISTRY_CONFIG`, `GANTRY_REGISTRY_CONFIG_FILE` or `GANTRY_REGISTRY_CONFIGS_FILE`. This can be done by adding the following label to the service `gantry.auth.config=<configuration>`. *Gantry* creates [Docker configuration files](https://docs.docker.com/engine/reference/commandline/cli/#configuration-files) and adds `--config <configuration>` to the Docker command line for the corresponding services.
 
-> NOTE: *Gantry* automatically adds `--with-registry-auth` to the `docker service update` command for a sevice, when it finds the label `gantry.auth.config=<config-name>` on the service. Without `--with-registry-auth`, the service will be updated to an image without digest. See this [comment](https://github.com/shizunge/gantry/issues/53#issuecomment-2348376336).
+> NOTE: *Gantry* automatically adds `--with-registry-auth` to the `docker service update` command for a service, when it finds the label `gantry.auth.config=<configuration>` on the service, or when it logs in with the default Docker configuration. Without `--with-registry-auth`, the service will be updated to an image without digest. See this [comment](https://github.com/shizunge/gantry/issues/53#issuecomment-2348376336).
 
 > NOTE: You can use `GANTRY_REGISTRY_CONFIGS_FILE` together with other authentication environment variables.
 
@@ -131,7 +131,7 @@ Labels can be added to services to modify the behavior of *Gantry* for particula
 
 | Label  | Description |
 |--------|-------------|
-| `gantry.auth.config=<config-name>`       | See [Authentication](#authentication). |
+| `gantry.auth.config=<configuration>`     | See [Authentication](#authentication). |
 | `gantry.services.excluded=true`          | Exclude the services from updating if you are using the default [`GANTRY_SERVICES_EXCLUDED_FILTERS`](#to-select-services). |
 | `gantry.manifest.cmd=<command>`          | Override [`GANTRY_MANIFEST_CMD`](#to-check-if-new-images-are-available) |
 | `gantry.manifest.options=<string> `      | Override [`GANTRY_MANIFEST_OPTIONS`](#to-check-if-new-images-are-available) |

docs/faq.md

@@ -16,17 +16,7 @@ At the end of updating, *Gantry* optionally removes the old images.
 
 It will not work by setting multiple filters with different names, because filters are logical **ANDED**.
 
-To filter multiple services, you can set a label on each service then let *Gantry* filter on that label. Or you can run multiple *Gantry* instances.
-
-Advanced user can also create their own entrypoint using functions in [lib-gantry.sh](../src/lib-gantry.sh).
-```
-# notification.sh is optional.
-source ./src/lib-common.sh;
-source ./src/lib-gantry.sh;
-gantry_initialize;
-gantry_update_services_list "${LIST_OF_SERVICES_TO_UPDATE}";
-gantry_finalize;
-```
+To filter multiple services, you can set a label on each service then let *Gantry* filter on that label via `GANTRY_SERVICES_FILTERS`. Or you can run multiple *Gantry* instances.
 
 ### How to run *Gantry* on a cron schedule?
 
@@ -38,13 +28,15 @@ As discussed [here](https://github.com/docker/cli/issues/627), the CLI will hang
 
 *Gantry* will check whether there are running tasks in a service. If there is no running task, *Gantry* automatically adds the option `--detach=true`. In addition to the detach option, *Gantry* also adds `--replicas=0` for services in replicated mode. You don't need to add these options manually.
 
-### When to set `GANTRY_MANIFEST_CMD`?
+### What `GANTRY_MANIFEST_CMD` to use?
 
-Before updating a service, *Gantry* will try to obtain the image's meta data to decide whether there is a new image. If there is no new image, *Gantry* skips calling `docker service update`.
+Before updating a service, *Gantry* will try to obtain the image's meta data to decide whether there is a new image. If there is no new image, *Gantry* skips calling `docker service update`, leading to a speedup of the overall process.
 
-`docker buildx imagetools inspect` is selected as the default, because `docker manifest inspect` could [fail on some registries](https://github.com/orgs/community/discussions/45779). Additionally, `docker buildx imagetools` can obtain the digest of multi-arch images, which could help not to run the `docker service update` CLI when there is no new images.
+In most cases, the default value `buildx` of `GANTRY_MANIFEST_CMD` should work. `docker buildx imagetools inspect` is selected as the default, because `docker manifest inspect` could [fail on some registries](https://github.com/orgs/community/discussions/45779). Additionally, `docker buildx imagetools` can obtain the digest of multi-arch images, which could help reduce the number of calling the `docker service update` CLI when there is no new images.
 
-You can switch back to use [`docker manifest inspect`](https://docs.docker.com/engine/reference/commandline/manifest_inspect/) for the features that are not supported by [`docker buildx imagetools inspect`](https://docs.docker.com/engine/reference/commandline/buildx_imagetools_inspect/).
+We keep [`docker manifest inspect`](https://docs.docker.com/engine/reference/commandline/manifest_inspect/) for debugging purpose. There is no known advantage to use `manifest`.
+
+You can disable the image inspection by setting `GANTRY_MANIFEST_CMD` to `none` in case there is a bug. Please report the bug through a [GitHub issue](https://github.com/shizunge/gantry/issues). Another use case of `none` is that you want to add `--force` to the `docker service update` command via `GANTRY_UPDATE_OPTIONS`, which updates the services even if there is nothing changed.
 
 ### I logged in my Docker Hub account, but the Docker Hub rate reported seems incorrect.
 

src/lib-common.sh

@@ -15,17 +15,36 @@
 # along with this program.  If not, see <https://www.gnu.org/licenses/>.
 #
 
+# Run "grep -q" and avoid broken pipe errors.
+grep_q() {
+  # "grep -q" will exit immediately when the first line of data matches, and leading to broken pipe errors.
+  grep -q -- "${@}";
+  local GREP_RETURN=$?;
+  # Add "cat > /dev/null" to avoid broken pipe errors.
+  cat >/dev/null;
+  return "${GREP_RETURN}"
+}
+
+# Similar to grep_q.
+# grep case insensitively.
+grep_q_i() {
+  grep -q -i -- "${@}";
+  local GREP_RETURN=$?;
+  cat >/dev/null;
+  return "${GREP_RETURN}"
+}
+
 # echo the number of the log level.
 # return 0 if LEVEL is supported.
 # return 1 if LEVLE is unsupported.
 _log_level() {
   local LEVEL="${1}";
   [ -z "${LEVEL}" ] && _log_level "INFO" && return 1;
-  echo "${LEVEL}" | grep -q -i "^DEBUG$" && echo 0 && return 0;
-  echo "${LEVEL}" | grep -q -i "^INFO$"  && echo 1 && return 0;
-  echo "${LEVEL}" | grep -q -i "^WARN$"  && echo 2 && return 0;
-  echo "${LEVEL}" | grep -q -i "^ERROR$" && echo 3 && return 0;
-  echo "${LEVEL}" | grep -q -i "^NONE$"  && echo 4 && return 0;
+  echo "${LEVEL}" | grep_q_i "^DEBUG$" && echo 0 && return 0;
+  echo "${LEVEL}" | grep_q_i "^INFO$"  && echo 1 && return 0;
+  echo "${LEVEL}" | grep_q_i "^WARN$"  && echo 2 && return 0;
+  echo "${LEVEL}" | grep_q_i "^ERROR$" && echo 3 && return 0;
+  echo "${LEVEL}" | grep_q_i "^NONE$"  && echo 4 && return 0;
   _log_level "NONE";
   return 1;
 }
@@ -37,10 +56,10 @@ _level_color() {
   local ORANGE='\033[0;33m'
   local GREEN='\033[0;32m'
   local BLUE='\033[0;34m'
-  echo "${LEVEL}" | grep -q -i "^DEBUG$" && echo "${BLUE}" && return 0;
-  echo "${LEVEL}" | grep -q -i "^INFO$"  && echo "${GREEN}" && return 0;
-  echo "${LEVEL}" | grep -q -i "^WARN$"  && echo "${ORANGE}" && return 0;
-  echo "${LEVEL}" | grep -q -i "^ERROR$" && echo "${RED}" && return 0;
+  echo "${LEVEL}" | grep_q_i "^DEBUG$" && echo "${BLUE}" && return 0;
+  echo "${LEVEL}" | grep_q_i "^INFO$"  && echo "${GREEN}" && return 0;
+  echo "${LEVEL}" | grep_q_i "^WARN$"  && echo "${ORANGE}" && return 0;
+  echo "${LEVEL}" | grep_q_i "^ERROR$" && echo "${RED}" && return 0;
   echo "${NO_COLOR}"
 }
 
@@ -164,7 +183,7 @@ is_number() {
 is_true() {
   local CONFIG="${1}"
   CONFIG=$(echo "${CONFIG} " | cut -d ' ' -f 1)
-  echo "${CONFIG}" | grep -q -i "true"
+  echo "${CONFIG}" | grep_q_i "true"
 }
 
 difference_between() {
@@ -228,15 +247,10 @@ read_config() {
 read_env() {
   local VNAME="${1}"; shift
   [ -z "${VNAME}" ] && return 1
-  # "grep -q" will exit immediately when the first line of data matches, and leading to broken pipe errors.
-  # Add "cat > /dev/null" to avoid broken pipe errors.
-  local GREP_RETURN=
-  env | (grep -q "^${VNAME}="; local R=$?; cat >/dev/null; test "${R}" == "0";);
-  GREP_RETURN=$?
-  if [ "${GREP_RETURN}" != "0" ]; then
-    echo "${@}"
-  else
+  if env | grep_q "^${VNAME}="; then
     eval "echo \"\${${VNAME}}\""
+  else
+    echo "${@}"
   fi
   return 0
 }
@@ -299,7 +313,7 @@ _get_docker_command_name_arg() {
 }
 
 _get_docker_command_detach() {
-  if echo "${@}" | grep -q -- "--detach"; then
+  if echo "${@}" | grep_q "--detach"; then
     echo "true"
     return 0
   fi
@@ -344,7 +358,7 @@ _docker_service_task_states() {
     NAME=$(echo "${LINE}" | cut -d ']' -f 1 | cut -d '[' -f 2)
     NODE_STATE_AND_ERROR=$(echo "${LINE}" | cut -d ']' -f 2-)
     # We assume that the first State of each task is the latest one that we want to report.
-    if ! echo "${NAME_LIST}" | grep -q "${NAME}"; then
+    if ! echo "${NAME_LIST}" | grep_q "${NAME}"; then
       echo "${NODE_STATE_AND_ERROR}"
     fi
     NAME_LIST=$(echo -e "${NAME_LIST}\n${NAME}" | sort | uniq)
@@ -358,8 +372,8 @@ _docker_service_task_states() {
 wait_service_state() {
   local SERVICE_NAME="${1}"; shift;
   local WAIT_RUNNING WAIT_COMPLETE;
-  WAIT_RUNNING=$(echo "${@}" | grep -q -- "--running" && echo "true" || echo "false")
-  WAIT_COMPLETE=$(echo "${@}" | grep -q -- "--complete" && echo "true" || echo "false")
+  WAIT_RUNNING=$(echo "${@}" | grep_q "--running" && echo "true" || echo "false")
+  WAIT_COMPLETE=$(echo "${@}" | grep_q "--complete" && echo "true" || echo "false")
   local RETURN_VALUE=0
   local DOCKER_CMD_ERROR=1
   local SLEEP_SECONDS=1
@@ -377,9 +391,9 @@ wait_service_state() {
     while read -r LINE; do
       [ -z "${LINE}" ] && continue;
       NUM_LINES=$((NUM_LINES+1));
-      echo "${LINE}" | grep -q "Running" && NUM_RUNS=$((NUM_RUNS+1));
-      echo "${LINE}" | grep -q "Complete" && NUM_DONES=$((NUM_DONES+1));
-      echo "${LINE}" | grep -q "Failed" && NUM_FAILS=$((NUM_FAILS+1));
+      echo "${LINE}" | grep_q "Running" && NUM_RUNS=$((NUM_RUNS+1));
+      echo "${LINE}" | grep_q "Complete" && NUM_DONES=$((NUM_DONES+1));
+      echo "${LINE}" | grep_q "Failed" && NUM_FAILS=$((NUM_FAILS+1));
     done < <(echo "${STATES}")
     if [ "${NUM_LINES}" -gt 0 ]; then
       if "${WAIT_RUNNING}" && [ "${NUM_RUNS}" -eq "${NUM_LINES}" ]; then