Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Making list-remote a little more stable #33

Open
wants to merge 54 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
54 commits
Select commit Hold shift + click to select a range
97ef626
Adding the 100 newest versions
avnerenv0 Feb 3, 2022
2536064
Adding the 100 newest versions
avnerenv0 Feb 3, 2022
dce2b26
fix local fallback
avnerenv0 Feb 7, 2022
e4b1d37
Merge pull request #1 from env0/0.0.3.1-fix-local-fallback
avnerenv0 Feb 7, 2022
591fb3d
extracting more pages and versions
sagilaufer1992 Mar 17, 2022
48edeb3
Merge pull request #2 from env0/fix-list-remote-fetch-more-versions
sagilaufer1992 Mar 17, 2022
480c866
added more versions
sagilaufer1992 Mar 17, 2022
7b0ee93
Merge pull request #3 from env0/chore-more-versions
sagilaufer1992 Mar 17, 2022
7a99cee
regex works
sagilaufer1992 Mar 17, 2022
bf6802c
make avners change work again
sagilaufer1992 Mar 17, 2022
f6612c8
typo
sagilaufer1992 Mar 17, 2022
742f8e4
remove useless
sagilaufer1992 Mar 17, 2022
2b730bd
Merge pull request #4 from env0/fix-set-e-again
sagilaufer1992 Mar 17, 2022
06b772a
adding versions and paging
sagilaufer1992 Mar 20, 2022
7c4465d
merge from hotfix
sagilaufer1992 Mar 20, 2022
7305c42
querying the amount of pages of tg versions and querying as this amount
sagilaufer1992 Mar 28, 2022
58af206
typo
sagilaufer1992 Mar 28, 2022
0cd23f8
not printing curl irrelevant data
sagilaufer1992 Apr 6, 2022
c777261
merged master to avoid conflicts
sagilaufer1992 Apr 6, 2022
9ab31e5
rename variable only_relevant_part -> last_page_phrase
sagilaufer1992 Apr 6, 2022
665f83f
Merge pull request #5 from env0/chore-pages-not-magic-number
sagilaufer1992 Apr 7, 2022
4f5beb7
rename variable only_relevant_part -> last_page_phrase
sagilaufer1992 Apr 10, 2022
32267a3
Revert "Port over auto install concept from tfenv"
sagilaufer1992 Apr 10, 2022
046bc3a
Merge pull request #6 from env0/fix-set-e-flag-removal
sagilaufer1992 Apr 10, 2022
dde4c00
Replace cunymatthieu with env0 in README
cjonesy May 25, 2022
22fe2fc
graceful kill on curl failure
tomer-landesman Jun 1, 2022
b40b8d2
fixed error message
tomer-landesman Jun 1, 2022
596926f
Update libexec/tgenv-install
tomer-landesman Jun 1, 2022
39d19d0
Merge pull request #8 from env0/fix-delete-version-folder-on-curl-fail
tomer-landesman Jun 1, 2022
9419748
Merge pull request #7 from cjonesy/update-readme-to-env0
chpl Jun 2, 2022
ff7c089
Update README.md
chpl Jun 2, 2022
76fd78b
add-configure-option-to-store-version-in-custom-path
HeverFarber Jul 4, 2022
36af258
revert
HeverFarber Jul 4, 2022
5b21474
remove-idea
HeverFarber Jul 4, 2022
acb29b7
add-option
HeverFarber Jul 4, 2022
9f5eb9a
add--readme
HeverFarber Jul 4, 2022
0c90fe0
Merge pull request #9 from env0/add-configure-option-to-store-version…
HeverFarber Jul 4, 2022
dc4c0c0
remove checking remote versions when installing
sagilaufer1992 Jul 12, 2022
97c31ed
Merge pull request #10 from env0/chore-not-list-remote-when-installing
sagilaufer1992 Jul 14, 2022
4d32cf3
add retry function for installation, handling curl 503 use cases
razbensimon Jul 2, 2023
03ecd35
retry for "use" command to handle flaky error:
razbensimon Jul 2, 2023
b3b7b89
WIP debug
razbensimon Jul 2, 2023
4d60309
Revert "WIP debug"
razbensimon Jul 2, 2023
f9cc0f1
fix argument
razbensimon Jul 2, 2023
6d22981
WIP 500 condition
razbensimon Jul 2, 2023
7e22de4
create new retry function as a script file
razbensimon Jul 3, 2023
228bdc6
refactor
razbensimon Jul 4, 2023
fe6dfcf
try transform to a function
razbensimon Jul 4, 2023
bf56767
Works after QA
razbensimon Jul 4, 2023
1605f02
fix folder cleanup after failed install
razbensimon Jul 5, 2023
a94f168
Merge pull request #11 from env0/add-retry-for-install
razbensimon Jul 6, 2023
9ebb1f3
fix logs go to stderr for displaying
razbensimon Jul 6, 2023
872166c
rename
razbensimon Jul 6, 2023
aafd4f2
Merge pull request #13 from env0/fix-err-logs
razbensimon Jul 6, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -2,3 +2,4 @@ versions/
version
.terragrunt-version
bin/terragrunt-*
.idea/*
28 changes: 7 additions & 21 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
[![Build Status](https://travis-ci.org/cunymatthieu/tgenv.svg?branch=master)](https://travis-ci.org/cunymatthieu/tgenv)
[![Build Status](https://travis-ci.org/env0/tgenv.svg?branch=master)](https://travis-ci.org/env0/tgenv)

# tgenv
[Terragrunt](https://github.com/gruntwork-io/terragrunt) version manager inspired by [tfenv](https://github.com/kamatama41/tfenv)
Expand All @@ -14,7 +14,7 @@ Currently tgenv supports the following OSes
1. Check out tgenv into any path (here is `${HOME}/.tgenv`)

```bash
$ git clone https://github.com/cunymatthieu/tgenv.git ~/.tgenv
$ git clone https://github.com/env0/tgenv.git ~/.tgenv
```

2. Add `~/.tgenv/bin` to your `$PATH` any way you like
Expand Down Expand Up @@ -130,29 +130,15 @@ $ echo latest:^0.10 > .terragrunt-version
$ terragrunt --version
terragrunt v0.10.3
```

### Environment Variables

#### TGENV

##### `TGENV_AUTO_INSTALL`

String (Default: true)

Should tgenv automatically install terragrunt if the version specified by defaults or a .terragrunt-version file is not currently installed.

```console
TGENV_AUTO_INSTALL=false terragrunt plan
```

##### `TGENV_DEBUG`

Integer (Default: "")
##### `TGENV_CONFIG_DIR`

Set the debug level for TGENV.
Path (Default: `$TGENV_ROOT`)

* unset/empty-string: No debug output
* set: Bash execution tracing
The path to a directory where the local terragrunt versions and configuration files exist.

## Upgrading
```bash
Expand All @@ -165,5 +151,5 @@ $ rm -rf /some/path/to/tgenv
```

## LICENSE
- [tgenv itself](https://github.com/cunymatthieu/tgenv/blob/master/LICENSE)
- [tfenv ](https://github.com/kamatama41/tgenv/blob/master/LICENSE) : tfenv mainly uses tfenv's source code
- [tgenv itself](https://github.com/env0/tgenv/blob/master/LICENSE)
- [tfenv ](https://github.com/kamatama41/tfenv/blob/master/LICENSE) : tgenv mainly uses tfenv's source code
7 changes: 7 additions & 0 deletions bin/tgenv
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,13 @@ PATH="${TGENV_ROOT}/libexec:${PATH}"
export PATH
export TGENV_DIR="${PWD}"

if [ -z "${TGENV_CONFIG_DIR:-""}" ]; then
TGENV_CONFIG_DIR="$TGENV_ROOT";
else
TGENV_CONFIG_DIR="${TGENV_CONFIG_DIR%/}";
fi
export TGENV_CONFIG_DIR;

abort() {
{ if [ "${#}" -eq 0 ]; then cat -
else echo "tgenv: ${*}"
Expand Down
4 changes: 4 additions & 0 deletions libexec/helpers
Original file line number Diff line number Diff line change
@@ -1,5 +1,9 @@
#!/usr/bin/env bash

function error_log() {
echo -e "tgenv: $(basename ${0}): \033[0;31m[ERROR] ${1}\033[0;39m" >&2
}

function error_and_die() {
echo -e "tgenv: $(basename ${0}): \033[0;31m[ERROR] ${1}\033[0;39m" >&2
exit 1
Expand Down
82 changes: 82 additions & 0 deletions libexec/retry
Original file line number Diff line number Diff line change
@@ -0,0 +1,82 @@
#!/usr/bin/env bash

info_log() {
echo -e "\033[0;32m[INFO] ${1}\033[0;39m" >&2
}

retry() {
# Default values
local tries=3
local sleep_interval=5
local stop_on_exit_code=""

# Function to display script usage
usage() {
echo "Usage: retry [options] <command>"
echo "Options:"
echo " -sleep <seconds> Sleep interval between tries (default: 5)"
echo " -tries <count> Maximum number of tries (default: 3)"
echo " -stop-on-exit-code <code> Exit loop if command returns the specified code"
echo " -help, -h Display this help message"
return 1
}

# Parse command-line options
while [[ $# -gt 0 ]]; do
case "$1" in
-sleep)
sleep_interval=$2
shift 2
;;
-tries)
tries=$2
shift 2
;;
-stop-on-exit-code)
stop_on_exit_code=$2
shift 2
;;
-help | -h)
usage
;;
*)
break
;;
esac
done

# Check if a command is provided
if [[ $# -eq 0 ]]; then
usage
return 1
fi

# Execute the command in a loop until success or maximum tries reached
local counter=0
while [[ $counter -lt $tries ]]; do
"$@"
local exit_code=$?

if [[ $exit_code -eq 0 ]]; then
return 0
elif [[ -n $stop_on_exit_code && $exit_code -eq $stop_on_exit_code ]]; then
info_log "Exit code ($stop_on_exit_code) matching to condition. Won't retry, exiting."
return 1
fi

counter=$((counter + 1))

if [[ $counter -lt $tries ]]; then
info_log "Attempt $counter failed. Retrying after $sleep_interval seconds..."
sleep "$sleep_interval"
fi
done

info_log "Command failed after $tries attempts."
return 1
}

# Call the retry function if the script is executed directly
if [[ "${BASH_SOURCE[0]}" == "${0}" ]]; then
retry "$@"
fi
25 changes: 3 additions & 22 deletions libexec/tgenv-exec
Original file line number Diff line number Diff line change
Expand Up @@ -15,27 +15,8 @@

set -e
[ -n "${TGENV_DEBUG}" ] && set -x
source "${TGENV_ROOT}/libexec/helpers"

info 'Getting version from tgenv-version-name';
TGENV_VERSION="$(tgenv-version-name)" \
&& info "TGENV_VERSION is ${TGENV_VERSION}" \
|| {
# Errors will be logged from tgenv-version name,
# we don't need to trouble STDERR with repeat information here
error_and_die 'Failed to get version from tgenv-version-name';
};
export TGENV_VERSION;

if [ ! -d "${TGENV_ROOT}/versions/${TGENV_VERSION}" ]; then
if [ "${TGENV_AUTO_INSTALL:-false}" == "true" ]; then
info "version '${TGENV_VERSION}' is not installed (set by $(tgenv-version-file)). Installing now as TGENV_AUTO_INSTALL==true";
tgenv-install;
else
error_and_die "version '${TGENV_VERSION}' was requested, but not installed and TGENV_AUTO_INSTALL is not 'true'";
fi;
fi;

TG_BIN_PATH="${TGENV_ROOT}/versions/${TGENV_VERSION}/terragrunt"
export TGENV_VERSION="$(tgenv-version-name)"
TG_BIN_PATH="${TGENV_CONFIG_DIR}/versions/${TGENV_VERSION}/terragrunt"
export PATH="${TG_BIN_PATH}:${PATH}"
"${TG_BIN_PATH}" "${@}"
"${TG_BIN_PATH}" "${@}"
46 changes: 38 additions & 8 deletions libexec/tgenv-install
Original file line number Diff line number Diff line change
Expand Up @@ -2,14 +2,15 @@

[ -n "${TGENV_DEBUG}" ] && set -x
source "${TGENV_ROOT}/libexec/helpers"
source "${TGENV_ROOT}/libexec/retry"

[ "${#}" -gt 1 ] && error_and_die "usage: tgenv install [<version>]"

declare version_requested version regex

if [ -z "${1}" ]; then
version_file="$(tgenv-version-file)"
if [ "${version_file}" != "${TGENV_ROOT}/version" ]; then
if [ "${version_file}" != "${TGENV_CONFIG_DIR}/version" ]; then
version_requested="$(cat "${version_file}" || true)"
fi
else
Expand All @@ -28,10 +29,8 @@ else
fi

[ -n "${version}" ] || error_and_die "Version is not specified"
version="$(tgenv-list-remote | grep -e "${regex}" | head -n 1)"
[ -n "${version}" ] || error_and_die "No versions matching '${1}' found in remote"
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hello,

I don't understand why you have removed the possibility to use the tag latest. Sometimes, you just want the latest version no matter what.


dst_path="${TGENV_ROOT}/versions/${version}"
dst_path="${TGENV_CONFIG_DIR}/versions/${version}"
if [ -f "${dst_path}/terragrunt" ]; then
echo "Terragrunt v${version} is already installed"
exit 0
Expand Down Expand Up @@ -62,12 +61,43 @@ tarball_name="terragrunt_${os}";

info "Installing Terragrunt v${version}"

function install_tg_version() {
local http_code
http_code=$(curlw -# --location-trusted -w "%{http_code}" -o "${dst_path}/terragrunt" "${version_url}/${tarball_name}")
local curl_exit_code=$?

local result_code
if [[ $http_code =~ ^4 ]]; then
error_log "Got $http_code http code, TG version is invalid"
result_code=2 # do not retry
elif [[ $http_code =~ ^5 ]]; then
error_log "Got $http_code http code"
result_code=1 # retry
elif [[ $curl_exit_code -ne 0 ]]; then
result_code=1 # retry
else
result_code=0 # success
fi

if [[ $result_code -ne 0 ]]; then
# make sure versions folder does not exist, so we can retry installation
rm -rf "${dst_path}"
fi

return $result_code
}


mkdir -p "${dst_path}" || error_and_die "Failed to make directory ${dst_path}"

info "Downloading release tarball from ${version_url}/${tarball_name}"
curlw -# --location-trusted -f -o "${dst_path}/terragrunt" "${version_url}/${tarball_name}" || error_and_die "Tarball download failed"
retry -tries 5 -sleep 5 -stop-on-exit-code 2 install_tg_version || error_and_die "Tarball download failed"
install_exist_code=$?

chmod +x "${dst_path}/terragrunt" || error_and_die "Change rights failed"
if [ "$install_exist_code" -eq 0 ]; then

info "Installation of terragrunt v${version} successful"
tgenv-use "${version}"
chmod +x "${dst_path}/terragrunt" || error_and_die "Change rights failed"

info "Installation of terragrunt v${version} successful"
retry -sleep 3 tgenv-use "${version}" || error_and_die "tgenv use failed"
fi
6 changes: 3 additions & 3 deletions libexec/tgenv-list
Original file line number Diff line number Diff line change
Expand Up @@ -6,10 +6,10 @@ source "${TGENV_ROOT}/libexec/helpers"
[ "${#}" -ne 0 ] \
&& error_and_die "usage: tgenv list"

[ -d "${TGENV_ROOT}/versions" ] \
[ -d "${TGENV_CONFIG_DIR}/versions" ] \
|| error_and_die "No versions available. Please install one with: tgenv install"

[[ -x "${TGENV_ROOT}/versions" && -r "${TGENV_ROOT}/versions" ]] \
[[ -x "${TGENV_CONFIG_DIR}/versions" && -r "${TGENV_CONFIG_DIR}/versions" ]] \
|| error_and_die "tgenv versions directory is inaccessible!"

print_version () {
Expand All @@ -20,6 +20,6 @@ print_version () {
fi
}

for local_version in $(ls -1 "${TGENV_ROOT}/versions" | sort -t'.' -k 1nr,1 -k 2nr,2 -k 3nr,3); do
for local_version in $(ls -1 "${TGENV_CONFIG_DIR}/versions" | sort -t'.' -k 1nr,1 -k 2nr,2 -k 3nr,3); do
print_version "${local_version}"
done
20 changes: 18 additions & 2 deletions libexec/tgenv-list-remote
Original file line number Diff line number Diff line change
@@ -1,5 +1,15 @@
#!/usr/bin/env bash
set -e
link_release="https://api.github.com/repos/gruntwork-io/terragrunt/tags?per_page=1000"

# the response header (--head) contains a link to the last page, contains the last page number and the "last" keyword to help identify
page_curl=$(curl -s --tlsv1.2 --head "$link_release")
last_page_phrase=$(echo "$page_curl" | grep -o -E "&page=[0-9]+>; rel=\"last")
re="([0-9]+)"
if [[ $last_page_phrase =~ $re ]] ; then
pages=${BASH_REMATCH[0]}
else
pages=4 # default in case something changes in the way the responses are formatted
fi

[ -n "${TGENV_DEBUG}" ] && set -x
source "${TGENV_ROOT}/libexec/helpers"
Expand All @@ -12,9 +22,15 @@ fi
link_release="https://api.github.com/repos/gruntwork-io/terragrunt/tags?per_page=1000"
print=$(curl --tlsv1.2 -sf $link_release)

for (( page=2; page <= pages; page++))
do
link_release="https://api.github.com/repos/gruntwork-io/terragrunt/tags?per_page=1000&page=${page}"
print+=$(curl --tlsv1.2 -sf $link_release)
done

return_code=$?
if [ $return_code -eq 22 ];then
warn_and_continue "Failed to get list verion on $link_release"
warn_and_continue "Failed to get list version on $link_release"
print=`cat ${TGENV_ROOT}/list_all_versions_offline`
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hello,

I don't see why the file list_all_version_offline is not in the .gitignore and updated each time the website is online

fi

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
echo $print | grep -o -E "[0-9]+\.[0-9]+\.[0-9]+(-(rc|beta)[0-9]+)?" | uniq | tee ${TGENV_ROOT}/list_all_versions_offline

Expand Down
4 changes: 2 additions & 2 deletions libexec/tgenv-uninstall
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ declare version_requested version regex

if [ -z "${1}" ]; then
version_file="$(tgenv-version-file)"
if [ "${version_file}" != "${TGENV_ROOT}/version" ];then
if [ "${version_file}" != "${TGENV_CONFIG_DIR}/version" ];then
version_requested="$(cat "${version_file}" || true)"
fi
else
Expand All @@ -31,7 +31,7 @@ fi
version="$(tgenv-list | sed -E 's/^(\*| )? //g; s/ \(set by .+\)$//' | grep -e "${regex}" | head -n 1)"
[ -n "${version}" ] || error_and_die "No versions matching '${1}' found in local"

dst_path="${TGENV_ROOT}/versions/${version}"
dst_path="${TGENV_CONFIG_DIR}/versions/${version}"
if [ -f "${dst_path}/terragrunt" ]; then
info "Uninstall Terragrunt v${version}"
rm -r "${dst_path}"
Expand Down
6 changes: 3 additions & 3 deletions libexec/tgenv-use
Original file line number Diff line number Diff line change
Expand Up @@ -20,18 +20,18 @@ else
regex="^${version_requested}$"
fi

[ -d "${TGENV_ROOT}/versions" ] \
[ -d "${TGENV_CONFIG_DIR}/versions" ] \
|| error_and_die "No versions of terragrunt installed. Please install one with: tgenv install"

version="$(\ls "${TGENV_ROOT}/versions" \
version="$(\ls "${TGENV_CONFIG_DIR}/versions" \
| sort -t'.' -k 1nr,1 -k 2nr,2 -k 3nr,3 \
| grep -e "${regex}" \
| head -n 1
)"

[ -n "${version}" ] || error_and_die "No installed versions of terragrunt matched '${1}'"

target_path="${TGENV_ROOT}/versions/${version}"
target_path="${TGENV_CONFIG_DIR}/versions/${version}"
[ -f "${target_path}/terragrunt" ] \
|| error_and_die "Version directory for ${version} is present, but the terragrunt binary is not! Manual intervention required."
[ -x "${target_path}/terragrunt" ] \
Expand Down
2 changes: 1 addition & 1 deletion libexec/tgenv-version-file
Original file line number Diff line number Diff line change
Expand Up @@ -17,4 +17,4 @@ find_local_version_file() {
return 1
}

find_local_version_file "${TGENV_DIR}" || find_local_version_file "${HOME}" || echo "${TGENV_ROOT}/version"
find_local_version_file "${TGENV_DIR}" || find_local_version_file "${HOME}" || echo "${TGENV_CONFIG_DIR}/version"
Loading