added prune based on backup count to tar/rsync

[hpf3]

based on local tests of the functions it works ok, but if someone else can verify it that would be great, I don't often work in bash so im not 100% on things

also, I only did tar/rsync since I have even less of an idea of how rustic and rclone are doing stuff

[itzg]

This part

xargs -0 -I {} sh -c 'rm -v "{}"'

seems overly complicated. Maybe it could just be

Suggested change

	_find_extra_backups | xargs -0 -I {} sh -c 'rm -v "{}"' | log INFO
	_find_extra_backups | xargs -n 1 rm -v | log INFO

[itzg]

Please apply similar change here

[itzg]

...and here, if you're good with the suggestion above

[itzg]

Ah, missing the colon at the start of the line. It's an obscure bash syntax to perform a no-op.

[hpf3]

ive got a test script, and im getting issues with the changes

#!/bin/bash

# Configuration for the test
DEST_DIR="/tmp/test_backup_prune"
BACKUP_NAME="backup"
PRUNE_BACKUPS_COUNT=2
is_elem_in_array() {
  # $1 = element
  # All remaining arguments are array to search for the element in
  if [ "$#" -lt 2 ]; then
    log INTERNALERROR "Wrong number of arguments passed to is_elem_in_array function"
    return 2
  fi
  local element="${1}"
  shift
  local e
  for e; do
    if [ "${element}" == "${e}" ]; then
      return 0
    fi
  done
  return 1
}

log() {
  if [ "$#" -lt 1 ]; then
    log INTERNALERROR "Wrong number of arguments passed to log function"
    return 2
  fi
  local level="${1}"
  shift
  local valid_levels=(
    "INFO"
    "WARN"
    "ERROR"
    "INTERNALERROR"
  )
  if ! is_elem_in_array "${level}" "${valid_levels[@]}"; then
    log INTERNALERROR "Log level ${level} is not a valid level."
    return 2
  fi
  (
    # If any arguments are passed besides log level
    if [ "$#" -ge 1 ]; then
      # then use them as log message(s)
      <<<"${*}" cat -
    else
      # otherwise read log messages from standard input
      cat -
    fi
    if [ "${level}" == "INTERNALERROR" ]; then
      echo "Please report this: https://github.com/itzg/docker-mc-backup/issues"
    fi
  ) | awk -v level="${level}" '{ printf("%s %s %s\n", strftime("%FT%T%z"), level, $0); fflush(); }'
} >&2
# Create test backup directories with different modification times
rm -rf "${DEST_DIR}"
mkdir -p "${DEST_DIR}"
for i in {1..10}; do
  dir2=$(date -d "-${i} days" +"%Y%m%d-%H%M%S")
  dir1="${DEST_DIR}/${BACKUP_NAME}-$dir2"
  mkdir -p "${dir1}"
  mkdir -p "${dir1}/cake"
  touch "${dir1}/cake/frosting.test"
  touch "${dir1}/potato.test"
  touch "$dir1.tar"
  touch -d "-${i} days" "${dir1}"
done


# Function to find extra backups
_find_extra_backups() {
find "${DEST_DIR}" -maxdepth 1 -type d ! -path "${DEST_DIR}" -exec ls -t {} \+ | \
    tail -n +$((PRUNE_BACKUPS_COUNT + 1)) | \
     tr '\n' '\0'
}
_find_extra_backups2() {
  find "${DEST_DIR}" -maxdepth 1 -name "*.${backup_extension}" -exec ls -t {} \+ | \
    tail -n +$((PRUNE_BACKUPS_COUNT + 1))
  }

# Prune extra backup dirs
_find_extra_backups | xargs -n 1 | rm -v $@ | awk -v dest_dir="${DEST_DIR}" '
  {
    sub(/removed directory /, "")
    if ($0 !~ dest_dir "/.*/.*") {
      printf "Removing %s\n", $0
    }
  }'| log INFO
echo "removing tars"

_find_extra_backups2 | xargs -n 1 | rm -v $@| log INFO
# List remaining backup files
echo "Remaining backup files:"
ls "${DEST_DIR}"

# Clean up test environment
# Commenting out for debugging purposes
rm -rf "${DEST_DIR}"

output:

rm: missing operand
Try 'rm --help' for more information.
xargs: WARNING: a NUL character occurred in the input.  It cannot be passed through in the argument list.  Did you mean to use the --null option?
xargs: echo: terminated by signal 13
removing tars
rm: missing operand
Try 'rm --help' for more information.
xargs: echo: terminated by signal 13
Remaining backup files:
backup-20240627-124427	    backup-20240628-124427.tar	backup-20240630-124427	    backup-20240701-124427.tar	backup-20240703-124427	    backup-20240704-124427.tar	backup-20240706-124427
backup-20240627-124427.tar  backup-20240629-124427	backup-20240630-124427.tar  backup-20240702-124427	backup-20240703-124427.tar  backup-20240705-124427	backup-20240706-124427.tar
backup-20240628-124427	    backup-20240629-124427.tar	backup-20240701-124427	    backup-20240702-124427.tar	backup-20240704-124427	    backup-20240705-124427.tar

[itzg]

Are these latest changes ready to go? Thank you again for the extra testing with the suggested changes.

[hpf3]

should be