Add Bandwidth limiters to backups

[LichLord91]

I noticed in my Oracle Cloud VPS setup when the backup is ran it sometimes blows up my network latency on the server for a short period of time it'll go from 50ms to up to 300ms for a bit and even disconnect some far away bedrock players because of this spike.

I'm using rclone/restic and found some command to help limit the backup network bandwidth

Restic:
https://restic.readthedocs.io/en/latest/manual_rest.html#usage-help

--limit-download rate        limits downloads to a maximum rate in KiB/s. (default: unlimited)
--limit-upload rate          limits uploads to a maximum rate in KiB/s. (default: unlimited)

Rclone:
https://rclone.org/docs/#bwlimit-bandwidth-spec

--bwlimit=BANDWIDTH_SPEC

This option controls the bandwidth limit. For example

--bwlimit 10M

would mean limit the upload and download bandwidth to 10 MiB/s. NB this is bytes per second not bits per second. To use a single limit, specify the desired bandwidth in KiB/s, or use a suffix B|K|M|G|T|P. The default is 0 which means to not limit bandwidth.

I'm playing with the backup-loop.sh script right now to get this working so far changed the following bits of code

(this would not be the default values, i'd change them to the unlimited default values and then use a environment variable in the docker compose/dockerfile to set it.)

: "${RESTIC_BW_DL:=20480}"
: "${RESTIC_BW_UP:=5120}"
: "${RCLONE_BW_LIMIT:=10m:20m}" 

  if ! command rclone copy "${outFile}" "${RCLONE_REMOTE}:${RCLONE_DEST_DIR}" --bwlimit "${RCLONE_BW_LIMIT}" ; then

  backup() {
    log INFO "Backing up content in ${SRC_DIR} as host ${RESTIC_HOSTNAME}"
    args=(
      --host "${RESTIC_HOSTNAME}"
      --limit-download "${RESTIC_BW_DL}" #change made
      --limit-upload "${RESTIC_BW_UP}" #change made
    )
    if isDebug || isTrue "$RESTIC_VERBOSE"; then
      args+=(-vv)
    fi
    (cd "$SRC_DIR" &&
          command restic backup "${args[@]}" "${restic_tags_arguments[@]}" "${excludes[@]}" "${includes_patterns[@]}" | log INFO
    )
  }

Just starting to test this but figured I'd put this in as and Idea and may be other backup methods that have this argument as well that may have a similar feature

[LichLord91]

There's also a bandwidth limiter per file for rclone that's interesting

https://rclone.org/docs/#bwlimit-file-bandwidth-spec

This option controls per file bandwidth limit. For the options see the --bwlimit flag.

For example use this to allow no transfers to be faster than 1 MiB/s

--bwlimit-file 1

This can be used in conjunction with --bwlimit.

Note that if a schedule is provided the file will use the schedule in effect at the start of the transfer.

[itzg]

Great research and progress! Feel free to PR the changes when you're ready.

[LichLord91]

Thanks! So far its proved fruitful i run backups every 15minutes and whatever limit i've set between restic and rclone has stopped those lag spikes even with the user connecting from UK using bedrock. So it appears to be working! I'll play around with it more