Allow Passing Additional Flags to Restic

[sebt3]

Description

QNAP QuObjects is an "s3 compliant" service on QNAP nas machine.
It can be configured as using tls, but it use a self-signed certificate. Restic can use it but requires "--insecure-tls" argument to use it. Yet k8up doesn't allow to pass that argument to restic.
It can also but used as http, but then restic use a minio compatibility mode which doesnt suit QuObjects and restic fail with "The Content-MD5 you specified was invalid".

Additional Context

No response

Logs

using tls:
1.6723221876801217e+09  INFO    k8up.restic.restic.RepoInit.restic.stderr       Fatal: create repository at s3:https://s3.rennes.home:8443/solidite failed: client.BucketExists: Get "https://s3.rennes.home:8443/solidite/?location=": x509: certificate is not valid for any names, but wanted to match s3.rennes.home

using plain http:
1.6723222329661973e+09  INFO    k8up.restic.restic.RepoInit.restic.stderr       Fatal: create key in repository at s3:http://s3.rennes.home:80/solidite failed: client.PutObject: The Content-MD5 you specified was invalid

Expected Behavior

Using restic command-line it works :

export RESTIC_REPOSITORY="s3:https://s3.rennes.home:8443/solidite"
restic init --insecure-tls
created restic repository 493e88aa9c at s3:https://s3.rennes.home:8443/solidite
....

Please provide a way to propagate that --insecure-tls to restic

Acceptance Criteria

• Add a new field "additionalResticArgs" to all jobs and the schedule object
  ** These flags will be passed to all restic calls within K8up ad verbatim
• Document this new feature
  ** It should be clearly marked as an advanced configuration as we can't guarantee the correctness of all passed restic flags

Steps To Reproduce

No response

Version of K8up

v2.5.2

Version of Kubernetes

v1.25.3+k3s1

Distribution of Kubernetes

k3s

[ccremer]

Hi @sebt3
Thanks for opening this issue and the interest and sorry for the late response.
I've allowed myself to change the title to a generic feature request where one can specify to ignore TLS verification. However, TLS verification being enabled by default is by design and the skipping verification is not a bug.

As of now, there's no possibility to backup to a Restic repository with a self-signed certificate. You might be able to backup using an alternative backend, e.g. non-TLS S3 backend (http) or using the Restic REST server without TLS.

[ccremer]

It can also but used as http, but then restic use a minio compatibility mode which doesnt suit QuObjects and restic fail with "The Content-MD5 you specified was invalid".

Just extending up this. I'm not sure what exactly you mean with "minio compatibility mode".
Restic is indeed using the S3 client by Minio, regardless whether TLS is enabled or not. See here: https://github.com/restic/restic/blob/f5f13f6648528906e1ce73a5b0773a2a0f0a1509/internal/backend/s3/s3.go#L28. So, I'm not entirely convinced if a --insecure-tls flag is actually going to solve your "Content-MD5" issue with QNAP even with TLS, it sounds more like QNAP "QuObjects" itself is actually the incompatible part (or it is working but the error is completely unrelated to QNAP/Minio).

[sebt3]

As of now, there's no possibility to backup to a Restic repository with a self-signed certificate. You might be able to backup using an alternative backend, e.g. non-TLS S3 backend (http) or using the Restic REST server without TLS.

I went that road : installed minio on my NAS using the qpkg alternative store, and got backup working even before oping this issue ;)

So, I'm not entirely convinced if a --insecure-tls flag is actually going to solve your "Content-MD5" issue with QNAP even with TLS, it sounds more like QNAP "QuObjects" itself is actually the incompatible part (or it is working but the error is completely unrelated to QNAP/Minio).

I was more speculating than anything but since QuOjbects is close source, i'll side with your guess.

Still, being able to propagate restic specific arguments (including --insecure-tls) would be a great addition to k8up

[Kidswiss]

@sebt3

I'm chaning the issue a bit so that it's about a more generic "Allow Passing Additional Flags to Restic" use-case.

[rreckel]

How about using a similar environment variable as $BACKUP_RESTIC_OPTIONS, let's say $BACKUP_RESTIC_FLAGS and adding those flags to the restic command?

[hugosxm]

How about using a similar environment variable as $BACKUP_RESTIC_OPTIONS, let's say $BACKUP_RESTIC_FLAGS and adding those flags to the restic command?

This would totally solve my problem, and could be a answer to this one : #317

I need to exclude the "lost+found" folder from the backup because i use longhorn volume.

i would like the pass the "--exclude /**/lost+found" flag...

EDIT : I managed to use the BACKUP_COMMAND_RESTIC env var to pass the flag but the actual k8up binary seems to not support it yet.

[croissong]

@hugosxm how did you manage to use BACKUP_COMMAND_RESTIC to set --exclude? I can't figure it out 😕