Restore from S3 snapshot via web identity credentials not working (Failed list objects in S3 bucket: InvalidBucketName)

[applike-ss]

Describe the bug
I have enabled s3 snapshotting to aws s3 (eu-central-1) on a test instance of dragonfly in my kubernetes cluster and grant it assume rights for a role via a service accounr (IRSA).
Snapshotting to the s3 bucket (same account) works just fine, however restoring upon pod reschedule does not.

Here's the output:

I20240918 12:05:31.724699 11 snapshot_storage.cc:190] Creating AWS S3 client; region=eu-central-1; https=true; endpoint=
I20240918 12:05:31.727696 11 credentials_provider_chain.cc:28] aws: disabled EC2 metadata
I20240918 12:05:31.775650 11 credentials_provider_chain.cc:36] aws: loaded credentials; provider=web-identity
I20240918 12:05:31.779950 11 snapshot_storage.cc:247] Load snapshot: Searching for snapshot in S3 path: s3://xyz-my-bucket-name-foo-bar/
E20240918 12:05:31.851604 1 server_family.cc:988] Failed to load snapshot: Failed list objects in S3 bucket: InvalidBucketName

Expected behavior
When dragonfly is able to write snapshots, it surely should be able to recover from them.

Environment (please complete the following information):

• OS: bottlerocket 1.27
• Kernel: linux 6.1.106
• Containerized?: Kubernetes
• Dragonfly Version: 1.22.2

Reproducible Code Snippet

# Dragonfly operator CR
apiVersion: dragonflydb.io/v1alpha1
kind: Dragonfly
metadata:
  name: dragonfly
spec:
  image: ghcr.io/dragonflydb/dragonfly:v1.22.2
  args:
    - '--cache_mode'
    - '--primary_port_http_enabled=true'
    - '--cluster_mode=emulated'
    - '--maxmemory=256M'
    - '--version_check=false'
    - '--dbfilename=dump' # ensure dragonfly does create only one snapshot
  serviceAccountName: dragonfly
  snapshot:
    cron: '*/5 * * * *'
    dir: 's3://xyz-my-bucket-name-foo-bar'
  resources:
    limits:
      cpu: 1
      memory: 320Mi
    requests:
      cpu: 100m
      memory: 320Mi
      ephemeral-storage: 1Gi
  replicas: 3
  affinity:
    podAntiAffinity:
      requiredDuringSchedulingIgnoredDuringExecution:
        - topologyKey: kubernetes.io/hostname
          labelSelector:
            matchExpressions:
              - key: app
                operator: In
                values:
                  - dragonfly

Let me know if there is any more information needed to resolve this.

[chakaz]

Thanks @applike-ss for reporting this.
@andydunstall - does this sound like something that you fixed in either #3717 or #3720 ?

[andydunstall]

@andydunstall - does this sound like something that you fixed in either #3717 or #3720 ?

Yep fixed in #3717 (it returns InvalidBucketName as the bucket name isn't being parsed correctly)

[applike-ss]

Sorry, i didn't see these when i was searching for them. So waiting for the next release then :-)

[chakaz]

Sure thing, next release should be soon, thanks again!