Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

After some time mount disappears #33

Open
Dmitry-Filippov-Rival opened this issue Jun 3, 2024 · 1 comment
Open

After some time mount disappears #33

Dmitry-Filippov-Rival opened this issue Jun 3, 2024 · 1 comment

Comments

@Dmitry-Filippov-Rival
Copy link

Hi, I am using this repository as a reference in my project. I have a pod with an s3 mount running an FFMPEG microservice that reads a file from S3 and then uploads the transcoded HLS stream. On small files this works just fine, but when the files are big and take more than 5 minutes to transcode, at some point the transcoder pod fails liveness check and gets restarted, losing the transcoding progress. Do you know what the issue might be? Here are my YAML files for reference:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: rival-transcoder-deployment
  labels:
    app: rival-transcoder
spec:
  replicas: 1
  selector:
    matchLabels:
      app: rival-transcoder
  template:
    metadata:
      labels:
        app: rival-transcoder
    spec:
      serviceAccountName: rival-server-service-account
      affinity:
        nodeAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            nodeSelectorTerms:
            - matchExpressions:
              - key: dedicated
                operator: NotIn
                values:
                - infra
      containers:
        - name: rival-transcoder
          image: our trancoder image
          securityContext:
            privileged: true
          volumeMounts:
            - name: mntdatas3fs
              mountPath: /var/s3
              mountPropagation: Bidirectional
          livenessProbe:
            exec:
              command: ["ls", "/var/s3"]
            failureThreshold: 3
            initialDelaySeconds: 10
            periodSeconds: 5
            successThreshold: 1
            timeoutSeconds: 1
          resources:
            requests:
              memory: "24Gi"
              cpu: "1"
            limits:
              memory: "24Gi"
              cpu: "4"
          ports:
          - containerPort: 1050
          imagePullPolicy: Always
          env: 
          - name: API_URL
            value: {{ .Values.api_url }}
          - name: AWS_S3_BUCKET
            valueFrom:
              secretKeyRef:
                name: s3-secret
                key: s3-bucket
          - name: AWS_REGION
            value: {{ .Values.aws_region }}
          - name: RABBITMQ_HOST
            valueFrom:
              secretKeyRef:
                name: rabbitmq-default-user
                key: host
          - name: RABBITMQ_PORT
            valueFrom:
              secretKeyRef:
                name: rabbitmq-default-user
                key: port
          - name: RABBITMQ_USER
            valueFrom:
              secretKeyRef:
                name: rabbitmq-default-user
                key: username
          - name: RABBITMQ_PASSWORD
            valueFrom:
              secretKeyRef:
                name: rabbitmq-default-user
                key: password
          - name: env
            value: "{{ .Values.environment }}" 
      volumes:
        - name: mntdatas3fs
          hostPath:
            path: /mnt/data-s3-fs/root
      imagePullSecrets:
      - name: ghcr-login-secret
---
apiVersion: v1
kind: Service
metadata:
  name: rival-transcoder-service
  labels:
    run: rival-transcoder
spec:
  type: ClusterIP
  selector:
    app: rival-transcoder
  ports:
    - protocol: TCP
      port: 1050

apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: s3-provider
  labels:
    app: s3-provider
spec:
  selector:
    matchLabels:
      app: s3-provider
  template:
    metadata:
      labels:
        app: s3-provider
    spec:
      initContainers:
      - name: init-myservice
        image: bash
        command: ['bash', '-c', 'umount -l /mnt/data-s3-fs/root ; true']
        securityContext:
          privileged: true
          capabilities:
            add:
            - SYS_ADMIN
        envFrom:
        - secretRef:
            name: s3-credentials
        volumeMounts:
        - name: devfuse
          mountPath: /dev/fuse
        - name: mntdatas3fs-init
          mountPath: /mnt
          mountPropagation: Bidirectional
      containers:
      - name: s3fuse
        image: ghcr.io/rival-xr/rival-s3-mount:1.0 
        imagePullPolicy: Always
        lifecycle:
          preStop:
            exec:
              command: ["/bin/sh","-c","umount -f /var/s3/root"]
        securityContext:
          privileged: true
          capabilities:
            add:
            - SYS_ADMIN
        envFrom:
        - secretRef:
            name: s3-credentials
        env:
        - name: MNT_POINT
          value: /var/s3/root
        volumeMounts:
        - name: devfuse
          mountPath: /dev/fuse
        - name: mntdatas3fs
          mountPath: /var/s3/root
          mountPropagation: Bidirectional
      volumes:
      - name: devfuse
        hostPath:
          path: /dev/fuse
      - name: mntdatas3fs
        hostPath:
          type: DirectoryOrCreate
          path: /mnt/data-s3-fs/root
      - name: mntdatas3fs-init
        hostPath:
          type: DirectoryOrCreate
          path: /mnt
      imagePullSecrets:
      - name: ghcr-login-secret

Docker image is built as per instructions in this repo.
@Dmitry-Filippov-Rival
Copy link
Author

Dmitry-Filippov-Rival commented Jun 3, 2024 •
edited
Loading

Here are the logs of s3 provider pod if it helps. It seems like the issue is the "flush" at the end of these logs. That seems to temporaly coinside with the probe fail.

2024-06-03T15:39:25.141Z [INF]       curl_util.cpp:url_to_host(334): url is https://s3.amazonaws.com
2024-06-03T15:39:25.213Z [INF]       curl.cpp:RequestPerform(2572): HTTP response code 404 was returned, returning ENOENT
2024-06-03T15:39:25.257Z [INF]       curl.cpp:insertV4Headers(2892): computing signature [HEAD] [/4/] [] []
2024-06-03T15:39:25.257Z [INF]       curl_util.cpp:url_to_host(334): url is https://s3.amazonaws.com
2024-06-03T15:39:25.282Z [INF]       curl.cpp:RequestPerform(2572): HTTP response code 404 was returned, returning ENOENT
2024-06-03T15:39:25.289Z [INF] s3fs.cpp:s3fs_getattr(1026): [path=/1]
2024-06-03T15:39:25.294Z [INF] s3fs.cpp:s3fs_getattr(1026): [path=/123456]
2024-06-03T15:39:25.317Z [INF] s3fs.cpp:s3fs_getattr(1026): [path=/2]
2024-06-03T15:39:25.329Z [INF] s3fs.cpp:s3fs_getattr(1026): [path=/aba123]
2024-06-03T15:39:25.329Z [INF]       cache.cpp:DelStat(596): delete stat cache entry[path=/aba123/]
2024-06-03T15:39:25.329Z [INF]       curl.cpp:HeadRequest(3299): [tpath=/aba123]
2024-06-03T15:39:25.329Z [INF]       curl.cpp:PreHeadRequest(3259): [tpath=/aba123][bpath=][save=][sseckeypos=18446744073709551615]
2024-06-03T15:39:25.329Z [INF]       curl_util.cpp:prepare_url(257): URL is https://s3.amazonaws.com/rival-hls-testing/aba123
2024-06-03T15:39:25.337Z [INF]       curl_util.cpp:prepare_url(290): URL changed is https://rival-hls-testing.s3.amazonaws.com/aba123
2024-06-03T15:39:25.337Z [INF]       curl.cpp:insertV4Headers(2892): computing signature [HEAD] [/aba123] [] []
2024-06-03T15:39:25.337Z [INF]       curl_util.cpp:url_to_host(334): url is https://s3.amazonaws.com
2024-06-03T15:39:25.413Z [INF]       curl.cpp:RequestPerform(2572): HTTP response code 404 was returned, returning ENOENT
2024-06-03T15:39:25.413Z [INF]       curl.cpp:HeadRequest(3299): [tpath=/aba123/]
2024-06-03T15:39:25.413Z [INF]       curl.cpp:PreHeadRequest(3259): [tpath=/aba123/][bpath=][save=][sseckeypos=18446744073709551615]
2024-06-03T15:39:25.413Z [INF]       curl_util.cpp:prepare_url(257): URL is https://s3.amazonaws.com/rival-hls-testing/aba123/
2024-06-03T15:39:25.413Z [INF]       curl_util.cpp:prepare_url(290): URL changed is https://rival-hls-testing.s3.amazonaws.com/aba123/
2024-06-03T15:39:25.414Z [INF]       curl.cpp:insertV4Headers(2892): computing signature [HEAD] [/aba123/] [] []
2024-06-03T15:39:25.414Z [INF]       curl_util.cpp:url_to_host(334): url is https://s3.amazonaws.com
2024-06-03T15:39:25.529Z [INF]       curl.cpp:RequestPerform(2572): HTTP response code 404 was returned, returning ENOENT
2024-06-03T15:39:25.529Z [INF]   s3fs.cpp:list_bucket(3434): [path=/aba123]
2024-06-03T15:39:25.529Z [INF]       curl.cpp:ListBucketRequest(3749): [tpath=/aba123]
2024-06-03T15:39:25.530Z [INF]       curl_util.cpp:prepare_url(257): URL is https://s3.amazonaws.com/rival-hls-testing?delimiter=/&max-keys=2&prefix=aba123/
2024-06-03T15:39:25.530Z [INF]       curl_util.cpp:prepare_url(290): URL changed is https://rival-hls-testing.s3.amazonaws.com?delimiter=/&max-keys=2&prefix=aba123/
   NODEID: 17
   unique: 1788328, success, outsize: 144
unique: 1788330, opcode: LOOKUP (1), nodeid: 1, insize: 50, pid: 0
LOOKUP /abccde239
getattr /abccde239
   NODEID: 15
   unique: 1788330, success, outsize: 144
unique: 1788332, opcode: LOOKUP (1), nodeid: 1, insize: 45, pid: 0
LOOKUP /h264
getattr /h264
2024-06-03T15:39:25.530Z [INF]       curl.cpp:insertV4Headers(2892): computing signature [GET] [/] [delimiter=/&max-keys=2&prefix=aba123/] []
2024-06-03T15:39:25.530Z [INF]       curl_util.cpp:url_to_host(334): url is https://s3.amazonaws.com
2024-06-03T15:39:25.669Z [INF]       curl.cpp:RequestPerform(2520): HTTP response code 200
2024-06-03T15:39:25.670Z [ERR] s3fs_xml.cpp:get_base_exp(114): marker_xp->nodesetval is empty.
2024-06-03T15:39:25.670Z [INF]       cache.cpp:AddStat(342): add stat cache entry[path=/aba123/]
2024-06-03T15:39:25.745Z [INF] s3fs.cpp:s3fs_getattr(1026): [path=/abccde239]
2024-06-03T15:39:25.746Z [INF] s3fs.cpp:s3fs_getattr(1026): [path=/h264]
2024-06-03T15:39:25.746Z [INF]       cache.cpp:DelStat(596): delete stat cache entry[path=/h264/]
2024-06-03T15:39:25.746Z [INF]       curl.cpp:HeadRequest(3299): [tpath=/h264]
2024-06-03T15:39:25.777Z [INF]       curl.cpp:PreHeadRequest(3259): [tpath=/h264][bpath=][save=][sseckeypos=18446744073709551615]
2024-06-03T15:39:25.777Z [INF]       curl_util.cpp:prepare_url(257): URL is https://s3.amazonaws.com/rival-hls-testing/h264
2024-06-03T15:39:25.797Z [INF]       curl_util.cpp:prepare_url(290): URL changed is https://rival-hls-testing.s3.amazonaws.com/h264
2024-06-03T15:39:25.797Z [INF]       curl.cpp:insertV4Headers(2892): computing signature [HEAD] [/h264] [] []
2024-06-03T15:39:25.797Z [INF]       curl_util.cpp:url_to_host(334): url is https://s3.amazonaws.com
2024-06-03T15:39:25.849Z [INF]       curl.cpp:RequestPerform(2572): HTTP response code 404 was returned, returning ENOENT
2024-06-03T15:39:25.849Z [INF]       curl.cpp:HeadRequest(3299): [tpath=/h264/]
2024-06-03T15:39:25.849Z [INF]       curl.cpp:PreHeadRequest(3259): [tpath=/h264/][bpath=][save=][sseckeypos=18446744073709551615]
2024-06-03T15:39:25.849Z [INF]       curl_util.cpp:prepare_url(257): URL is https://s3.amazonaws.com/rival-hls-testing/h264/
2024-06-03T15:39:25.849Z [INF]       curl_util.cpp:prepare_url(290): URL changed is https://rival-hls-testing.s3.amazonaws.com/h264/
unique: 1788333, opcode: INTERRUPT (36), nodeid: 0, insize: 48, pid: 0
INTERRUPT: 1788332
2024-06-03T15:39:25.849Z [INF]       curl.cpp:insertV4Headers(2892): computing signature [HEAD] [/h264/] [] []
2024-06-03T15:39:25.849Z [INF]       curl_util.cpp:url_to_host(334): url is https://s3.amazonaws.com
2024-06-03T15:39:26.037Z [INF]       curl.cpp:RequestPerform(2572): HTTP response code 404 was returned, returning ENOENT
2024-06-03T15:39:26.037Z [INF]   s3fs.cpp:list_bucket(3434): [path=/h264]
2024-06-03T15:39:26.037Z [INF]       curl.cpp:ListBucketRequest(3749): [tpath=/h264]
2024-06-03T15:39:26.038Z [INF]       curl_util.cpp:prepare_url(257): URL is https://s3.amazonaws.com/rival-hls-testing?delimiter=/&max-keys=2&prefix=h264/
2024-06-03T15:39:26.038Z [INF]       curl_util.cpp:prepare_url(290): URL changed is https://rival-hls-testing.s3.amazonaws.com?delimiter=/&max-keys=2&prefix=h264/
2024-06-03T15:39:26.038Z [INF]       curl.cpp:insertV4Headers(2892): computing signature [GET] [/] [delimiter=/&max-keys=2&prefix=h264/] []
2024-06-03T15:39:26.038Z [INF]       curl_util.cpp:url_to_host(334): url is https://s3.amazonaws.com
   NODEID: 9
   unique: 1788332, success, outsize: 144
unique: 1788334, opcode: RELEASEDIR (29), nodeid: 1, insize: 64, pid: 0
   unique: 1788334, success, outsize: 16
unique: 1788336, opcode: READ (15), nodeid: 16, insize: 80, pid: 0
read[2] 131072 bytes from 38338560 flags: 0x8000
unique: 1788338, opcode: READ (15), nodeid: 16, insize: 80, pid: 0
read[2] 131072 bytes from 38469632 flags: 0x8000
   read[2] 131072 bytes from 38338560
   unique: 1788336, success, outsize: 131088
unique: 1788340, opcode: READ (15), nodeid: 16, insize: 80, pid: 0
read[2] 131072 bytes from 38600704 flags: 0x8000
   read[2] 131072 bytes from 38600704
   unique: 1788340, success, outsize: 131088
   read[2] 131072 bytes from 38469632
   unique: 1788338, success, outsize: 131088
unique: 1788342, opcode: READ (15), nodeid: 16, insize: 80, pid: 0
read[2] 131072 bytes from 38731776 flags: 0x8000
   read[2] 131072 bytes from 38731776
   unique: 1788342, success, outsize: 131088
unique: 1788344, opcode: READ (15), nodeid: 16, insize: 80, pid: 0
read[2] 131072 bytes from 38862848 flags: 0x8000
unique: 1788346, opcode: READ (15), nodeid: 16, insize: 80, pid: 0
read[2] 131072 bytes from 38993920 flags: 0x8000
   read[2] 131072 bytes from 38862848
   unique: 1788344, success, outsize: 131088
unique: 1788348, opcode: READ (15), nodeid: 16, insize: 80, pid: 0
read[2] 131072 bytes from 39124992 flags: 0x8000
   read[2] 131072 bytes from 39124992
   unique: 1788348, success, outsize: 131088
   read[2] 131072 bytes from 38993920
   unique: 1788346, success, outsize: 131088
unique: 1788350, opcode: READ (15), nodeid: 16, insize: 80, pid: 0
read[2] 131072 bytes from 39256064 flags: 0x8000
   read[2] 131072 bytes from 39256064
   unique: 1788350, success, outsize: 131088
unique: 1788354, opcode: READ (15), nodeid: 16, insize: 80, pid: 0
read[2] 131072 bytes from 39518208 flags: 0x8000
   read[2] 131072 bytes from 39518208
   unique: 1788354, success, outsize: 131088
unique: 1788352, opcode: READ (15), nodeid: 16, insize: 80, pid: 0
read[2] 131072 bytes from 39387136 flags: 0x8000
   read[2] 131072 bytes from 39387136
   unique: 1788352, success, outsize: 131088
unique: 1788356, opcode: GETATTR (3), nodeid: 1, insize: 56, pid: 0
getattr /
   unique: 1788356, success, outsize: 120
unique: 1788358, opcode: OPENDIR (27), nodeid: 1, insize: 48, pid: 0
opendir flags: 0x18000 /
   opendir[0] flags: 0x18000 /
   unique: 1788358, success, outsize: 32
unique: 1788360, opcode: READDIR (28), nodeid: 1, insize: 80, pid: 0
readdir[0] from 0
2024-06-03T15:39:26.214Z [INF]       curl.cpp:RequestPerform(2520): HTTP response code 200
2024-06-03T15:39:26.215Z [ERR] s3fs_xml.cpp:get_base_exp(114): marker_xp->nodesetval is empty.
2024-06-03T15:39:26.215Z [INF]       cache.cpp:AddStat(342): add stat cache entry[path=/h264/]
2024-06-03T15:39:29.985Z [INF] s3fs.cpp:s3fs_getattr(1026): [path=/]
2024-06-03T15:39:30.109Z [INF] s3fs.cpp:s3fs_opendir(3139): [path=/][flags=0x18000]
2024-06-03T15:39:30.205Z [INF] s3fs.cpp:s3fs_readdir(3390): [path=/]
2024-06-03T15:39:30.205Z [INF]   s3fs.cpp:list_bucket(3434): [path=/]
2024-06-03T15:39:30.205Z [INF]       curl.cpp:ListBucketRequest(3749): [tpath=/]
2024-06-03T15:39:30.205Z [INF]       curl_util.cpp:prepare_url(257): URL is https://s3.amazonaws.com/rival-hls-testing?delimiter=/&max-keys=1000&prefix=
2024-06-03T15:39:30.205Z [INF]       curl_util.cpp:prepare_url(290): URL changed is https://rival-hls-testing.s3.amazonaws.com?delimiter=/&max-keys=1000&prefix=
2024-06-03T15:39:30.205Z [INF]       curl.cpp:insertV4Headers(2892): computing signature [GET] [/] [delimiter=/&max-keys=1000&prefix=] []
2024-06-03T15:39:30.205Z [INF]       curl_util.cpp:url_to_host(334): url is https://s3.amazonaws.com
2024-06-03T15:39:30.293Z [INF]       curl.cpp:RequestPerform(2520): HTTP response code 200
2024-06-03T15:39:30.294Z [INF]   s3fs.cpp:readdir_multi_head(3251): [path=/][list=0]
2024-06-03T15:39:30.294Z [INF]       curl.cpp:PreHeadRequest(3259): [tpath=/3/][bpath=3/][save=/3/][sseckeypos=18446744073709551615]
2024-06-03T15:39:30.294Z [INF]       curl_util.cpp:prepare_url(257): URL is https://s3.amazonaws.com/rival-hls-testing/3/
2024-06-03T15:39:30.294Z [INF]       curl_util.cpp:prepare_url(290): URL changed is https://rival-hls-testing.s3.amazonaws.com/3/
2024-06-03T15:39:30.294Z [INF]       curl.cpp:PreHeadRequest(3259): [tpath=/4/][bpath=4/][save=/4/][sseckeypos=18446744073709551615]
2024-06-03T15:39:30.294Z [INF]       curl_util.cpp:prepare_url(257): URL is https://s3.amazonaws.com/rival-hls-testing/4/
2024-06-03T15:39:30.294Z [INF]       curl_util.cpp:prepare_url(290): URL changed is https://rival-hls-testing.s3.amazonaws.com/4/
2024-06-03T15:39:30.294Z [INF]       cache.cpp:DelStat(596): delete stat cache entry[path=/h265/]
2024-06-03T15:39:30.294Z [INF]       curl.cpp:PreHeadRequest(3259): [tpath=/h265/][bpath=h265/][save=/h265/][sseckeypos=18446744073709551615]
2024-06-03T15:39:30.294Z [INF]       curl_util.cpp:prepare_url(257): URL is https://s3.amazonaws.com/rival-hls-testing/h265/
2024-06-03T15:39:30.294Z [INF]       curl_util.cpp:prepare_url(290): URL changed is https://rival-hls-testing.s3.amazonaws.com/h265/
2024-06-03T15:39:30.294Z [INF]       cache.cpp:DelStat(596): delete stat cache entry[path=/hanger-h264/]
2024-06-03T15:39:30.294Z [INF]       curl.cpp:PreHeadRequest(3259): [tpath=/hanger-h264/][bpath=hanger-h264/][save=/hanger-h264/][sseckeypos=18446744073709551615]
2024-06-03T15:39:30.294Z [INF]       curl_util.cpp:prepare_url(257): URL is https://s3.amazonaws.com/rival-hls-testing/hanger-h264/
2024-06-03T15:39:30.294Z [INF]       curl_util.cpp:prepare_url(290): URL changed is https://rival-hls-testing.s3.amazonaws.com/hanger-h264/
2024-06-03T15:39:30.294Z [INF]       cache.cpp:DelStat(596): delete stat cache entry[path=/hanger/]
2024-06-03T15:39:30.294Z [INF]       curl.cpp:PreHeadRequest(3259): [tpath=/hanger/][bpath=hanger/][save=/hanger/][sseckeypos=18446744073709551615]
2024-06-03T15:39:30.294Z [INF]       curl_util.cpp:prepare_url(257): URL is https://s3.amazonaws.com/rival-hls-testing/hanger/
2024-06-03T15:39:30.294Z [INF]       curl_util.cpp:prepare_url(290): URL changed is https://rival-hls-testing.s3.amazonaws.com/hanger/
2024-06-03T15:39:30.341Z [INF]       curl_multi.cpp:Request(324): [count=5]
2024-06-03T15:39:30.345Z [INF]       curl.cpp:insertV4Headers(2892): computing signature [HEAD] [/4/] [] []
2024-06-03T15:39:30.345Z [INF]       curl_util.cpp:url_to_host(334): url is https://s3.amazonaws.com
2024-06-03T15:39:30.429Z [INF]       curl.cpp:insertV4Headers(2892): computing signature [HEAD] [/h265/] [] []
2024-06-03T15:39:30.429Z [INF]       curl_util.cpp:url_to_host(334): url is https://s3.amazonaws.com
unique: 1788361, opcode: INTERRUPT (36), nodeid: 0, insize: 48, pid: 0
INTERRUPT: 1788360
2024-06-03T15:39:30.429Z [INF]       curl.cpp:RequestPerform(2572): HTTP response code 404 was returned, returning ENOENT
2024-06-03T15:39:30.625Z [INF]       curl.cpp:insertV4Headers(2892): computing signature [HEAD] [/hanger-h264/] [] []
2024-06-03T15:39:30.625Z [INF]       curl_util.cpp:url_to_host(334): url is https://s3.amazonaws.com
2024-06-03T15:39:30.713Z [INF]       curl.cpp:RequestPerform(2572): HTTP response code 404 was returned, returning ENOENT
2024-06-03T15:39:31.215Z [INF]       curl.cpp:insertV4Headers(2892): computing signature [HEAD] [/hanger/] [] []
2024-06-03T15:39:31.215Z [INF]       curl_util.cpp:url_to_host(334): url is https://s3.amazonaws.com
2024-06-03T15:39:31.216Z [INF]       curl.cpp:insertV4Headers(2892): computing signature [HEAD] [/3/] [] []
2024-06-03T15:39:31.216Z [INF]       curl_util.cpp:url_to_host(334): url is https://s3.amazonaws.com
2024-06-03T15:39:31.249Z [INF]       curl.cpp:RequestPerform(2572): HTTP response code 404 was returned, returning ENOENT
   unique: 1788360, success, outsize: 480
unique: 1788362, opcode: RELEASEDIR (29), nodeid: 1, insize: 64, pid: 0
   unique: 1788362, success, outsize: 16
unique: 1788364, opcode: GETATTR (3), nodeid: 1, insize: 56, pid: 0
getattr /
   unique: 1788364, success, outsize: 120
unique: 1788366, opcode: OPENDIR (27), nodeid: 1, insize: 48, pid: 0
opendir flags: 0x18000 /
   opendir[0] flags: 0x18000 /
   unique: 1788366, success, outsize: 32
unique: 1788368, opcode: READDIR (28), nodeid: 1, insize: 80, pid: 0
readdir[0] from 0
2024-06-03T15:39:35.813Z [INF]       curl.cpp:RequestPerform(2572): HTTP response code 404 was returned, returning ENOENT
2024-06-03T15:39:36.046Z [INF]       curl.cpp:RequestPerform(2572): HTTP response code 404 was returned, returning ENOENT
2024-06-03T15:39:36.649Z [INF] s3fs.cpp:s3fs_getattr(1026): [path=/]
2024-06-03T15:39:36.665Z [INF] s3fs.cpp:s3fs_opendir(3139): [path=/][flags=0x18000]
2024-06-03T15:39:36.681Z [INF] s3fs.cpp:s3fs_readdir(3390): [path=/]
2024-06-03T15:39:36.681Z [INF]   s3fs.cpp:list_bucket(3434): [path=/]
2024-06-03T15:39:36.681Z [INF]       curl.cpp:ListBucketRequest(3749): [tpath=/]
2024-06-03T15:39:36.681Z [INF]       curl_util.cpp:prepare_url(257): URL is https://s3.amazonaws.com/rival-hls-testing?delimiter=/&max-keys=1000&prefix=
2024-06-03T15:39:36.681Z [INF]       curl_util.cpp:prepare_url(290): URL changed is https://rival-hls-testing.s3.amazonaws.com?delimiter=/&max-keys=1000&prefix=
2024-06-03T15:39:36.681Z [INF]       curl.cpp:insertV4Headers(2892): computing signature [GET] [/] [delimiter=/&max-keys=1000&prefix=] []
2024-06-03T15:39:36.681Z [INF]       curl_util.cpp:url_to_host(334): url is https://s3.amazonaws.com
2024-06-03T15:39:36.817Z [INF]       curl.cpp:RequestPerform(2520): HTTP response code 200
2024-06-03T15:39:36.817Z [INF]   s3fs.cpp:readdir_multi_head(3251): [path=/][list=0]
2024-06-03T15:39:36.817Z [INF]       curl.cpp:PreHeadRequest(3259): [tpath=/3/][bpath=3/][save=/3/][sseckeypos=18446744073709551615]
2024-06-03T15:39:36.817Z [INF]       curl_util.cpp:prepare_url(257): URL is https://s3.amazonaws.com/rival-hls-testing/3/
2024-06-03T15:39:36.817Z [INF]       curl_util.cpp:prepare_url(290): URL changed is https://rival-hls-testing.s3.amazonaws.com/3/
2024-06-03T15:39:36.817Z [INF]       curl.cpp:PreHeadRequest(3259): [tpath=/4/][bpath=4/][save=/4/][sseckeypos=18446744073709551615]
2024-06-03T15:39:36.817Z [INF]       curl_util.cpp:prepare_url(257): URL is https://s3.amazonaws.com/rival-hls-testing/4/
2024-06-03T15:39:36.817Z [INF]       curl_util.cpp:prepare_url(290): URL changed is https://rival-hls-testing.s3.amazonaws.com/4/
2024-06-03T15:39:36.817Z [INF]       curl.cpp:PreHeadRequest(3259): [tpath=/h265/][bpath=h265/][save=/h265/][sseckeypos=18446744073709551615]
2024-06-03T15:39:36.817Z [INF]       curl_util.cpp:prepare_url(257): URL is https://s3.amazonaws.com/rival-hls-testing/h265/
2024-06-03T15:39:36.817Z [INF]       curl_util.cpp:prepare_url(290): URL changed is https://rival-hls-testing.s3.amazonaws.com/h265/
2024-06-03T15:39:36.818Z [INF]       curl.cpp:PreHeadRequest(3259): [tpath=/hanger-h264/][bpath=hanger-h264/][save=/hanger-h264/][sseckeypos=18446744073709551615]
2024-06-03T15:39:36.818Z [INF]       curl_util.cpp:prepare_url(257): URL is https://s3.amazonaws.com/rival-hls-testing/hanger-h264/
2024-06-03T15:39:36.818Z [INF]       curl_util.cpp:prepare_url(290): URL changed is https://rival-hls-testing.s3.amazonaws.com/hanger-h264/
2024-06-03T15:39:36.818Z [INF]       curl.cpp:PreHeadRequest(3259): [tpath=/hanger/][bpath=hanger/][save=/hanger/][sseckeypos=18446744073709551615]
2024-06-03T15:39:36.818Z [INF]       curl_util.cpp:prepare_url(257): URL is https://s3.amazonaws.com/rival-hls-testing/hanger/
2024-06-03T15:39:36.818Z [INF]       curl_util.cpp:prepare_url(290): URL changed is https://rival-hls-testing.s3.amazonaws.com/hanger/
2024-06-03T15:39:36.818Z [INF]       curl_multi.cpp:Request(324): [count=5]
2024-06-03T15:39:36.973Z [INF]       curl.cpp:insertV4Headers(2892): computing signature [HEAD] [/h265/] [] []
2024-06-03T15:39:36.973Z [INF]       curl_util.cpp:url_to_host(334): url is https://s3.amazonaws.com
2024-06-03T15:39:36.994Z [INF]       curl.cpp:insertV4Headers(2892): computing signature [HEAD] [/4/] [] []
2024-06-03T15:39:36.994Z [INF]       curl_util.cpp:url_to_host(334): url is https://s3.amazonaws.com
2024-06-03T15:39:37.033Z [INF]       curl.cpp:insertV4Headers(2892): computing signature [HEAD] [/3/] [] []
2024-06-03T15:39:37.033Z [INF]       curl_util.cpp:url_to_host(334): url is https://s3.amazonaws.com
2024-06-03T15:39:37.081Z [INF]       curl.cpp:RequestPerform(2572): HTTP response code 404 was returned, returning ENOENT
unique: 1788369, opcode: INTERRUPT (36), nodeid: 0, insize: 48, pid: 0
INTERRUPT: 1788368
   unique: 1788368, success, outsize: 480
unique: 1788370, opcode: RELEASEDIR (29), nodeid: 1, insize: 64, pid: 0
   unique: 1788370, success, outsize: 16
unique: 1788372, opcode: READ (15), nodeid: 16, insize: 80, pid: 0
read[2] 131072 bytes from 39649280 flags: 0x8000
   read[2] 131072 bytes from 39649280
   unique: 1788372, success, outsize: 131088
unique: 1788374, opcode: READ (15), nodeid: 16, insize: 80, pid: 0
read[2] 131072 bytes from 39780352 flags: 0x8000
unique: 1788376, opcode: READ (15), nodeid: 16, insize: 80, pid: 0
read[2] 131072 bytes from 39911424 flags: 0x8000
   read[2] 131072 bytes from 39780352
   unique: 1788374, success, outsize: 131088
   read[2] 131072 bytes from 39911424
   unique: 1788376, success, outsize: 131088
unique: 1788378, opcode: READ (15), nodeid: 16, insize: 80, pid: 0
read[2] 131072 bytes from 40042496 flags: 0x8000
   read[2] 131072 bytes from 40042496
   unique: 1788378, success, outsize: 131088
unique: 1788382, opcode: READ (15), nodeid: 16, insize: 80, pid: 0
read[2] 131072 bytes from 40304640 flags: 0x8000
unique: 1788380, opcode: READ (15), nodeid: 16, insize: 80, pid: 0
read[2] 131072 bytes from 40173568 flags: 0x8000
   read[2] 131072 bytes from 40304640
   unique: 1788382, success, outsize: 131088
   read[2] 131072 bytes from 40173568
   unique: 1788380, success, outsize: 131088
unique: 1788384, opcode: READ (15), nodeid: 16, insize: 80, pid: 0
read[2] 131072 bytes from 40435712 flags: 0x8000
   read[2] 131072 bytes from 40435712
   unique: 1788384, success, outsize: 131088
unique: 1788388, opcode: READ (15), nodeid: 16, insize: 80, pid: 0
read[2] 131072 bytes from 40697856 flags: 0x8000
   read[2] 131072 bytes from 40697856
   unique: 1788388, success, outsize: 131088
unique: 1788386, opcode: READ (15), nodeid: 16, insize: 80, pid: 0
read[2] 131072 bytes from 40566784 flags: 0x8000
   read[2] 131072 bytes from 40566784
   unique: 1788386, success, outsize: 131088
unique: 1788390, opcode: READ (15), nodeid: 16, insize: 80, pid: 0
read[2] 131072 bytes from 40828928 flags: 0x8000
   read[2] 131072 bytes from 40828928
   unique: 1788390, success, outsize: 131088
unique: 1788392, opcode: READ (15), nodeid: 16, insize: 80, pid: 0
read[2] 131072 bytes from 40960000 flags: 0x8000
unique: 1788394, opcode: READ (15), nodeid: 16, insize: 80, pid: 0
read[2] 131072 bytes from 41091072 flags: 0x8000
   read[2] 131072 bytes from 40960000
   unique: 1788392, success, outsize: 131088
unique: 1788396, opcode: READ (15), nodeid: 16, insize: 80, pid: 0
read[2] 131072 bytes from 41222144 flags: 0x8000
   read[2] 131072 bytes from 41222144
   unique: 1788396, success, outsize: 131088
   read[2] 131072 bytes from 41091072
   unique: 1788394, success, outsize: 131088
unique: 1788398, opcode: READ (15), nodeid: 16, insize: 80, pid: 0
read[2] 131072 bytes from 41353216 flags: 0x8000
   read[2] 131072 bytes from 41353216
   unique: 1788398, success, outsize: 131088
unique: 1788400, opcode: READ (15), nodeid: 16, insize: 80, pid: 0
read[2] 131072 bytes from 41484288 flags: 0x8000
unique: 1788402, opcode: READ (15), nodeid: 16, insize: 80, pid: 0
read[2] 131072 bytes from 41615360 flags: 0x8000
   read[2] 131072 bytes from 41484288
   unique: 1788400, success, outsize: 131088
   read[2] 131072 bytes from 41615360
   unique: 1788402, success, outsize: 131088
unique: 1788404, opcode: READ (15), nodeid: 16, insize: 80, pid: 0
read[2] 131072 bytes from 41746432 flags: 0x8000
   read[2] 131072 bytes from 41746432
   unique: 1788404, success, outsize: 131088
unique: 1788408, opcode: READ (15), nodeid: 16, insize: 80, pid: 0
read[2] 131072 bytes from 42008576 flags: 0x8000
   read[2] 131072 bytes from 42008576
   unique: 1788408, success, outsize: 131088
unique: 1788406, opcode: READ (15), nodeid: 16, insize: 80, pid: 0
read[2] 131072 bytes from 41877504 flags: 0x8000
   read[2] 131072 bytes from 41877504
   unique: 1788406, success, outsize: 131088
unique: 1788410, opcode: READ (15), nodeid: 16, insize: 80, pid: 0
read[2] 131072 bytes from 42139648 flags: 0x8000
   read[2] 131072 bytes from 42139648
   unique: 1788410, success, outsize: 131088
unique: 1788412, opcode: READ (15), nodeid: 16, insize: 80, pid: 0
read[2] 131072 bytes from 42270720 flags: 0x8000
   read[2] 131072 bytes from 42270720
   unique: 1788412, success, outsize: 131088
unique: 1788416, opcode: READ (15), nodeid: 16, insize: 80, pid: 0
read[2] 131072 bytes from 42532864 flags: 0x8000
   read[2] 131072 bytes from 42532864
   unique: 1788416, success, outsize: 131088
unique: 1788414, opcode: READ (15), nodeid: 16, insize: 80, pid: 0
read[2] 131072 bytes from 42401792 flags: 0x8000
   read[2] 131072 bytes from 42401792
   unique: 1788414, success, outsize: 131088
unique: 1788418, opcode: READ (15), nodeid: 16, insize: 80, pid: 0
read[2] 131072 bytes from 42663936 flags: 0x8000
   read[2] 131072 bytes from 42663936
   unique: 1788418, success, outsize: 131088
unique: 1788420, opcode: READ (15), nodeid: 16, insize: 80, pid: 0
read[2] 131072 bytes from 42795008 flags: 0x8000
   read[2] 131072 bytes from 42795008
   unique: 1788420, success, outsize: 131088
unique: 1788422, opcode: READ (15), nodeid: 16, insize: 80, pid: 0
read[2] 131072 bytes from 42926080 flags: 0x8000
   read[2] 131072 bytes from 42926080
   unique: 1788422, success, outsize: 131088
unique: 1788424, opcode: READ (15), nodeid: 16, insize: 80, pid: 0
read[2] 131072 bytes from 43057152 flags: 0x8000
   read[2] 131072 bytes from 43057152
   unique: 1788424, success, outsize: 131088
unique: 1788426, opcode: READ (15), nodeid: 16, insize: 80, pid: 0
unique: 1788428, opcode: READ (15), nodeid: 16, insize: 80, pid: 0
read[2] 131072 bytes from 43319296 flags: 0x8000
   read[2] 131072 bytes from 43319296
   unique: 1788428, success, outsize: 131088
read[2] 131072 bytes from 43188224 flags: 0x8000
   read[2] 131072 bytes from 43188224
   unique: 1788426, success, outsize: 131088
unique: 1788430, opcode: READ (15), nodeid: 16, insize: 80, pid: 0
read[2] 131072 bytes from 43450368 flags: 0x8000
   read[2] 131072 bytes from 43450368
   unique: 1788430, success, outsize: 131088
unique: 1788432, opcode: READ (15), nodeid: 16, insize: 80, pid: 0
read[2] 131072 bytes from 43581440 flags: 0x8000
   read[2] 131072 bytes from 43581440
   unique: 1788432, success, outsize: 131088
unique: 1788434, opcode: FLUSH (25), nodeid: 16, insize: 64, pid: 0
unique: 1788435, opcode: INTERRUPT (36), nodeid: 0, insize: 48, pid: 0
INTERRUPT: 1788434
2024-06-03T15:39:37.133Z [INF]       curl.cpp:insertV4Headers(2892): computing signature [HEAD] [/hanger/] [] []
2024-06-03T15:39:37.133Z [INF]       curl_util.cpp:url_to_host(334): url is https://s3.amazonaws.com
2024-06-03T15:39:37.225Z [INF]       curl.cpp:RequestPerform(2572): HTTP response code 404 was returned, returning ENOENT
2024-06-03T15:39:37.273Z [INF]       curl.cpp:RequestPerform(2572): HTTP response code 404 was returned, returning ENOENT
2024-06-03T15:39:37.825Z [INF]       curl.cpp:RequestPerform(2572): HTTP response code 404 was returned, returning ENOENT
2024-06-03T15:39:37.905Z [INF]       curl.cpp:insertV4Headers(2892): computing signature [HEAD] [/hanger-h264/] [] []
2024-06-03T15:39:37.905Z [INF]       curl_util.cpp:url_to_host(334): url is https://s3.amazonaws.com
2024-06-03T15:39:38.005Z [INF]       curl.cpp:RequestPerform(2572): HTTP response code 404 was returned, returning ENOENT
2024-06-03T15:40:08.787Z [INF] s3fs.cpp:s3fs_flush(2984): [path=/abccde239/original.mp4][pseudo_fd=2]
flush[2]
   unique: 1788434, success, outsize: 16
unique: 1788436, opcode: RELEASE (18), nodeid: 16, insize: 64, pid: 0
release[2] flags: 0x8000
   unique: 1788436, success, outsize: 16
unique: 1788438, opcode: GETATTR (3), nodeid: 1, insize: 56, pid: 0
getattr /
   unique: 1788438, success, outsize: 120

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@Dmitry-Filippov-Rival