Ocis under load. 040 test

[ScharfViktor]

use fresh ocis master branch:

Steps:

• start ocis binary
• run k6 tests:
  • clone https://github.com/owncloud/cdperf.git
  • go to the cdperf dir
  • install k6 sudo apt-get install k6 or brew install k6
  • run sh script(in zip)TEST_SERVER_URL=your_ocis_url sh location/run_k6_test.sh run_k6_040_test.sh.zip

Result: ERR error while indexing a space is not in the error log.

## ocis_log

2024-01-11T10:01:43+01:00 ERR failed to stat the moved resource res={"status":{"code":6,"message":"stat: error: not found: 8b07a36d-5cee-4890-8255-fd1f9184fab5/perftestuser27-pqryhuxjyv-iteration-2","trace":"4d240f866167ec393edb8de1aeb12347"}} service=search
2024-01-11T10:01:43+01:00 ERR failed to stat the moved resource res={"status":{"code":6,"message":"stat: error: not found: 002f1103-033b-4294-998d-82315cd1bc55/perftestuser35-uwfyhvxlnb-iteration-2","trace":"7137339932dd2afd65f1368c839ae1ee"}} service=search
2024-01-11T10:01:43+01:00 ERR failed to stat the moved resource res={"status":{"code":6,"message":"stat: error: not found: perftestuser27-pqryhuxjyv-iteration-2","trace":"22caf542dfd7db6e6a0e5e2054613925"}} service=search
2024-01-11T10:01:44+01:00 ERR failed to stat the moved resource res={"status":{"code":6,"message":"stat: error: not found: perftestuser35-uwfyhvxlnb-iteration-2","trace":"edbf78c0a31c63587cb5ea92974aab16"}} service=search
2024-01-11T10:01:44+01:00 ERR error walking the tree error="error: not found: list container: error: not found: " service=search
2024-01-11T10:01:44+01:00 ERR error while indexing a space error="error: not found: list container: error: not found: " service=search spaceID={"opaque_id":"2c794f34-d2b6-4a44-9aaa-25dacc451519$f84e0ad7-97bf-4574-b55e-f561db3754d3"} userID={"idp":"https://localhost:9200","opaque_id":"f84e0ad7-97bf-4574-b55e-f561db3754d3","type":1}
2024-01-11T10:01:44+01:00 ERR failed to stat the moved resource res={"status":{"code":6,"message":"stat: error: not found: 3a3ba745-16ee-4696-9355-54c1caac88e5/perftestuser8-hsfyvyiezr-iteration-4","trace":"ed418408738b917ffc79860e834acaff"}} service=search
2024-01-11T10:01:44+01:00 ERR failed to stat the moved resource res={"status":{"code":6,"message":"stat: error: not found: c6965c33-3fb7-4b2e-868e-8b24651e1653/perftestuser33-wyvaccgmqn-iteration-2","trace":"9b04fef9a34223ab0b9fd420495fb147"}} service=search
2024-01-11T10:01:44+01:00 ERR failed to stat the moved resource res={"status":{"code":6,"message":"stat: error: not found: c8e5a84c-e5a0-47c3-8d8c-c48fe9a7144b/perftestuser4-mymwpcwaha-iteration-4","trace":"b85707d0d7938efb68ec1eea0ed02387"}} service=search
2024-01-11T10:01:44+01:00 ERR error walking the tree error="internal error: list container:readlink /Users/scharfviktor/.ocis/storage/users/spaces/f3/648c89-6b92-4dee-a3ea-e2daaa16bf64/nodes/f3/64/8c/89/-6b92-4dee-a3ea-e2daaa16bf64/perftestuser20-ynvarajcbo-iteration-3-renamed: no such file or directory" service=search
2024-01-11T10:01:44+01:00 ERR error while indexing a space error="internal error: list container:readlink /Users/scharfviktor/.ocis/storage/users/spaces/f3/648c89-6b92-4dee-a3ea-e2daaa16bf64/nodes/f3/64/8c/89/-6b92-4dee-a3ea-e2daaa16bf64/perftestuser20-ynvarajcbo-iteration-3-renamed: no such file or directory" service=search spaceID={"opaque_id":"2c794f34-d2b6-4a44-9aaa-25dacc451519$f3648c89-6b92-4dee-a3ea-e2daaa16bf64"} userID={"idp":"https://localhost:9200","opaque_id":"f3648c89-6b92-4dee-a3ea-e2daaa16bf64","type":1}
2024-01-11T10:01:44+01:00 ERR failed to stat the moved resource res={"status":{"code":6,"message":"stat: error: not found: perftestuser8-hsfyvyiezr-iteration-4","trace":"4bb663c458ab9dec488b36697dc8ea76"}} service=search
2024-01-11T10:01:44+01:00 ERR failed to stat the moved resource res={"status":{"code":6,"message":"stat: error: not found: perftestuser33-wyvaccgmqn-iteration-2","trace":"d138e5ab9d901445ac84570a1e5851c9"}} service=search
2024-01-11T10:01:44+01:00 ERR failed to stat the moved resource res={"status":{"code":6,"message":"stat: error: not found: perftestuser4-mymwpcwaha-iteration-4","trace":"aa76849b95e3ec9cac35a98f9f152743"}} service=search
2024-01-11T10:01:44+01:00 ERR failed to stat the moved resource res={"status":{"code":6,"message":"stat: error: not found: 288b3b78-1f11-4da6-be9a-cc9e74cde0b1/perftestuser43-snynqgjaxi-iteration-1","trace":"a38e989ad75aab72d36c4605f4c5d1a8"}} service=search
2024-01-11T10:01:44+01:00 ERR failed to stat the moved resource res={"status":{"code":6,"message":"stat: error: not found: a8dd9a28-d6d2-4f3d-9bd9-464474adf8d7/perftestuser19-aaqfpmppgq-iteration-3","trace":"e15aeb7629281f7152d4d0cea568110c"}} service=search
2024-01-11T10:01:44+01:00 ERR failed to stat the moved resource res={"status":{"code":6,"message":"stat: error: not found: perftestuser19-aaqfpmppgq-iteration-3","trace":"697009fb00483e36f1d6c8fc4b4aacf3"}} service=search
2024-01-11T10:01:44+01:00 ERR failed to stat the moved resource res={"status":{"code":6,"message":"stat: error: not found: 6df2ebd3-d30a-422e-9fff-eabf56e9d431/perftestuser22-oplmdidmbn-iteration-3","trace":"05f9d07fe834613ea585f238b8c92721"}} service=search
2024-01-11T10:01:44+01:00 ERR failed to stat the moved resource res={"status":{"code":6,"message":"stat: error: not found: perftestuser22-oplmdidmbn-iteration-3","trace":"453568e46bf6c514fcbef8dc58f4721d"}} service=search
2024-01-11T10:01:45+01:00 ERR error walking the tree error="error: not found: list container: error: not found: " service=search
2024-01-11T10:01:45+01:00 ERR error while indexing a space error="error: not found: list container: error: not found: " service=search spaceID={"opaque_id":"2c794f34-d2b6-4a44-9aaa-25dacc451519$002f1103-033b-4294-998d-82315cd1bc55"} userID={"idp":"https://localhost:9200","opaque_id":"002f1103-033b-4294-998d-82315cd1bc55","type":1}
2024-01-11T10:01:45+01:00 ERR failed to stat the moved resource res={"status":{"code":6,"message":"stat: error: not found: 8a486093-e694-42f1-a257-c5524a32eb81/perftestuser38-vnyqzsvafk-iteration-2","trace":"6414fc324e9d76ff682b5a54d23153ce"}} service=search
2024-01-11T10:01:45+01:00 ERR failed to stat the moved resource res={"status":{"code":6,"message":"stat: error: not found: perftestuser38-vnyqzsvafk-iteration-2","trace":"326ca36e57a67929c2987dc560eaebcf"}} service=search
2024-01-11T10:01:45+01:00 ERR error walking the tree error="error: not found: list container: error: not found: " service=search
2024-01-11T10:01:45+01:00 ERR error while indexing a space error="error: not found: list container: error: not found: " service=search spaceID={"opaque_id":"2c794f34-d2b6-4a44-9aaa-25dacc451519$c8e5a84c-e5a0-47c3-8d8c-c48fe9a7144b"} userID={"idp":"https://localhost:9200","opaque_id":"c8e5a84c-e5a0-47c3-8d8c-c48fe9a7144b","type":1}
2024-01-11T10:01:45+01:00 ERR error walking the tree error="error: not found: list container: error: not found: " service=search
2024-01-11T10:01:45+01:00 ERR error while indexing a space error="error: not found: list container: error: not found: " service=search spaceID={"opaque_id":"2c794f34-d2b6-4a44-9aaa-25dacc451519$c6965c33-3fb7-4b2e-868e-8b24651e1653"} userID={"idp":"https://localhost:9200","opaque_id":"c6965c33-3fb7-4b2e-868e-8b24651e1653","type":1}
2024-01-11T10:01:46+01:00 ERR error walking the tree error="error: not found: list container: error: not found: " service=search
2024-01-11T10:01:46+01:00 ERR error while indexing a space error="error: not found: list container: error: not found: " service=search spaceID={"opaque_id":"2c794f34-d2b6-4a44-9aaa-25dacc451519$a8dd9a28-d6d2-4f3d-9bd9-464474adf8d7"} userID={"idp":"https://localhost:9200","opaque_id":"a8dd9a28-d6d2-4f3d-9bd9-464474adf8d7","type":1}
2024-01-11T10:01:46+01:00 ERR error walking the tree error="error: not found: list container: error: not found: " service=search
2024-01-11T10:01:46+01:00 ERR error while indexing a space error="error: not found: list container: error: not found: " service=search spaceID={"opaque_id":"2c794f34-d2b6-4a44-9aaa-25dacc451519$6df2ebd3-d30a-422e-9fff-eabf56e9d431"} userID={"idp":"https://localhost:9200","opaque_id":"6df2ebd3-d30a-422e-9fff-eabf56e9d431","type":1}
2024-01-11T10:01:46+01:00 ERR failed to stat the moved resource res={"status":{"code":6,"message":"stat: error: not found: ccde2e81-dd5c-4d85-86bf-da2f586ab2d8/perftestuser46-xvxblqhvzh-iteration-1","trace":"4b3364f04c43ed9bb63cc5760d739a6b"}} service=search
2024-01-11T10:01:50+01:00 ERR error walking the tree error="error: not found: list container: error: not found: " service=search
2024-01-11T10:01:50+01:00 ERR error while indexing a space error="error: not found: list container: error: not found: " service=search spaceID={"opaque_id":"2c794f34-d2b6-4a44-9aaa-25dacc451519$22929c12-7e13-4f74-851a-d27f3b3a3090"} userID={"idp":"https://localhost:9200","opaque_id":"22929c12-7e13-4f74-851a-d27f3b3a3090","type":1}
2024-01-11T10:01:50+01:00 ERR error walking the tree error="error: not found: list container: error: not found: " service=search
2024-01-11T10:01:50+01:00 ERR error while indexing a space error="error: not found: list container: error: not found: " service=search spaceID={"opaque_id":"2c794f34-d2b6-4a44-9aaa-25dacc451519$2c7ebed8-3550-411f-a5a3-8c02a7ee690a"} userID={"idp":"https://localhost:9200","opaque_id":"2c7ebed8-3550-411f-a5a3-8c02a7ee690a","type":1}
2024-01-11T10:01:50+01:00 ERR failed to move the changed resource in the index error="entity not found" service=search
2024-01-11T10:01:51+01:00 ERR error walking the tree error="error: not found: list container: error: not found: " service=search
2024-01-11T10:01:51+01:00 ERR error while indexing a space error="error: not found: list container: error: not found: " service=search spaceID={"opaque_id":"2c794f34-d2b6-4a44-9aaa-25dacc451519$e95fe638-d9a6-4396-9ee5-59602acac7b3"} userID={"idp":"https://localhost:9200","opaque_id":"e95fe638-d9a6-4396-9ee5-59602acac7b3","type":1}
2024-01-11T10:01:51+01:00 ERR error walking the tree error="error: not found: list container: error: not found: " service=search
2024-01-11T10:01:51+01:00 ERR error while indexing a space error="error: not found: list container: error: not found: " service=search spaceID={"opaque_id":"2c794f34-d2b6-4a44-9aaa-25dacc451519$5e11e807-0d96-409b-8fa2-5309a2ef23cd"} userID={"idp":"https://localhost:9200","opaque_id":"5e11e807-0d96-409b-8fa2-5309a2ef23cd","type":1}
2024-01-11T10:01:52+01:00 ERR error walking the tree error="error: not found: list container: error: not found: " service=search
2024-01-11T10:01:52+01:00 ERR error while indexing a space error="error: not found: list container: error: not found: " service=search spaceID={"opaque_id":"2c794f34-d2b6-4a44-9aaa-25dacc451519$aebfd897-48ad-4345-91f7-9beaf1e33f8b"} userID={"idp":"https://localhost:9200","opaque_id":"aebfd897-48ad-4345-91f7-9beaf1e33f8b","type":1}
2024-01-11T10:01:52+01:00 ERR failed to remove item from index error="entity not found" Id={"opaque_id":"7327f2de-38f1-45fb-b3ea-0755d53150a2","space_id":"aebfd897-48ad-4345-91f7-9beaf1e33f8b","storage_id":"2c794f34-d2b6-4a44-9aaa-25dacc451519"} service=search
2024-01-11T10:01:52+01:00 ERR error walking the tree error="error: not found: list container: error: not found: " service=search
2024-01-11T10:01:52+01:00 ERR error while indexing a space error="error: not found: list container: error: not found: " service=search spaceID={"opaque_id":"2c794f34-d2b6-4a44-9aaa-25dacc451519$9f450234-c7ba-4091-8d3c-560c4eb1e84b"} userID={"idp":"https://localhost:9200","opaque_id":"9f450234-c7ba-4091-8d3c-560c4eb1e84b","type":1}
2024-01-11T10:01:52+01:00 ERR error getting resource error="unexpected status code while getting resource: CODE_NOT_FOUND" event={"Event":{"ExecutingUser":{"id":{"idp":"https://localhost:9200","opaque_id":"f3648c89-6b92-4dee-a3ea-e2daaa16bf64"},"username":"perf-test-user-20"},"Failed":false,"FileRef":{"path":"./perftestuser20-uwvzfuidcp-iteration-4/small.zip","resource_id":{"opaque_id":"f3648c89-6b92-4dee-a3ea-e2daaa16bf64","space_id":"f3648c89-6b92-4dee-a3ea-e2daaa16bf64","storage_id":"2c794f34-d2b6-4a44-9aaa-25dacc451519"}},"Filename":"small.zip","SpaceOwner":{"idp":"https://localhost:9200","opaque_id":"f3648c89-6b92-4dee-a3ea-e2daaa16bf64","type":1},"Timestamp":{"nanos":148608000,"seconds":1704963712},"UploadID":"256e0ad0-9bbf-4210-ab22-5791696758d2"},"ID":"5b77b779-28ed-4383-b4a2-4d14fa48563d","TraceParent":"00-213c6ecb6f3601f46ea90bd100fdaa6e-55abd3628ee865b6-00","Type":"events.UploadReady"} service=clientlog
2024-01-11T10:01:52+01:00 ERR error getting resource error="unexpected status code while getting resource: CODE_NOT_FOUND" event={"Event":{"ExecutingUser":{"id":{"idp":"https://localhost:9200","opaque_id":"36dd8ce7-0d5e-4ee4-90cd-002e542cad02"},"username":"perf-test-user-7"},"Failed":false,"FileRef":{"path":"./perftestuser7-jooxotwuxw-iteration-5/small.zip","resource_id":{"opaque_id":"36dd8ce7-0d5e-4ee4-90cd-002e542cad02","space_id":"36dd8ce7-0d5e-4ee4-90cd-002e542cad02","storage_id":"2c794f34-d2b6-4a44-9aaa-25dacc451519"}},"Filename":"small.zip","SpaceOwner":{"idp":"https://localhost:9200","opaque_id":"36dd8ce7-0d5e-4ee4-90cd-002e542cad02","type":1},"Timestamp":{"nanos":317378000,"seconds":1704963712},"UploadID":"f25cf6f2-ba37-4e9c-868f-1b8ba2ddc8ac"},"ID":"558962a5-9aa2-4f0d-8d15-5d2af92abafb","TraceParent":"00-213c6ecb6f3601f46ea90bd100fdaa6e-55abd3628ee865b6-00","Type":"events.UploadReady"} service=clientlog
2024-01-11T10:01:52+01:00 ERR error walking the tree error="error: not found: list container: error: not found: " service=search
2024-01-11T10:01:52+01:00 ERR error while indexing a space error="error: not found: list container: error: not found: " service=search spaceID={"opaque_id":"2c794f34-d2b6-4a44-9aaa-25dacc451519$a6ed876c-bf2c-4131-a4b2-83f83346d793"} userID={"idp":"https://localhost:9200","opaque_id":"a6ed876c-bf2c-4131-a4b2-83f83346d793","type":1}
2024-01-11T10:01:52+01:00 ERR error getting resource error="unexpected status code while getting resource: CODE_NOT_FOUND" event={"Event":{"ExecutingUser":{"id":{"idp":"https://localhost:9200","opaque_id":"3c1eadb5-ff8d-4892-bd41-32cd3fdb18b7"},"username":"perf-test-user-45"},"Failed":false,"FileRef":{"path":"./perftestuser45-wqvoqzoupg-iteration-2/small.zip","resource_id":{"opaque_id":"3c1eadb5-ff8d-4892-bd41-32cd3fdb18b7","space_id":"3c1eadb5-ff8d-4892-bd41-32cd3fdb18b7","storage_id":"2c794f34-d2b6-4a44-9aaa-25dacc451519"}},"Filename":"small.zip","SpaceOwner":{"idp":"https://localhost:9200","opaque_id":"3c1eadb5-ff8d-4892-bd41-32cd3fdb18b7","type":1},"Timestamp":{"nanos":423706000,"seconds":1704963712},"UploadID":"a51db443-70d4-4e38-a074-6b6775bf22e3"},"ID":"463ef9ee-9aab-43f2-9fc3-3a98fc44b8d5","TraceParent":"00-213c6ecb6f3601f46ea90bd100fdaa6e-55abd3628ee865b6-00","Type":"events.UploadReady"} service=clientlog
2024-01-11T10:01:52+01:00 ERR error getting resource error="unexpected status code while getting resource: CODE_NOT_FOUND" event={"Event":{"ExecutingUser":{"id":{"idp":"https://localhost:9200","opaque_id":"d2dc1a75-78b2-4909-ad53-9577b4bd84bd"},"username":"perf-test-user-49"},"Failed":false,"FileRef":{"path":"./perftestuser49-ixncfuyffu-iteration-1/medium.zip","resource_id":{"opaque_id":"d2dc1a75-78b2-4909-ad53-9577b4bd84bd","space_id":"d2dc1a75-78b2-4909-ad53-9577b4bd84bd","storage_id":"2c794f34-d2b6-4a44-9aaa-25dacc451519"}},"Filename":"medium.zip","SpaceOwner":{"idp":"https://localhost:9200","opaque_id":"d2dc1a75-78b2-4909-ad53-9577b4bd84bd","type":1},"Timestamp":{"nanos":674533000,"seconds":1704963712},"UploadID":"4133c23a-839c-4577-b127-303152dd99da"},"ID":"8459619d-5538-43dc-9e26-814548f88859","TraceParent":"00-213c6ecb6f3601f46ea90bd100fdaa6e-55abd3628ee865b6-00","Type":"events.UploadReady"} service=clientlog
2024-01-11T10:01:52+01:00 ERR missing parent id attrs={} messagepack= nodeid=aef87197-af97-4bba-afb3-a00539ee4909 path=/Users/scharfviktor/.ocis/storage/users/spaces/96/3adb7d-a5d0-4441-ad4c-f919b28afa2c/nodes/ae/f8/71/97/-af97-4bba-afb3-a00539ee4909 pkg=rgrpc service=storage-users traceid=ca75df6176c6d06b75eece4e7231c985
2024-01-11T10:01:52+01:00 ERR error walking the tree error="internal error: list container:internal error: Missing parent ID on node" service=search
2024-01-11T10:01:52+01:00 ERR error while indexing a space error="internal error: list container:internal error: Missing parent ID on node" service=search spaceID={"opaque_id":"2c794f34-d2b6-4a44-9aaa-25dacc451519$963adb7d-a5d0-4441-ad4c-f919b28afa2c"} userID={"idp":"https://localhost:9200","opaque_id":"963adb7d-a5d0-4441-ad4c-f919b28afa2c","type":1}
2024-01-11T10:01:53+01:00 ERR error getting resource error="unexpected status code while getting resource: CODE_NOT_FOUND" event={"Event":{"ExecutingUser":{"id":{"idp":"https://localhost:9200","opaque_id":"56060c6a-b4cb-43ba-9de2-26b61a00f3f1"},"username":"perf-test-user-11"},"Failed":false,"FileRef":{"path":"./perftestuser11-bfpuizbuwb-iteration-4/medium.zip","resource_id":{"opaque_id":"56060c6a-b4cb-43ba-9de2-26b61a00f3f1","space_id":"56060c6a-b4cb-43ba-9de2-26b61a00f3f1","storage_id":"2c794f34-d2b6-4a44-9aaa-25dacc451519"}},"Filename":"medium.zip","SpaceOwner":{"idp":"https://localhost:9200","opaque_id":"56060c6a-b4cb-43ba-9de2-26b61a00f3f1","type":1},"Timestamp":{"nanos":814881000,"seconds":1704963712},"UploadID":"9e983b88-416b-4aa3-a56f-88114497e437"},"ID":"5920e272-51a5-44e4-9b48-da4ef020c270","TraceParent":"00-213c6ecb6f3601f46ea90bd100fdaa6e-55abd3628ee865b6-00","Type":"events.UploadReady"} service=clientlog
2024-01-11T10:01:53+01:00 ERR error getting resource error="unexpected status code while getting resource: CODE_NOT_FOUND" event={"Event":{"ExecutingUser":{"id":{"idp":"https://localhost:9200","opaque_id":"f5910fb9-6d6b-4d85-a999-e68d0d144a2d"},"username":"perf-test-user-2"},"Failed":false,"FileRef":{"path":"./perftestuser2-udrtyatjqu-iteration-5/medium.zip","resource_id":{"opaque_id":"f5910fb9-6d6b-4d85-a999-e68d0d144a2d","space_id":"f5910fb9-6d6b-4d85-a999-e68d0d144a2d","storage_id":"2c794f34-d2b6-4a44-9aaa-25dacc451519"}},"Filename":"medium.zip","SpaceOwner":{"idp":"https://localhost:9200","opaque_id":"f5910fb9-6d6b-4d85-a999-e68d0d144a2d","type":1},"Timestamp":{"nanos":980021000,"seconds":1704963712},"UploadID":"7106b997-1f24-4713-b08d-9ea3aaa8a1be"},"ID":"f6ddd2fe-9a13-419f-977e-de10c63a8cff","TraceParent":"00-213c6ecb6f3601f46ea90bd100fdaa6e-55abd3628ee865b6-00","Type":"events.UploadReady"} service=clientlog
2024-01-11T10:01:53+01:00 ERR error getting resource error="unexpected status code while getting resource: CODE_NOT_FOUND" event={"Event":{"ExecutingUser":{"id":{"idp":"https://localhost:9200","opaque_id":"56173a57-0ee9-4be9-b963-4a9a85ea3001"},"username":"perf-test-user-6"},"Failed":false,"FileRef":{"path":"./perftestuser6-rxtkdfxfrg-iteration-5/medium.zip","resource_id":{"opaque_id":"56173a57-0ee9-4be9-b963-4a9a85ea3001","space_id":"56173a57-0ee9-4be9-b963-4a9a85ea3001","storage_id":"2c794f34-d2b6-4a44-9aaa-25dacc451519"}},"Filename":"medium.zip","SpaceOwner":{"idp":"https://localhost:9200","opaque_id":"56173a57-0ee9-4be9-b963-4a9a85ea3001","type":1},"Timestamp":{"nanos":74574000,"seconds":1704963713},"UploadID":"e783dfb5-c9e1-407d-8d87-72c369e71b5f"},"ID":"2e2349bc-23a1-4dc4-89d7-2777419f2f72","TraceParent":"00-213c6ecb6f3601f46ea90bd100fdaa6e-55abd3628ee865b6-00","Type":"events.UploadReady"} service=clientlog
2024-01-11T10:01:53+01:00 ERR error walking the tree error="error: not found: list container: error: not found: " service=search
2024-01-11T10:01:53+01:00 ERR error while indexing a space error="error: not found: list container: error: not found: " service=search spaceID={"opaque_id":"2c794f34-d2b6-4a44-9aaa-25dacc451519$82d9f959-4e65-42d3-8f10-5c1d06db5bc6"} userID={"idp":"https://localhost:9200","opaque_id":"82d9f959-4e65-42d3-8f10-5c1d06db5bc6","type":1}
2024-01-11T10:01:53+01:00 ERR error getting resource error="unexpected status code while getting resource: CODE_NOT_FOUND" event={"Event":{"ExecutingUser":{"id":{"idp":"https://localhost:9200","opaque_id":"dca1cb4c-bbb6-421e-be3c-d1b1ed246283"},"username":"perf-test-user-29"},"Failed":false,"FileRef":{"path":"./perftestuser29-oabclpklnh-iteration-3/medium.zip","resource_id":{"opaque_id":"dca1cb4c-bbb6-421e-be3c-d1b1ed246283","space_id":"dca1cb4c-bbb6-421e-be3c-d1b1ed246283","storage_id":"2c794f34-d2b6-4a44-9aaa-25dacc451519"}},"Filename":"medium.zip","SpaceOwner":{"idp":"https://localhost:9200","opaque_id":"dca1cb4c-bbb6-421e-be3c-d1b1ed246283","type":1},"Timestamp":{"nanos":171678000,"seconds":1704963713},"UploadID":"1f628255-fd89-445c-ac39-5e113a32031f"},"ID":"5220b5c8-1188-4c9c-ac21-72a2140cbff9","TraceParent":"00-213c6ecb6f3601f46ea90bd100fdaa6e-55abd3628ee865b6-00","Type":"events.UploadReady"} service=clientlog
2024-01-11T10:01:53+01:00 ERR failed to stat the moved resource res={"status":{"code":6,"message":"stat: error: not found: perftestuser44-qwakchltxl-iteration-2-renamed","trace":"d700602e448de92d9fe1bc5fd387bcfe"}} service=search
2024-01-11T10:01:53+01:00 ERR error getting resource error="unexpected status code while getting resource: CODE_NOT_FOUND" event={"Event":{"ExecutingUser":{"id":{"idp":"https://localhost:9200","opaque_id":"28d71e5c-bfce-4317-9cd1-f8a01dc51d06"},"username":"perf-test-user-63"},"Failed":false,"FileRef":{"path":"./perftestuser63-uomsxlsvie-iteration-1/small.zip","resource_id":{"opaque_id":"28d71e5c-bfce-4317-9cd1-f8a01dc51d06","space_id":"28d71e5c-bfce-4317-9cd1-f8a01dc51d06","storage_id":"2c794f34-d2b6-4a44-9aaa-25dacc451519"}},"Filename":"small.zip","SpaceOwner":{"idp":"https://localhost:9200","opaque_id":"28d71e5c-bfce-4317-9cd1-f8a01dc51d06","type":1},"Timestamp":{"nanos":376730000,"seconds":1704963713},"UploadID":"3d02e342-bc6b-4c3f-b546-53c1b65f28ae"},"ID":"1524f728-9277-4895-a8e1-b5843a3cd97f","TraceParent":"00-213c6ecb6f3601f46ea90bd100fdaa6e-55abd3628ee865b6-00","Type":"events.UploadReady"} service=clientlog
2024-01-11T10:01:53+01:00 ERR error getting resource error="unexpected status code while getting resource: CODE_NOT_FOUND" event={"Event":{"ExecutingUser":{"id":{"idp":"https://localhost:9200","opaque_id":"3a3ba745-16ee-4696-9355-54c1caac88e5"},"username":"perf-test-user-8"},"Failed":false,"FileRef":{"path":"./perftestuser8-pthidnfzkn-iteration-5/small.zip","resource_id":{"opaque_id":"3a3ba745-16ee-4696-9355-54c1caac88e5","space_id":"3a3ba745-16ee-4696-9355-54c1caac88e5","storage_id":"2c794f34-d2b6-4a44-9aaa-25dacc451519"}},"Filename":"small.zip","SpaceOwner":{"idp":"https://localhost:9200","opaque_id":"3a3ba745-16ee-4696-9355-54c1caac88e5","type":1},"Timestamp":{"nanos":619962000,"seconds":1704963713},"UploadID":"5cda924d-1d03-459c-b4f4-6f6ce83c35c5"},"ID":"f6b015f6-fe94-467e-8766-d5fb0df9b02a","TraceParent":"00-213c6ecb6f3601f46ea90bd100fdaa6e-55abd3628ee865b6-00","Type":"events.UploadReady"} service=clientlog
2024-01-11T10:01:53+01:00 ERR error getting resource error="unexpected status code while getting resource: CODE_NOT_FOUND" event={"Event":{"ExecutingUser":{"id":{"idp":"https://localhost:9200","opaque_id":"7dfc3267-7896-46dc-a553-4a83a4d80c08"},"username":"perf-test-user-13"},"Failed":false,"FileRef":{"path":"./perftestuser13-ggzpquyihd-iteration-4/small.zip","resource_id":{"opaque_id":"7dfc3267-7896-46dc-a553-4a83a4d80c08","space_id":"7dfc3267-7896-46dc-a553-4a83a4d80c08","storage_id":"2c794f34-d2b6-4a44-9aaa-25dacc451519"}},"Filename":"small.zip","SpaceOwner":{"idp":"https://localhost:9200","opaque_id":"7dfc3267-7896-46dc-a553-4a83a4d80c08","type":1},"Timestamp":{"nanos":742281000,"seconds":1704963713},"UploadID":"c59d1727-7b4c-4d76-9436-8a37957dbe75"},"ID":"854b9788-e0d0-4e74-b8fd-65e7bbeb3aab","TraceParent":"00-213c6ecb6f3601f46ea90bd100fdaa6e-55abd3628ee865b6-00","Type":"events.UploadReady"} service=clientlog
2024-01-11T10:01:53+01:00 ERR error walking the tree error="error: not found: list container: error: not found: " service=search
2024-01-11T10:01:53+01:00 ERR error while indexing a space error="error: not found: list container: error: not found: " service=search spaceID={"opaque_id":"2c794f34-d2b6-4a44-9aaa-25dacc451519$3c1eadb5-ff8d-4892-bd41-32cd3fdb18b7"} userID={"idp":"https://localhost:9200","opaque_id":"3c1eadb5-ff8d-4892-bd41-32cd3fdb18b7","type":1}
2024-01-11T10:01:54+01:00 ERR error getting resource error="unexpected status code while getting resource: CODE_NOT_FOUND" event={"Event":{"ExecutingUser":{"id":{"idp":"https://localhost:9200","opaque_id":"c6965c33-3fb7-4b2e-868e-8b24651e1653"},"username":"perf-test-user-33"},"Failed":false,"FileRef":{"path":"./perftestuser33-upndqhgdcz-iteration-3/small.zip","resource_id":{"opaque_id":"c6965c33-3fb7-4b2e-868e-8b24651e1653","space_id":"c6965c33-3fb7-4b2e-868e-8b24651e1653","storage_id":"2c794f34-d2b6-4a44-9aaa-25dacc451519"}},"Filename":"small.zip","SpaceOwner":{"idp":"https://localhost:9200","opaque_id":"c6965c33-3fb7-4b2e-868e-8b24651e1653","type":1},"Timestamp":{"nanos":889237000,"seconds":1704963713},"UploadID":"8f3bbf00-5f98-431f-80f7-66edde6f6a73"},"ID":"9e29fe51-cae3-4704-86ab-ab79337438f4","TraceParent":"00-213c6ecb6f3601f46ea90bd100fdaa6e-55abd3628ee865b6-00","Type":"events.UploadReady"} service=clientlog
2024-01-11T10:01:54+01:00 ERR error getting resource error="unexpected status code while getting resource: CODE_NOT_FOUND" event={"Event":{"ExecutingUser":{"id":{"idp":"https://localhost:9200","opaque_id":"c8e5a84c-e5a0-47c3-8d8c-c48fe9a7144b"},"username":"perf-test-user-4"},"Failed":false,"FileRef":{"path":"./perftestuser4-rgnsxihjqk-iteration-5/small.zip","resource_id":{"opaque_id":"c8e5a84c-e5a0-47c3-8d8c-c48fe9a7144b","space_id":"c8e5a84c-e5a0-47c3-8d8c-c48fe9a7144b","storage_id":"2c794f34-d2b6-4a44-9aaa-25dacc451519"}},"Filename":"small.zip","SpaceOwner":{"idp":"https://localhost:9200","opaque_id":"c8e5a84c-e5a0-47c3-8d8c-c48fe9a7144b","type":1},"Timestamp":{"nanos":50112000,"seconds":1704963714},"UploadID":"9cad093e-daf6-4f90-8f20-9830fe26652e"},"ID":"f9506596-3975-42c1-9053-82755a0b25de","TraceParent":"00-213c6ecb6f3601f46ea90bd100fdaa6e-55abd3628ee865b6-00","Type":"events.UploadReady"} service=clientlog
2024-01-11T10:01:54+01:00 ERR error getting resource error="unexpected status code while getting resource: CODE_NOT_FOUND" event={"Event":{"ExecutingUser":{"id":{"idp":"https://localhost:9200","opaque_id":"288b3b78-1f11-4da6-be9a-cc9e74cde0b1"},"username":"perf-test-user-43"},"Failed":false,"FileRef":{"path":"./perftestuser43-xlxzyahlkn-iteration-2/small.zip","resource_id":{"opaque_id":"288b3b78-1f11-4da6-be9a-cc9e74cde0b1","space_id":"288b3b78-1f11-4da6-be9a-cc9e74cde0b1","storage_id":"2c794f34-d2b6-4a44-9aaa-25dacc451519"}},"Filename":"small.zip","SpaceOwner":{"idp":"https://localhost:9200","opaque_id":"288b3b78-1f11-4da6-be9a-cc9e74cde0b1","type":1},"Timestamp":{"nanos":196855000,"seconds":1704963714},"UploadID":"d2b91318-4bc3-4c98-9d8b-dd8fd4fee08c"},"ID":"c2b4d9e6-8ae6-4d30-adcd-bbf11582c710","TraceParent":"00-213c6ecb6f3601f46ea90bd100fdaa6e-55abd3628ee865b6-00","Type":"events.UploadReady"} service=clientlog
2024-01-11T10:01:54+01:00 ERR failed to move the changed resource in the index error="entity not found" service=search
2024-01-11T10:01:54+01:00 ERR error getting resource error="unexpected status code while getting resource: CODE_NOT_FOUND" event={"Event":{"ExecutingUser":{"id":{"idp":"https://localhost:9200","opaque_id":"5d6d9568-c891-4848-8791-3c74db5769b5"},"username":"perf-test-user-65"},"Failed":false,"FileRef":{"path":"./perftestuser65-nyptpvtqbq-iteration-1/small.zip","resource_id":{"opaque_id":"5d6d9568-c891-4848-8791-3c74db5769b5","space_id":"5d6d9568-c891-4848-8791-3c74db5769b5","storage_id":"2c794f34-d2b6-4a44-9aaa-25dacc451519"}},"Filename":"small.zip","SpaceOwner":{"idp":"https://localhost:9200","opaque_id":"5d6d9568-c891-4848-8791-3c74db5769b5","type":1},"Timestamp":{"nanos":320424000,"seconds":1704963714},"UploadID":"0405a242-2b5b-4bad-a738-26b3b5ce7af3"},"ID":"10134c83-f8fd-490f-984b-7f644761b975","TraceParent":"00-213c6ecb6f3601f46ea90bd100fdaa6e-55abd3628ee865b6-00","Type":"events.UploadReady"} service=clientlog
2024-01-11T10:01:54+01:00 ERR error getting resource error="unexpected status code while getting resource: CODE_NOT_FOUND" event={"Event":{"ExecutingUser":{"id":{"idp":"https://localhost:9200","opaque_id":"ed44383d-ee3b-4ace-bdf9-078d59e86cd3"},"username":"perf-test-user-55"},"Failed":false,"FileRef":{"path":"./perftestuser55-ydnheghmxq-iteration-1/small.zip","resource_id":{"opaque_id":"ed44383d-ee3b-4ace-bdf9-078d59e86cd3","space_id":"ed44383d-ee3b-4ace-bdf9-078d59e86cd3","storage_id":"2c794f34-d2b6-4a44-9aaa-25dacc451519"}},"Filename":"small.zip","SpaceOwner":{"idp":"https://localhost:9200","opaque_id":"ed44383d-ee3b-4ace-bdf9-078d59e86cd3","type":1},"Timestamp":{"nanos":411567000,"seconds":1704963714},"UploadID":"ce8c8b8a-6cf6-434a-a018-baa60430920f"},"ID":"33a01023-a371-4c3a-8fcb-7407eb4571de","TraceParent":"00-213c6ecb6f3601f46ea90bd100fdaa6e-55abd3628ee865b6-00","Type":"events.UploadReady"} service=clientlog
2024-01-11T10:01:54+01:00 ERR error getting resource error="unexpected status code while getting resource: CODE_NOT_FOUND" event={"Event":{"ExecutingUser":{"id":{"idp":"https://localhost:9200","opaque_id":"8b07a36d-5cee-4890-8255-fd1f9184fab5"},"username":"perf-test-user-27"},"Failed":false,"FileRef":{"path":"./perftestuser27-ynyxndadeh-iteration-3/medium.zip","resource_id":{"opaque_id":"8b07a36d-5cee-4890-8255-fd1f9184fab5","space_id":"8b07a36d-5cee-4890-8255-fd1f9184fab5","storage_id":"2c794f34-d2b6-4a44-9aaa-25dacc451519"}},"Filename":"medium.zip","SpaceOwner":{"idp":"https://localhost:9200","opaque_id":"8b07a36d-5cee-4890-8255-fd1f9184fab5","type":1},"Timestamp":{"nanos":498057000,"seconds":1704963714},"UploadID":"38afbe67-0cf4-4b3f-879e-a9b305a19a19"},"ID":"de982860-4d4d-4366-b503-1bf7c374089c","TraceParent":"00-213c6ecb6f3601f46ea90bd100fdaa6e-55abd3628ee865b6-00","Type":"events.UploadReady"} service=clientlog
2024-01-11T10:01:54+01:00 ERR error getting resource error="unexpected status code while getting resource: CODE_NOT_FOUND" event={"Event":{"ExecutingUser":{"id":{"idp":"https://localhost:9200","opaque_id":"06750f65-12bb-4074-89f6-dc6fb1cac24e"},"username":"perf-test-user-24"},"Failed":false,"FileRef":{"path":"./perftestuser24-elqsbdkdau-iteration-4/small.zip","resource_id":{"opaque_id":"06750f65-12bb-4074-89f6-dc6fb1cac24e","space_id":"06750f65-12bb-4074-89f6-dc6fb1cac24e","storage_id":"2c794f34-d2b6-4a44-9aaa-25dacc451519"}},"Filename":"small.zip","SpaceOwner":{"idp":"https://localhost:9200","opaque_id":"06750f65-12bb-4074-89f6-dc6fb1cac24e","type":1},"Timestamp":{"nanos":644596000,"seconds":1704963714},"UploadID":"479cdf32-5ff6-436c-b361-affda9f932b9"},"ID":"680c3a93-5d6f-4f9d-98cb-77bdfaec5e5e","TraceParent":"00-213c6ecb6f3601f46ea90bd100fdaa6e-55abd3628ee865b6-00","Type":"events.UploadReady"} service=clientlog
2024-01-11T10:01:54+01:00 ERR error getting resource error="unexpected status code while getting resource: CODE_NOT_FOUND" event={"Event":{"ExecutingUser":{"id":{"idp":"https://localhost:9200","opaque_id":"8a486093-e694-42f1-a257-c5524a32eb81"},"username":"perf-test-user-38"},"Failed":false,"FileRef":{"path":"./perftestuser38-jfhhcxnclp-iteration-3/small.zip","resource_id":{"opaque_id":"8a486093-e694-42f1-a257-c5524a32eb81","space_id":"8a486093-e694-42f1-a257-c5524a32eb81","storage_id":"2c794f34-d2b6-4a44-9aaa-25dacc451519"}},"Filename":"small.zip","SpaceOwner":{"idp":"https://localhost:9200","opaque_id":"8a486093-e694-42f1-a257-c5524a32eb81","type":1},"Timestamp":{"nanos":762928000,"seconds":1704963714},"UploadID":"bc4655ed-7351-4aef-ae70-1bcf9d2157c8"},"ID":"a56ea28d-259d-465f-9b36-25e97d084d16","TraceParent":"00-213c6ecb6f3601f46ea90bd100fdaa6e-55abd3628ee865b6-00","Type":"events.UploadReady"} service=clientlog
2024-01-11T10:01:54+01:00 ERR error getting resource error="unexpected status code while getting resource: CODE_NOT_FOUND" event={"Event":{"ExecutingUser":{"id":{"idp":"https://localhost:9200","opaque_id":"963adb7d-a5d0-4441-ad4c-f919b28afa2c"},"username":"perf-test-user-51"},"Failed":false,"FileRef":{"path":"./perftestuser51-bhsoznhoau-iteration-2/small.zip","resource_id":{"opaque_id":"963adb7d-a5d0-4441-ad4c-f919b28afa2c","space_id":"963adb7d-a5d0-4441-ad4c-f919b28afa2c","storage_id":"2c794f34-d2b6-4a44-9aaa-25dacc451519"}},"Filename":"small.zip","SpaceOwner":{"idp":"https://localhost:9200","opaque_id":"963adb7d-a5d0-4441-ad4c-f919b28afa2c","type":1},"Timestamp":{"nanos":889996000,"seconds":1704963714},"UploadID":"7eb6795a-ffe7-467b-9cfc-fcf4f9f32345"},"ID":"d9515a2c-42ee-4abf-bba4-66d7b5975299","TraceParent":"00-213c6ecb6f3601f46ea90bd100fdaa6e-55abd3628ee865b6-00","Type":"events.UploadReady"} service=clientlog
2024-01-11T10:01:55+01:00 ERR error getting resource error="unexpected status code while getting resource: CODE_NOT_FOUND" event={"Event":{"ExecutingUser":{"id":{"idp":"https://localhost:9200","opaque_id":"095796e3-5e42-4454-b9d7-27cf93736c1f"},"username":"perf-test-user-5"},"Failed":false,"FileRef":{"path":"./perftestuser5-fbsensirea-iteration-5/medium.zip","resource_id":{"opaque_id":"095796e3-5e42-4454-b9d7-27cf93736c1f","space_id":"095796e3-5e42-4454-b9d7-27cf93736c1f","storage_id":"2c794f34-d2b6-4a44-9aaa-25dacc451519"}},"Filename":"medium.zip","SpaceOwner":{"idp":"https://localhost:9200","opaque_id":"095796e3-5e42-4454-b9d7-27cf93736c1f","type":1},"Timestamp":{"nanos":6906000,"seconds":1704963715},"UploadID":"c4d68181-c201-45e8-94b4-4cc5ef6c84d8"},"ID":"253d9c54-691b-4aae-99b2-51d577aef645","TraceParent":"00-213c6ecb6f3601f46ea90bd100fdaa6e-55abd3628ee865b6-00","Type":"events.UploadReady"} service=clientlog
2024-01-11T10:01:55+01:00 ERR error getting resource error="unexpected status code while getting resource: CODE_NOT_FOUND" event={"Event":{"ExecutingUser":{"id":{"idp":"https://localhost:9200","opaque_id":"002f1103-033b-4294-998d-82315cd1bc55"},"username":"perf-test-user-35"},"Failed":false,"FileRef":{"path":"./perftestuser35-uapsnqfphv-iteration-3/medium.zip","resource_id":{"opaque_id":"002f1103-033b-4294-998d-82315cd1bc55","space_id":"002f1103-033b-4294-998d-82315cd1bc55","storage_id":"2c794f34-d2b6-4a44-9aaa-25dacc451519"}},"Filename":"medium.zip","SpaceOwner":{"idp":"https://localhost:9200","opaque_id":"002f1103-033b-4294-998d-82315cd1bc55","type":1},"Timestamp":{"nanos":146585000,"seconds":1704963715},"UploadID":"f9c4b91b-3d23-4368-a5bf-8567dece457a"},"ID":"497ed9fa-0ce7-467f-b28c-b6d90e0daa10","TraceParent":"00-213c6ecb6f3601f46ea90bd100fdaa6e-55abd3628ee865b6-00","Type":"events.UploadReady"} service=clientlog
2024-01-11T10:01:55+01:00 ERR failed to remove item from index error="entity not found" Id={"opaque_id":"29cfbf28-a753-4aa9-a3c1-18b1199a1991","space_id":"963adb7d-a5d0-4441-ad4c-f919b28afa2c","storage_id":"2c794f34-d2b6-4a44-9aaa-25dacc451519"} service=search
2024-01-11T10:01:55+01:00 ERR error getting resource error="unexpected status code while getting resource: CODE_NOT_FOUND" event={"Event":{"ExecutingUser":{"id":{"idp":"https://localhost:9200","opaque_id":"035b244a-5832-453c-8bbe-0ac6c21fb356"},"username":"perf-test-user-3"},"Failed":false,"FileRef":{"path":"./perftestuser3-olwpenglra-iteration-5/small.zip","resource_id":{"opaque_id":"035b244a-5832-453c-8bbe-0ac6c21fb356","space_id":"035b244a-5832-453c-8bbe-0ac6c21fb356","storage_id":"2c794f34-d2b6-4a44-9aaa-25dacc451519"}},"Filename":"small.zip","SpaceOwner":{"idp":"https://localhost:9200","opaque_id":"035b244a-5832-453c-8bbe-0ac6c21fb356","type":1},"Timestamp":{"nanos":335147000,"seconds":1704963715},"UploadID":"c46f7026-65fc-4a05-99d3-2a5d844e420c"},"ID":"4e682e7a-1f19-48ea-a696-71c3db0f167d","TraceParent":"00-213c6ecb6f3601f46ea90bd100fdaa6e-55abd3628ee865b6-00","Type":"events.UploadReady"} service=clientlog
2024-01-11T10:01:55+01:00 ERR error getting resource error="unexpected status code while getting resource: CODE_NOT_FOUND" event={"Event":{"ExecutingUser":{"id":{"idp":"https://localhost:9200","opaque_id":"67940ad7-6b2a-40ee-ad87-a47dec9f372e"},"username":"perf-test-user-36"},"Failed":false,"FileRef":{"path":"./perftestuser36-mcpchbyhpv-iteration-3/small.zip","resource_id":{"opaque_id":"67940ad7-6b2a-40ee-ad87-a47dec9f372e","space_id":"67940ad7-6b2a-40ee-ad87-a47dec9f372e","storage_id":"2c794f34-d2b6-4a44-9aaa-25dacc451519"}},"Filename":"small.zip","SpaceOwner":{"idp":"https://localhost:9200","opaque_id":"67940ad7-6b2a-40ee-ad87-a47dec9f372e","type":1},"Timestamp":{"nanos":530152000,"seconds":1704963715},"UploadID":"b314955a-e82e-4bce-822b-3f98ea33d298"},"ID":"dfad35e7-a71b-4b68-93ab-8ce6cf6d01f6","TraceParent":"00-213c6ecb6f3601f46ea90bd100fdaa6e-55abd3628ee865b6-00","Type":"events.UploadReady"} service=clientlog
2024-01-11T10:01:55+01:00 ERR error getting resource error="unexpected status code while getting resource: CODE_NOT_FOUND" event={"Event":{"ExecutingUser":{"id":{"idp":"https://localhost:9200","opaque_id":"6df2ebd3-d30a-422e-9fff-eabf56e9d431"},"username":"perf-test-user-22"},"Failed":false,"FileRef":{"path":"./perftestuser22-hvmctcluyt-iteration-4/medium.zip","resource_id":{"opaque_id":"6df2ebd3-d30a-422e-9fff-eabf56e9d431","space_id":"6df2ebd3-d30a-422e-9fff-eabf56e9d431","storage_id":"2c794f34-d2b6-4a44-9aaa-25dacc451519"}},"Filename":"medium.zip","SpaceOwner":{"idp":"https://localhost:9200","opaque_id":"6df2ebd3-d30a-422e-9fff-eabf56e9d431","type":1},"Timestamp":{"nanos":666745000,"seconds":1704963715},"UploadID":"93e0e69f-e9b4-43b2-b157-de5a7d918573"},"ID":"f130f3db-8926-4561-bd72-4b4a3afd5801","TraceParent":"00-213c6ecb6f3601f46ea90bd100fdaa6e-55abd3628ee865b6-00","Type":"events.UploadReady"} service=clientlog
2024-01-11T10:01:56+01:00 ERR error getting resource error="unexpected status code while getting resource: CODE_NOT_FOUND" event={"Event":{"ExecutingUser":{"id":{"idp":"https://localhost:9200","opaque_id":"a8dd9a28-d6d2-4f3d-9bd9-464474adf8d7"},"username":"perf-test-user-19"},"Failed":false,"FileRef":{"path":"./perftestuser19-zgggpfpqlc-iteration-4/medium.zip","resource_id":{"opaque_id":"a8dd9a28-d6d2-4f3d-9bd9-464474adf8d7","space_id":"a8dd9a28-d6d2-4f3d-9bd9-464474adf8d7","storage_id":"2c794f34-d2b6-4a44-9aaa-25dacc451519"}},"Filename":"medium.zip","SpaceOwner":{"idp":"https://localhost:9200","opaque_id":"a8dd9a28-d6d2-4f3d-9bd9-464474adf8d7","type":1},"Timestamp":{"nanos":751514000,"seconds":1704963715},"UploadID":"df33eb53-fc74-4103-bcd7-03881f0723d5"},"ID":"bb12994b-ce6d-47b5-b3d9-978f4ccd5d47","TraceParent":"00-213c6ecb6f3601f46ea90bd100fdaa6e-55abd3628ee865b6-00","Type":"events.UploadReady"} service=clientlog
2024-01-11T10:01:56+01:00 ERR error getting resource error="unexpected status code while getting resource: CODE_NOT_FOUND" event={"Event":{"ExecutingUser":{"id":{"idp":"https://localhost:9200","opaque_id":"4f044900-5b59-415f-9cd4-623b45971e71"},"username":"perf-test-user-67"},"Failed":false,"FileRef":{"path":"./perftestuser67-rrvbzuxybd-iteration-1/small.zip","resource_id":{"opaque_id":"4f044900-5b59-415f-9cd4-623b45971e71","space_id":"4f044900-5b59-415f-9cd4-623b45971e71","storage_id":"2c794f34-d2b6-4a44-9aaa-25dacc451519"}},"Filename":"small.zip","SpaceOwner":{"idp":"https://localhost:9200","opaque_id":"4f044900-5b59-415f-9cd4-623b45971e71","type":1},"Timestamp":{"nanos":859705000,"seconds":1704963715},"UploadID":"c16d7803-c4ef-4f5b-8b67-fe4578ac95ae"},"ID":"4678c479-3957-42e8-9fd7-a16b1d354105","TraceParent":"00-213c6ecb6f3601f46ea90bd100fdaa6e-55abd3628ee865b6-00","Type":"events.UploadReady"} service=clientlog
2024-01-11T10:01:58+01:00 ERR error walking the tree error="error: not found: list container: error: not found: " service=search
2024-01-11T10:01:58+01:00 ERR error while indexing a space error="error: not found: list container: error: not found: " service=search spaceID={"opaque_id":"2c794f34-d2b6-4a44-9aaa-25dacc451519$a8dd9a28-d6d2-4f3d-9bd9-464474adf8d7"} userID={"idp":"https://localhost:9200","opaque_id":"a8dd9a28-d6d2-4f3d-9bd9-464474adf8d7","type":1}
2024-01-11T10:01:58+01:00 ERR error walking the tree error="error: not found: list container: error: not found: " service=search
2024-01-11T10:01:58+01:00 ERR error while indexing a space error="error: not found: list container: error: not found: " service=search spaceID={"opaque_id":"2c794f34-d2b6-4a44-9aaa-25dacc451519$4f044900-5b59-415f-9cd4-623b45971e71"} userID={"idp":"https://localhost:9200","opaque_id":"4f044900-5b59-415f-9cd4-623b45971e71","type":1}
2024-01-11T10:01:59+01:00 ERR error getting resource error="unexpected status code while getting resource: CODE_NOT_FOUND" event={"Event":{"ExecutingUser":{"id":{"idp":"https://localhost:9200","opaque_id":"4083275c-e4b4-46fd-98f8-545a93659ff9"},"username":"perf-test-user-52"},"Failed":false,"FileRef":{"path":"./perftestuser52-mcgyhkhjvo-iteration-2/small.zip","resource_id":{"opaque_id":"4083275c-e4b4-46fd-98f8-545a93659ff9","space_id":"4083275c-e4b4-46fd-98f8-545a93659ff9","storage_id":"2c794f34-d2b6-4a44-9aaa-25dacc451519"}},"Filename":"small.zip","SpaceOwner":{"idp":"https://localhost:9200","opaque_id":"4083275c-e4b4-46fd-98f8-545a93659ff9","type":1},"Timestamp":{"nanos":446277000,"seconds":1704963716},"UploadID":"f20af015-b14d-495e-bc5f-afd3e9dad3d9"},"ID":"a6440ea6-5344-49f4-8576-08074dd140eb","TraceParent":"00-213c6ecb6f3601f46ea90bd100fdaa6e-55abd3628ee865b6-00","Type":"events.UploadReady"} service=clientlog
2024-01-11T10:01:59+01:00 ERR failed to stat the moved resource res={"status":{"code":6,"message":"stat: error: not found: ccde2e81-dd5c-4d85-86bf-da2f586ab2d8/perftestuser46-qhlswcndwx-iteration-2-renamed","trace":"e72d06beb800e8e360d3e141b7621507"}} service=search
2024-01-11T10:01:59+01:00 ERR error getting resource error="unexpected status code while getting resource: CODE_NOT_FOUND" event={"Event":{"ExecutingUser":{"id":{"idp":"https://localhost:9200","opaque_id":"4421596c-42d7-4840-934d-bf90154dc9e9"},"username":"perf-test-user-1"},"Failed":false,"FileRef":{"path":"./perftestuser1-dhppabkofr-iteration-5/small.zip","resource_id":{"opaque_id":"4421596c-42d7-4840-934d-bf90154dc9e9","space_id":"4421596c-42d7-4840-934d-bf90154dc9e9","storage_id":"2c794f34-d2b6-4a44-9aaa-25dacc451519"}},"Filename":"small.zip","SpaceOwner":{"idp":"https://localhost:9200","opaque_id":"4421596c-42d7-4840-934d-bf90154dc9e9","type":1},"Timestamp":{"nanos":609137000,"seconds":1704963716},"UploadID":"53d092b6-9bcb-41e8-9329-a369f387444d"},"ID":"bba49344-63d3-4068-8bc1-b9e97d673504","TraceParent":"00-213c6ecb6f3601f46ea90bd100fdaa6e-55abd3628ee865b6-00","Type":"events.UploadReady"} service=clientlog
2024-01-11T10:01:59+01:00 ERR failed to stat the moved resource res={"status":{"code":6,"message":"stat: error: not found: 064ca3e6-6ed5-416e-b162-e5e23fb131cd/perftestuser69-ufuuqswoix-iteration-1","trace":"6db1d2b656b831bb98bbb01e18aff7ab"}} service=search
2024-01-11T10:01:59+01:00 ERR error getting resource error="unexpected status code while getting resource: CODE_NOT_FOUND" event={"Event":{"ExecutingUser":{"id":{"idp":"https://localhost:9200","opaque_id":"f2f0ec51-0003-4e43-bcda-475ee19f72ad"},"username":"perf-test-user-53"},"Failed":false,"FileRef":{"path":"./perftestuser53-bgaqgynvuu-iteration-2/small.zip","resource_id":{"opaque_id":"f2f0ec51-0003-4e43-bcda-475ee19f72ad","space_id":"f2f0ec51-0003-4e43-bcda-475ee19f72ad","storage_id":"2c794f34-d2b6-4a44-9aaa-25dacc451519"}},"Filename":"small.zip","SpaceOwner":{"idp":"https://localhost:9200","opaque_id":"f2f0ec51-0003-4e43-bcda-475ee19f72ad","type":1},"Timestamp":{"nanos":759081000,"seconds":1704963716},"UploadID":"1f681d8c-fe0e-4686-8352-e0e5ad57356a"},"ID":"15f51a21-cecc-47d8-a5ce-ea8793f783c6","TraceParent":"00-213c6ecb6f3601f46ea90bd100fdaa6e-55abd3628ee865b6-00","Type":"events.UploadReady"} service=clientlog
2024-01-11T10:01:59+01:00 ERR failed to stat the moved resource res={"status":{"code":6,"message":"stat: error: not found: perftestuser69-ufuuqswoix-iteration-1","trace":"332eaf919e8c2b8535d531b77682223f"}} service=search
2024-01-11T10:01:59+01:00 ERR failed to stat the moved resource res={"status":{"code":6,"message":"stat: error: not found: perftestuser46-qhlswcndwx-iteration-2-renamed","trace":"6914ca09e4326373bd2ec0bb3a16fdce"}} service=search
2024-01-11T10:01:59+01:00 ERR failed to stat the moved resource res={"status":{"code":6,"message":"stat: error: not found: 81621846-a09d-4893-8092-f99e1bedc879/perftestuser71-gwmekpbhan-iteration-1","trace":"833f41c47cadc6a1ffd5563bf64d1013"}} service=search
2024-01-11T10:01:59+01:00 ERR error getting resource error="unexpected status code while getting resource: CODE_NOT_FOUND" event={"Event":{"ExecutingUser":{"id":{"idp":"https://localhost:9200","opaque_id":"b3418cd7-6c9b-473c-891d-2c7712185f30"},"username":"perf-test-user-23"},"Failed":false,"FileRef":{"path":"./perftestuser23-scsrxmjiwr-iteration-4/small.zip","resource_id":{"opaque_id":"b3418cd7-6c9b-473c-891d-2c7712185f30","space_id":"b3418cd7-6c9b-473c-891d-2c7712185f30","storage_id":"2c794f34-d2b6-4a44-9aaa-25dacc451519"}},"Filename":"small.zip","SpaceOwner":{"idp":"https://localhost:9200","opaque_id":"b3418cd7-6c9b-473c-891d-2c7712185f30","type":1},"Timestamp":{"nanos":869345000,"seconds":1704963716},"UploadID":"dc40d4c2-0a51-454e-ab83-08fad1c95601"},"ID":"0c807f4d-8b9f-4a58-a846-4554927be189","TraceParent":"00-213c6ecb6f3601f46ea90bd100fdaa6e-55abd3628ee865b6-00","Type":"events.UploadReady"} service=clientlog
2024-01-11T10:01:59+01:00 ERR failed to stat the moved resource res={"status":{"code":6,"message":"stat: error: not found: 803bce0a-df6c-4bef-9463-490201b77d88/perftestuser10-xwlvzhqjlu-iteration-5","trace":"af180740dcbc3f2d5348569009c853a5"}} service=search
2024-01-11T10:01:59+01:00 ERR failed to stat the moved resource res={"status":{"code":6,"message":"stat: error: not found: perftestuser71-gwmekpbhan-iteration-1","trace":"da439be8cf8ee5b6f712037327f5b453"}} service=search
2024-01-11T10:01:59+01:00 ERR error getting resource error="unexpected status code while getting resource: CODE_NOT_FOUND" event={"Event":{"ExecutingUser":{"id":{"idp":"https://localhost:9200","opaque_id":"81621846-a09d-4893-8092-f99e1bedc879"},"username":"perf-test-user-71"},"Failed":false,"FileRef":{"path":"./perftestuser71-gwmekpbhan-iteration-1/medium.zip","resource_id":{"opaque_id":"81621846-a09d-4893-8092-f99e1bedc879","space_id":"81621846-a09d-4893-8092-f99e1bedc879","storage_id":"2c794f34-d2b6-4a44-9aaa-25dacc451519"}},"Filename":"medium.zip","SpaceOwner":{"idp":"https://localhost:9200","opaque_id":"81621846-a09d-4893-8092-f99e1bedc879","type":1},"Timestamp":{"nanos":24402000,"seconds":1704963717},"UploadID":"cf95b7ef-e1ad-4539-9fc6-a41d20ae55a6"},"ID":"69690ed6-ff8f-4ffc-8bba-8a42f86da201","TraceParent":"00-213c6ecb6f3601f46ea90bd100fdaa6e-55abd3628ee865b6-00","Type":"events.UploadReady"} service=clientlog
2024-01-11T10:01:59+01:00 ERR failed to stat the moved resource res={"status":{"code":6,"message":"stat: error: not found: perftestuser10-xwlvzhqjlu-iteration-5","trace":"f6cd717e3ab1933520ed38c1ca615450"}} service=search
2024-01-11T10:02:00+01:00 ERR error getting resource error="unexpected status code while getting resource: CODE_NOT_FOUND" event={"Event":{"ExecutingUser":{"id":{"idp":"https://localhost:9200","opaque_id":"803bce0a-df6c-4bef-9463-490201b77d88"},"username":"perf-test-user-10"},"Failed":false,"FileRef":{"path":"./perftestuser10-xwlvzhqjlu-iteration-5/small.zip","resource_id":{"opaque_id":"803bce0a-df6c-4bef-9463-490201b77d88","space_id":"803bce0a-df6c-4bef-9463-490201b77d88","storage_id":"2c794f34-d2b6-4a44-9aaa-25dacc451519"}},"Filename":"small.zip","SpaceOwner":{"idp":"https://localhost:9200","opaque_id":"803bce0a-df6c-4bef-9463-490201b77d88","type":1},"Timestamp":{"nanos":395994000,"seconds":1704963717},"UploadID":"cb8d5713-f98a-4bff-9bbc-3841ba4f4ba6"},"ID":"ef1607da-678c-41f6-8b9f-3388fbbbc497","TraceParent":"00-213c6ecb6f3601f46ea90bd100fdaa6e-55abd3628ee865b6-00","Type":"events.UploadReady"} service=clientlog
2024-01-11T10:02:00+01:00 ERR error getting resource error="unexpected status code while getting resource: CODE_NOT_FOUND" event={"Event":{"ExecutingUser":{"id":{"idp":"https://localhost:9200","opaque_id":"b90c3a66-450e-4f07-8cbd-9a7da3c2504f"},"username":"perf-test-user-72"},"Failed":false,"FileRef":{"path":"./perftestuser72-rhhvxjetjk-iteration-1/small.zip","resource_id":{"opaque_id":"b90c3a66-450e-4f07-8cbd-9a7da3c2504f","space_id":"b90c3a66-450e-4f07-8cbd-9a7da3c2504f","storage_id":"2c794f34-d2b6-4a44-9aaa-25dacc451519"}},"Filename":"small.zip","SpaceOwner":{"idp":"https://localhost:9200","opaque_id":"b90c3a66-450e-4f07-8cbd-9a7da3c2504f","type":1},"Timestamp":{"nanos":476206000,"seconds":1704963718},"UploadID":"7b8b5846-9213-4867-8240-cc21622f6da1"},"ID":"c1a75cc0-f2b7-44b5-b358-133f3e3a15fa","TraceParent":"00-213c6ecb6f3601f46ea90bd100fdaa6e-55abd3628ee865b6-00","Type":"events.UploadReady"} service=clientlog
2024-01-11T10:02:00+01:00 ERR failed to stat the moved resource res={"status":{"code":6,"message":"stat: error: not found: b90c3a66-450e-4f07-8cbd-9a7da3c2504f/perftestuser72-rhhvxjetjk-iteration-1","trace":"9bb2aa72034e8146b0aec3acc5824bc2"}} service=search
2024-01-11T10:02:00+01:00 ERR failed to stat the moved resource res={"status":{"code":6,"message":"stat: error: not found: 2c7ebed8-3550-411f-a5a3-8c02a7ee690a/perftestuser26-zxlwlywpsb-iteration-4","trace":"9abb1357d28a899be3e480dacb63a72e"}} service=search
2024-01-11T10:02:00+01:00 ERR failed to stat the moved resource res={"status":{"code":6,"message":"stat: error: not found: 3622000d-3f62-41cc-bfee-3a10618623bb/perftestuser66-fgcwlvpeod-iteration-1","trace":"98c932b05beb048ac51b1c05f43542d6"}} service=search
2024-01-11T10:02:00+01:00 ERR error getting resource error="unexpected status code while getting resource: CODE_NOT_FOUND" event={"Event":{"ExecutingUser":{"id":{"idp":"https://localhost:9200","opaque_id":"3622000d-3f62-41cc-bfee-3a10618623bb"},"username":"perf-test-user-66"},"Failed":false,"FileRef":{"path":"./perftestuser66-fgcwlvpeod-iteration-1/large.zip","resource_id":{"opaque_id":"3622000d-3f62-41cc-bfee-3a10618623bb","space_id":"3622000d-3f62-41cc-bfee-3a10618623bb","storage_id":"2c794f34-d2b6-4a44-9aaa-25dacc451519"}},"Filename":"large.zip","SpaceOwner":{"idp":"https://localhost:9200","opaque_id":"3622000d-3f62-41cc-bfee-3a10618623bb","type":1},"Timestamp":{"nanos":604127000,"seconds":1704963718},"UploadID":"1c07a609-05ac-4b24-8a91-8155a3660b60"},"ID":"4f13c27a-4cc3-4810-907c-35f60b84bc54","TraceParent":"00-213c6ecb6f3601f46ea90bd100fdaa6e-55abd3628ee865b6-00","Type":"events.UploadReady"} service=clientlog
2024-01-11T10:02:01+01:00 ERR failed to stat the moved resource res={"status":{"code":6,"message":"stat: error: not found: perftestuser26-zxlwlywpsb-iteration-4","trace":"f9ff2a41632424cf2be551d8a88e8ac9"}} service=search
write tcp 127.0.0.1:50429->127.0.0.1:9233: write: no buffer space available on connection [35]
2024-01-11T10:02:01+01:00 ERR error doing PUT request to data service error="Put \"http://localhost:9158/data/simple/1495e11a-68e2-4edb-ad99-1a6099ca7721\": write tcp 127.0.0.1:52477->127.0.0.1:9158: write: no buffer space available" pkg=rhttp request-id=VIKTORS-AIR.local/uQZBuP34zU-006293 service=frontend traceid=1536ccfaeec56386ee2cc9f4814b9165
2024-01-11T10:02:01+01:00 ERR http end="11/Jan/2024:10:02:01 +0100" host=127.0.0.1 method=PUT pkg=rhttp proto=HTTP/1.1 service=frontend size=0 start="11/Jan/2024:10:01:59 +0100" status=500 time_ns=1991746125 traceid=1536ccfaeec56386ee2cc9f4814b9165 uri=/data url=/
2024/01/11 10:02:01 http: proxy error: context canceled
2024-01-11T10:02:01+01:00 ERR error doing PUT request to data service error="Put \"https://localhost:9200/data\": write tcp [::1]:52548->[::1]:9200: write: broken pipe" name=com.owncloud.web.ocdav path=/perftestuser68-dpcnenjvad-iteration-1/large.zip request-id=VIKTORS-AIR.local/uQZBuP34zU-006258 service=ocdav spaceid=2c794f34-d2b6-4a44-9aaa-25dacc451519$457b00f1-9492-4e35-baef-5fc1e8438b3e traceid=00a966721a73aad9ebe3ab21120b9721
write tcp 127.0.0.1:50429->127.0.0.1:9233: write: no buffer space available on connection [35]
2024-01-11T10:02:01+01:00 ERR error getting resource error="unexpected status code while getting resource: CODE_NOT_FOUND" event={"Event":{"ExecutingUser":{"id":{"idp":"https://localhost:9200","opaque_id":"2c7ebed8-3550-411f-a5a3-8c02a7ee690a"},"username":"perf-test-user-26"},"Failed":false,"FileRef":{"path":"./perftestuser26-zxlwlywpsb-iteration-4/small.zip","resource_id":{"opaque_id":"2c7ebed8-3550-411f-a5a3-8c02a7ee690a","space_id":"2c7ebed8-3550-411f-a5a3-8c02a7ee690a","storage_id":"2c794f34-d2b6-4a44-9aaa-25dacc451519"}},"Filename":"small.zip","SpaceOwner":{"idp":"https://localhost:9200","opaque_id":"2c7ebed8-3550-411f-a5a3-8c02a7ee690a","type":1},"Timestamp":{"nanos":699583000,"seconds":1704963718},"UploadID":"d1780a05-c327-4065-bb4c-88ac4b7ea3d1"},"ID":"ab8f8fa5-c507-455e-82c5-9ed2b31c2305","TraceParent":"00-213c6ecb6f3601f46ea90bd100fdaa6e-55abd3628ee865b6-00","Type":"events.UploadReady"} service=clientlog
2024-01-11T10:02:01+01:00 ERR failed to stat the moved resource res={"status":{"code":6,"message":"stat: error: not found: perftestuser66-fgcwlvpeod-iteration-1","trace":"0720c7af4f6587e8d3cec7dff02dddf6"}} service=search
2024-01-11T10:02:01+01:00 ERR failed to stat the moved resource res={"status":{"code":6,"message":"stat: error: not found: perftestuser72-rhhvxjetjk-iteration-1","trace":"970a88a83e70b04cae56b15853356b7c"}} service=search
write tcp 127.0.0.1:50429->127.0.0.1:9233: write: no buffer space available on connection [35]
write tcp 127.0.0.1:50426->127.0.0.1:9233: write: no buffer space available on connection [26]
write tcp 127.0.0.1:50426->127.0.0.1:9233: write: no buffer space available on connection [26]
2024-01-11T10:02:02+01:00 ERR error doing PUT request to data service error="Put \"https://localhost:9200/data\": write tcp [::1]:52769->[::1]:9200: write: no buffer space available" name=com.owncloud.web.ocdav path=/perftestuser14-rewfyadddt-iteration-5/medium.zip request-id=VIKTORS-AIR.local/uQZBuP34zU-006476 service=ocdav spaceid=2c794f34-d2b6-4a44-9aaa-25dacc451519$f0ce7134-2c51-4c88-84e1-12f4a9d80826 traceid=66bd6645246874e611e02e7a88c34641
write tcp 127.0.0.1:50426->127.0.0.1:9233: write: no buffer space available on connection [26]
2024-01-11T10:02:07+01:00 ERR failed to publish FileUploaded event error="Error publishing message to topic: nats: timeout" datatx=simple pkg=rhttp request-id=VIKTORS-AIR.local/uQZBuP34zU-006532 service=storage-users traceid=f186091355f9d1b684ca955fb498712c
2024-01-11T10:02:07+01:00 ERR failed to publish FileUploaded event error="Error publishing message to topic: nats: timeout" datatx=simple pkg=rhttp request-id=VIKTORS-AIR.local/uQZBuP34zU-006294 service=storage-users traceid=0e375940cc529fb5c5b3a3bbe436a0a3
{"level":"error","error":"Error publishing message to topic: nats: timeout","uploadID":"0598fc36-6fc4-4719-9a71-bc378c1a1d37","time":"2024-01-11T10:02:07+01:00","message":"Failed to publish UploadReady event"}
write tcp 127.0.0.1:50426->127.0.0.1:9233: write: no buffer space available on connection [26]
write tcp 127.0.0.1:50426->127.0.0.1:9233: write: no buffer space available on connection [26]
2024-01-11T10:02:07+01:00 ERR error uploading file error="Error publishing message to topic: nats: timeout" datatx=simple pkg=rhttp request-id=VIKTORS-AIR.local/uQZBuP34zU-006543 service=storage-users traceid=61ed9e4a377bce0529578a4a774555bf
2024-01-11T10:02:07+01:00 ERR http end="11/Jan/2024:10:02:07 +0100" host=127.0.0.1 method=PUT pkg=rhttp proto=HTTP/1.1 service=storage-users size=0 start="11/Jan/2024:10:02:02 +0100" status=500 time_ns=5676209958 traceid=61ed9e4a377bce0529578a4a774555bf uri=/data/simple/22d91d1d-7663-4d25-9452-dd996c8881cb url=/22d91d1d-7663-4d25-9452-dd996c8881cb
2024-01-11T10:02:07+01:00 ERR error uploading file error="Error publishing message to topic: nats: timeout" datatx=simple pkg=rhttp request-id=VIKTORS-AIR.local/uQZBuP34zU-006452 service=storage-users traceid=bd798d7ea6e2734f704f2ee1f84bd792
2024-01-11T10:02:07+01:00 ERR http end="11/Jan/2024:10:02:07 +0100" host=127.0.0.1 method=PUT pkg=rhttp proto=HTTP/1.1 service=storage-users size=0 start="11/Jan/2024:10:02:00 +0100" status=500 time_ns=7415354875 traceid=bd798d7ea6e2734f704f2ee1f84bd792 uri=/data/simple/139c4ad5-6bd9-4a0e-9cc4-e1bb06c7c2ea url=/139c4ad5-6bd9-4a0e-9cc4-e1bb06c7c2ea
2024-01-11T10:02:08+01:00 ERR error uploading file error="Error publishing message to topic: nats: timeout" datatx=simple pkg=rhttp request-id=VIKTORS-AIR.local/uQZBuP34zU-006268 service=storage-users traceid=eb0ce7ba60fd0500663aef7ca7d70193
2024-01-11T10:02:08+01:00 ERR http end="11/Jan/2024:10:02:08 +0100" host=127.0.0.1 method=PUT pkg=rhttp proto=HTTP/1.1 service=storage-users size=0 start="11/Jan/2024:10:01:58 +0100" status=500 time_ns=9115525541 traceid=eb0ce7ba60fd0500663aef7ca7d70193 uri=/data/simple/a52d7fc1-75dd-4c9e-8e54-9f198f866bc5 url=/a52d7fc1-75dd-4c9e-8e54-9f198f866bc5
2024-01-11T10:02:08+01:00 ERR http end="11/Jan/2024:10:02:08 +0100" host=127.0.0.1 method=PUT pkg=rhttp proto=HTTP/1.1 service=frontend size=0 start="11/Jan/2024:10:01:58 +0100" status=500 time_ns=9177486625 traceid=e98e8978f47460b02520079d95f23f95 uri=/data url=/

2024-01-11T10:02:08+01:00 ERR PUT request to data server failed name=com.owncloud.web.ocdav path=/perftestuser37-njkystytpp-iteration-3/large.zip request-id=VIKTORS-AIR.local/uQZBuP34zU-006220 service=ocdav spaceid=2c794f34-d2b6-4a44-9aaa-25dacc451519$ec45ba32-96ee-41e4-ba19-d0779e05db2d traceid=ce8cc32a1580da170a6009829341b10f
2024-01-11T10:02:09+01:00 ERR error uploading file error="Error publishing message to topic: nats: timeout" datatx=simple pkg=rhttp request-id=VIKTORS-AIR.local/uQZBuP34zU-006351 service=storage-users traceid=7db655c9fa4debff55e76e1687de8d7c

2024-01-11T10:02:09+01:00 ERR http end="11/Jan/2024:10:02:09 +0100" host=127.0.0.1 method=PUT pkg=rhttp proto=HTTP/1.1 service=storage-users size=0 start="11/Jan/2024:10:01:59 +0100" status=500 time_ns=9462621667 traceid=7db655c9fa4debff55e76e1687de8d7c uri=/data/simple/bba2404c-87f8-4ce2-aa4c-6f685d00c809 url=/bba2404c-87f8-4ce2-aa4c-6f685d00c809
2024-01-11T10:02:09+01:00 ERR http end="11/Jan/2024:10:02:09 +0100" host=127.0.0.1 method=PUT pkg=rhttp proto=HTTP/1.1 service=frontend size=0 start="11/Jan/2024:10:01:59 +0100" status=500 time_ns=9509731625 traceid=0d0f7e87fee05ede82e5e3d080eb0b55 uri=/data url=/

2024-01-11T10:02:09+01:00 ERR PUT request to data server failed name=com.owncloud.web.ocdav path=/perftestuser58-gvtvpxbmmi-iteration-2/large.zip request-id=VIKTORS-AIR.local/uQZBuP34zU-006307 service=ocdav spaceid=2c794f34-d2b6-4a44-9aaa-25dacc451519$61621bf5-6ed8-4f76-a68e-1f77e43bd0cd traceid=75a5befec54c3611d795504ba594b415

{"level":"error","error":"Error publishing message to topic: nats: timeout","uploadID":"ff88a09e-30b9-4163-ab10-18ac3d498361","time":"2024-01-11T10:02:12+01:00","message":"Failed to publish UploadReady event"}
write tcp 127.0.0.1:50426->127.0.0.1:9233: write: no buffer space available on connection [26]

2024-01-11T10:05:07+01:00 ERR Could not get user by claim error="could not get user by claim username with value perf-test-user-69: rpc error: code = Canceled desc = context canceled" service=proxy

## k6_log

          /\      |‾‾| /‾‾/   /‾‾/   
     /\  /  \     |  |/  /   /  /    
    /  \/    \    |     (   /   ‾‾\  
   /          \   |  |\  \ |  (‾)  | 
  / __________ \  |__| \__\ \_____/ .io

  execution: local
     script: packages/k6-tests/artifacts/koko-platform-040-create-upload-rename-delete-folder-and-file-ramping-k6.js
     output: -

  scenarios: (100.00%) 1 scenario, 100 max VUs, 7m30s max duration (incl. graceful stop):
           * create_upload_rename_delete_folder_and_file_040: Up to 100 looping VUs for 7m0s over 3 stages (gracefulRampDown: 30s, exec: create_upload_rename_delete_folder_and_file_040, gracefulStop: 30s)

WARN[0056] Request Failed                                error="Put \"https://localhost:9200/remote.php/dav/spaces/2c794f34-d2b6-4a44-9aaa-25dacc451519$36dd8ce7-0d5e-4ee4-90cd-002e542cad02/perftestuser7-wcwhzpbbtn-iteration-6/medium.zip\": write tcp 127.0.0.1:51166->127.0.0.1:9200: write: no buffer space available"
WARN[0056] Request Failed                                error="Put \"https://localhost:9200/remote.php/dav/spaces/2c794f34-d2b6-4a44-9aaa-25dacc451519$f5910fb9-6d6b-4d85-a999-e68d0d144a2d/perftestuser2-zswifccygp-iteration-6/medium.zip\": write tcp 127.0.0.1:51125->127.0.0.1:9200: write: no buffer space available"
WARN[0056] Request Failed                                error="Put \"https://localhost:9200/remote.php/dav/spaces/2c794f34-d2b6-4a44-9aaa-25dacc451519$457b00f1-9492-4e35-baef-5fc1e8438b3e/perftestuser68-dpcnenjvad-iteration-1/large.zip\": write tcp 127.0.0.1:52404->127.0.0.1:9200: write: no buffer space available"
WARN[0057] Request Failed                                error="Put \"https://localhost:9200/remote.php/dav/spaces/2c794f34-d2b6-4a44-9aaa-25dacc451519$56060c6a-b4cb-43ba-9de2-26b61a00f3f1/perftestuser11-przwrtdkks-iteration-5/large.zip\": write tcp 127.0.0.1:51309->127.0.0.1:9200: write: no buffer space available"
WARN[0057] Request Failed                                error="Put \"https://localhost:9200/remote.php/dav/spaces/2c794f34-d2b6-4a44-9aaa-25dacc451519$06750f65-12bb-4074-89f6-dc6fb1cac24e/perftestuser24-sbnavodfql-iteration-5/small.zip\": write tcp 127.0.0.1:51392->127.0.0.1:9200: write: no buffer space available"
WARN[0057] Request Failed                                error="Put \"https://localhost:9200/remote.php/dav/spaces/2c794f34-d2b6-4a44-9aaa-25dacc451519$82d9f959-4e65-42d3-8f10-5c1d06db5bc6/perftestuser32-kwzxzwkkyp-iteration-4/large.zip\": write tcp 127.0.0.1:51538->127.0.0.1:9200: write: no buffer space available"
WARN[0057] Request Failed                                error="Put \"https://localhost:9200/remote.php/dav/spaces/2c794f34-d2b6-4a44-9aaa-25dacc451519$f0ce7134-2c51-4c88-84e1-12f4a9d80826/perftestuser14-rewfyadddt-iteration-5/medium.zip\": write tcp 127.0.0.1:51283->127.0.0.1:9200: write: broken pipe"
INFO[0057] 2024/01/11 10:02:02 Unsolicited response received on idle HTTP channel starting with "HTTP/1.1 400 Bad Request\r\nContent-Type: text/plain; charset=utf-8\r\nConnection: close\r\n\r\n400 Bad Request"; err=<nil> 
WARN[0115] Request Failed                                error="Get \"https://localhost:9200/graph/v1.0/me/drives?%24filter=driveType+eq+%27personal%27\": request timeout"
WARN[0115] Request Failed                                error="Get \"https://localhost:9200/graph/v1.0/me/drives?%24filter=driveType+eq+%27personal%27\": request timeout"
WARN[0115] Request Failed                                error="Move \"https://localhost:9200/remote.php/dav/spaces/2c794f34-d2b6-4a44-9aaa-25dacc451519$13f0bb7b-269a-4294-a843-986d2717762c/perftestuser64-wtikvgfvsd-iteration-1\": request timeout"
WARN[0115] Request Failed                                error="Move \"https://localhost:9200/remote.php/dav/spaces/2c794f34-d2b6-4a44-9aaa-25dacc451519$22929c12-7e13-4f74-851a-d27f3b3a3090/perftestuser18-xdcaxkgktj-iteration-4\": request timeout"
WARN[0115] Request Failed                                error="Move \"https://localhost:9200/remote.php/dav/spaces/2c794f34-d2b6-4a44-9aaa-25dacc451519$57a9031b-e144-4455-a684-52b3f862977a/perftestuser57-nymyxnzvlf-iteration-2\": request timeout"
WARN[0115] Request Failed                                error="Move \"https://localhost:9200/remote.php/dav/spaces/2c794f34-d2b6-4a44-9aaa-25dacc451519$1929c7e3-1451-4c7c-934c-6d5b49edb744/perftestuser62-kvcniccjhh-iteration-1\": request timeout"
WARN[0115] Request Failed                                error="Move \"https://localhost:9200/remote.php/dav/spaces/2c794f34-d2b6-4a44-9aaa-25dacc451519$c5ebd98b-ef40-4943-8293-d70b6200fbae/perftestuser21-fhejnbeack-iteration-4\": request timeout"
WARN[0115] Request Failed                                error="Put \"https://localhost:9200/remote.php/dav/spaces/2c794f34-d2b6-4a44-9aaa-25dacc451519$2c0f2358-1576-4862-a8d0-936a0ba7ac87/perftestuser39-euqfdoqjyt-iteration-3/small.zip\": request timeout"

CC @aduffeck

[aduffeck]

@ScharfViktor We've already seen these issues in the past, the problem with indexing spaces or resources is that the search service is triggered with a delay (debounced) and the data in question can already be gone by the test because it's not needed anymore.
So, do these errors cause actual problems or is it just something you noticed?

[dragonchaser]

Action: This should not be an error log, change the log level to DEBUG, the behavior is expected.