Files not being downloaded from S3

[hummingbird1989]

My use-case is as follows:

• I have a file (filename.txt) with all the cp commands to copy files from an s3 bucket to another filesystem (cp "s3:src" "dst")
• I'm calling the commands through Python with a subprocess() call as follows:
  • s5cmd run filename.txt

Some files are being downloaded, but some files are not. Is there a way to log any errors or exceptions that might be happening? I tried --log debug, but didn't get anything about the errors

System settings:
Ubuntu 18.04.6 LTS
s5cmd v1.4.0

[ilkinulas]

Hello
Could you please try the same with the latest s5cmd release? v2.0.0-beta.2

And also could you please share the exit code and output of s5cmd run command (without python subprocess) ?

[hummingbird1989]

The command wasn't run through the command line, only through subprocess Files with a '#' in the name were not being downloaded. Is there a way to propagate the error to the subprocess call? Also, is there a timeline for the next stable release?

…

On Mon, Jun 27, 2022 at 1:17 PM İlkin Balkanay ***@***.***> wrote: Hello Could you please try the same with the latest s5cmd release? v2.0.0-beta.2 <https://github.com/peak/s5cmd/releases/tag/v2.0.0-beta.2> And also could you please share the exit code and output of s5cmd run command (without python subprocess) ? — Reply to this email directly, view it on GitHub <#443 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AT6LA5DBOA2RV5KGZPSU3GTVRHOZDANCNFSM5XRY3K7A> . You are receiving this because you authored the thread.Message ID: ***@***.***>

-- This message is intended for the use of the addressee and may contain  information that is privileged and/or confidential. If you are not the intended recipient, you are hereby notified that any dissemination, distribution or copying of the information contained in this message is strictly unauthorized and prohibited. If you have received this message in error, please notify the sender by reply e-mail and delete the message from your system.

[ilkinulas]

When this issue is closed we will release a stable version. (v2.0.0) (hopefully at the end of this week)

s5cmd should handle most special characters in filenames. Here is a test for # char.

> s5cmd version
v2.0.0-beta.2-7200415

> s5cmd cp test.txt s3://mybucket/test#1.txt
cp test.txt s3://mybucket/test#1.txt
> s5cmd cp s3://mybucket/test#1.txt .
cp s3://mybucket/test#1.txt test#1.txt

[kucukaslan]

Files with a '#' in the name were not being downloaded.

this might be related to #309. In the previous versions (including the current stable version v1.4.0) run subcommand supported inline comments that begins with hash mark (#). This bug is fixed in v2.0.0-beta.2 (by removing inline comment support).

[igungor]

v2.0.0 fixes this issue. Could you try it please?