Ability to define cache-control property

[pkuczynski]

Feature Request Description

Based on the documentation it is possible to define cache-control property when uploading a blob:

https://docs.microsoft.com/en-us/azure/cdn/cdn-manage-expiration-of-blob-content

azure storage blob upload -c <connectionstring> -p cacheControl="max-age=3600" .\<blob name> <container name> <blob name>

Such functionality is missing from blobxfer.

Describe Preferred Solution

Would you consider adding --cache-control "max-age=3600" option to the command line interface?

[alfpark]

This option was added in 1.7.0 as the CLI option --file-cache-control or cache_control under upload:options:store_file_properties in YAML config.

Thanks for the feedback.

[pkuczynski]

Thank you for fairly quick reaction! Unfortunately presented switch does not seem to have any effect. Here is the command I am running:

blobxfer upload --storage-account ... --remote-path '$web' --local-path build/app --exclude index.html --file-cache-control 'public, max-age=2592000'

Could that be caused by the fact I am uploading to the blob container and not the file share?

Could you please re-open this issue?

[pkuczynski]

In case you need this, here is the output of this command:

2019-02-27 14:22:23.050 DEBUG - credential: account=... endpoint=core.windows.net is_sas=False can_create_containers=True can_list_container_objects=True can_read_object=True can_write_object=True
2019-02-27 14:22:23.052 INFO - 
============================================
         Azure blobxfer parameters
============================================
         blobxfer version: 1.7.0
                 platform: Linux-4.4.0-141-generic-x86_64-with
               components: CPython=3.6.8-64bit azstor.blob=1.5.0 azstor.file=1.4.0 crypt=2.5 req=2.21.0
       transfer direction: local -> Azure
                  workers: disk=64 xfer=96 md5=0 crypto=0
                 log file: None
                  dry run: False
              resume file: None
                  timeout: connect=10 read=200 max_retries=1000
                     mode: StorageModes.Auto
                  skip on: fs_match=False lmt_ge=False md5=False
        delete extraneous: False
                overwrite: True
                recursive: True
            rename single: False
              access tier: None
         chunk size bytes: 0
           one shot bytes: 0
         strip components: 0
         store properties: attr=False cc=public, max-age=2592000 md5=False
           rsa public key: None
       local source paths: build/app
============================================
2019-02-27 14:22:23.052 INFO - blobxfer start time: 2019-02-27 14:22:23.052369+00:00
2019-02-27 14:22:23.052 DEBUG - spawning 64 disk threads
2019-02-27 14:22:23.080 DEBUG - spawning 96 transfer threads
2019-02-27 14:22:25.716 DEBUG - 0 files 0.0000 MiB filesize and/or lmt_ge skipped
2019-02-27 14:22:25.716 DEBUG - 18 local files processed, waiting for upload completion of approx. 1.9250 MiB
2019-02-27 14:22:25.821 INFO - elapsed upload + verify time and throughput of 0.0019 GiB: 2.257 sec, 6.8225 Mbps (0.853 MiB/s)
2019-02-27 14:22:25.821 INFO - blobxfer end time: 2019-02-27 14:22:25.821806+00:00 (elapsed: 2.769 sec)

[alfpark]

Can you re-execute your command with -v --enable-azure-storage-logger, scope the transfer to a single file, and post the resulting redacted log?

[pkuczynski]

Of course! Thanks for the help on this!

Here comes the gist https://gist.github.com/pkuczynski/3be3e7ce568776abd13de99666465576

I forgot to add -v. Should I run it again?

[alfpark]

If you search your gist for x-ms-blob-cache-control, the correct value is being sent/set. Are you positive you aren't seeing the CacheControl blob property on the associated blobs if viewed in Azure Storage Explorer?

[pkuczynski]

I checked now again and magically the header was there. Not sure, maybe it was a cache issue, but I swear I hard reloaded. Sorry for confusion and thanks for your work on this! :)