Feature Request: Log file splitting\rotating.

[cdiskan]

Feature request: Log file maximum size switch, and\or automatic log splitting.

AZCopy currently creates a single log file for an entire job. When jobs are sufficiently large in file count the log becomes functionally unusable due to the extreme file size of the .log file.

Example situation: 50TB Azcopy sync job. Current log file with default logging options is 4GB and growing. This is large enough that it cannot be opened with common application. This defeats the purpose of the log file, and prevents diagnosis of failed files.

Reducing the log verbosity works in some situations, but for copies where a chain of custody is needed for data transfers the full verbosity log may be required.

Please add automatic log splitting at a reasonable file size (100mb? 500mb?) to the logging function, or a switch that allows setting the maximum log size before log rotation.

[adreed-msft]

Hi there!

I'll propose this to the team during our upcoming sync this week. Chances are it will not make it into 10.19.0, but perhaps a later release.

[cdiskan]

Thanks. For anyone else who runs into this issue, the Linux App "Less" works well for opening these large logs. There's a version wrapped in GnuWin that works for folks limited to operating on Windows systems.

[cdiskan]

@adreed-msft did this ever make it onto a roadmap? Running into this issue again with another large sync (75+TB, with about 25 million files).

Thanks!

[cdiskan]

@adreed-msft Following up again. Was this feature ever added to the roadmap for AZCopy?

[gapra-msft]

Hi! This feature request has been resolved as of our latest AzCopy release 10.22.0. Please upgrade! Closing this issue as a result. Thanks again for the feature request!

[cdiskan]

Thank you!!!!