exclude big files?

[niksfirefly]

Is there a way to exclude big files (bigger/smaller --than-size) with borg create?

[ThomasWaldmann]

No (not yet?).

See also: jborg/attic#330

[niksfirefly]

ok I assume it is not on roadmap list?

[ThomasWaldmann]

it has not been assigned to a milestone yet.

if you want to help, you could try the workaround suggested in the linked ticket.
if it works well. we maybe do not need the feature.
if it doesn't work well, tell us why - this might be a reason to add this feature.

[edgewood]

This would be a good place to use --files-from (on my phone and can't find the ticket #, but it's been discussed recently).

[enkore]

#841

[cbothamy]

The solution linked in the ticket obviously works.

However, I'd rather have all the excluded files directive at one single place, and it's quite uneasy to have to run another process and rely on a temporary file before running borg. (also, the find -size syntax is quite clumsy, for example try find /home -size -1M ...).

I have a small 50 lines patch that extends patterns.py and adds the possibility to exclude files based on filesize and optionnally file extension.

If anybody is interested, I can add tests for this new directive, and forward the diff to the mailing list.

[n-st]

Picking this back up because I have another slightly different use case (→ #3097).

The workaround will cause a significant performance penalty on anything beyond tiny datasets (anything larger than your RAM, I'd guess) and, more importantly. introduces a race condition between generating the file list and actually processing the files.

We're already looking at the files' stat during processing, so theoretically we'd only need two new arguments in argparse, an "x MB"-to-"n bytes" conversion and an if condition in do_create (near the existing check for one_file_system).

I'd vote for using the same option names and syntax as rsync (--min-size=42K / --max-size=123M) as it is easy to understand for the user and easy to parse for the program (easier than find's -size or the --exclude-by-size '>=15M' from the attic bug report, at any rate).

[varac]

Thx for assiging this to a milestone, looking forward to it!

[ThomasWaldmann]

superseded by #4102.