Skip to content

latest

Pre-release
Pre-release
Compare
Choose a tag to compare
@github-actions github-actions released this 13 Apr 15:18
· 0 commits to c755dc088b5fc15b65f7b5537c34f7a90b228e2c since this release
Use an argsfile when calling merge_index_store.py

Summary:
## Problem

Previously, larger buck targets (those with several thousand transitive dependencies) would fail when building the `full-index-store`, because the CLI invocation would be too big and hit Linux limits (around [128KiB](https://unix.stackexchange.com/questions/110282/cp-max-source-files-number-arguments-for-copy-utility/)).

This would produce build failures like this:

```
$ buck build fbobjc/Apps/Instagram/Features/Sundial/IGSundial:IGSundial[full-index-store]
Action failed: fbsource//fbobjc/Apps/Instagram/Features/Sundial/IGSundial:IGSundial (merge_index_store __indexstore__/IGSundial/full_index_stores)
Local command returned non-zero exit code <no exit code>
Reproduce locally: `env -- 'BUCK_SCRATCH_PATH=buck-out/v2/tmp/fbsource/6e8c89e8087abbf6/merge_index_store/__indexstore__ ...<omitted>... ource/33ed8d67ac42318a/fbobjc/VendorLib/IGListKit/__IGListKit__/__indexstore__/IGListKit/index_store (run `buck2 log what-failed` to get the full command)`
```

See P1746393724 for an example output from `buck log what-failed`, it's clearly a ton of data.

## Solution

Instead of invoking `merge_index_store.py --source dir1 dir2 dir3 ...`, use an arguments file as `merge_index_store.py --source directories.txt` so the CLI invocation is short.

This is a pattern used in other buck rules, so I based this code on these examples:

https://www.internalfb.com/code/fbsource/[834112476033]/fbcode/buck2/prelude/cxx/symbols.bzl?lines=293

https://www.internalfb.com/code/fbsource/[fa0f657e098379489efb65a65b20ce0d269fb276][history]/fbcode/buck2/prelude/apple/swift/swift_compilation.bzl?lines=690

Differential Revision: D70747622

fbshipit-source-id: 4f7f187e3e01352ca2da687b46591dc8601335a8