In vendor and external repos, only generate rules needed to resolve dependencies

[jayconrod]

By default, Gazelle generates rules for every Go package in directory trees it runs in. This causes two problems:

• The dep tool copies whole repositories, including packages that are not used. Transitive dependencies of unused packages are not copied, which means some packages are not buildable. go build ./... does not attempt to build everything in vendor, but bazel build //... does. Gazelle should not generate rules for these packages.
• The same problem will happen when Gazelle supports automatically adding repository rules to WORKSPACE to satisfy unresolved imports. In external repositories, we should only generate rules for packages that will be built, and we should not consider transitive dependencies of unused packages.

[fejta]

This is the (partially) automated solution we are using for kubernetes/test-infra to address this:

dep ensure
dep prune
hack/update-bazel.sh # Delete all _test.go code, run gazelle and then kazel
drop-dep //vendor/random_child //vendor//unused_parent_or_sibling # Manually delete unnecessary targets from partially pruned packages 
hack/prune-libraries.sh # Run bazel query rdeps(//vendor/foo, //...) for every go library and delete libraries that only have one depencency, keep running until nothing changes
hack/update-bazel.sh # Rerun kazel essentially to fix up BUILD files

https://github.com/kubernetes/test-infra/blob/master/hack/update-deps.sh

It would obvioulsy be much nicer if this was a feature of gazelle (only produce targets for imported packages) and/or dep (prune test code, ununsed parent packages and optional build files)

[jayconrod]

@fejta Oof, sorry that's such a hassle. I can't speak for dep, but I'll make this simpler on the Gazelle side.

[MarkusTeufelberger]

Our current workaround:

Build:

bazel query --keep_going '//... except tests(//...) except attr("tags", "manual", //...)' | grep -v vendor | xargs bazel build

Run tests:

bazel query "tests(//...)" | grep -v vendor | xargs bazel test