Generate compilation database on load

[lionel-]

LSPs like clangd provide intellisense features for native files but they need to know the build flags (mostly include flags) for each root file in the project (flags for included files are inferred from root files). These tools are not able to infer the flags by themselves and so require the generation of a "compilation database": https://clang.llvm.org/docs/JSONCompilationDatabase.html.

For R packages the flags are generated by R CMD build and depend on the package Makevars file, possibly generated from configure. Generating the database after a load_all() seems like a reasonable update point to keep track of the changing source files (which can be added or removed during development) while ensuring the presence of an up-to-date Makevars file.

There are two main approaches for the generation of a compilation database. The first is to intercept the compiler calls during an actual build to collect arguments. This is the approach used by tools like bear and it works great in conjunction with R CMD build. The other is to generate build commands, e.g. with make --dry-run.

@kevinushey suggested we could use the latter approach with R CMD shlib --dry-run. This is a nice idea because it means we don't need any external tools to create the database. This PR implements this approach. A few caveats:

• For some reason the shlib command does not add include flags for LinkingTo dependencies, so I had to inject those manually.

• In addition to --dry-run we need to pass --always-make to make. See sneak '--always-make' into R CMD SHLIB invocation rstudio/rstudio#11917 for the motivation and the workaround reused here.

• We need to pass target source files to shlib. These can easily be inferred when Makevars does not override OBJECTS. When it is overridden, we need to retrieve the contents of this variable, and then match the object files to potential sources. This sometimes clashes, e.g. in igraph there are a .c and .f files that match a single object file. We just select one file in this case and ignore the others. This should not be a big deal and should be a rare occurrence.

• There could be make directives interspersed between compile commands in the dry-run output (this happens with rlang). To detect actual commands, we simply select all lines starting with a compiler command. These commands (CC, CXX, FC, etc) could be retrieved from R CMD config but this tool seems extremely slow. Instead we retrieve them from the makefiles manually. These commands should be in the same order as the source files passed to shlib. A consistency check verifies that we have as many commands as source files and that each command indeed references the corresponding source file (typically in a -f flag).

The compilation database is not generated by default, the user must explicitly opt in by inserting this line in their DESCRIPTION file:

Config/devtools/compilation-database: true

In addition the compile_commands.json file generated at the root should be gitignored and Rbuildignored. We'll provide a usethis function to accomplish these tasks.

The tools for generating this database should probably be moved and reexported from pkgbuild in the future (cc @gaborcsardi), but I'm implementing them here as I'd like to include the feature in the next release of pkgload.

Untested on Windows but should work on Unixes. Tested on macOS with readxl, rlang, igraph, and r-tree-sitter.
Edit: now working on Windows

[DavisVaughan]

Works on macOS for many packages for me, works on Windows for vctrs with the newest changes!

[lionel-]

I've added a currently unexported util to help set this up: pkgload:::use_compilation_db()