Compilation database

Guillaume Papin(@Sarcasm) has a thorough article about compilation databases.

compile_commands.json

compile_commands.json should reside in the project root because Emacs/VSCode plugins use it to indicate the project root, which will be used to initialize InitializeParams.rootUri.

Clang

Clang’s -MJ option generates a compilation database entry per input (requires Clang >= 5.0).

Usage:

clang++ -MJ a.o.json -Wall -std=c++11 -o a.o -c a.cpp
clang++ -MJ b.o.json -Wall -std=c++11 -o b.o -c b.cpp

To merge the compilation database entries into a valid compilation database, it is possible to use sed:

sed -e '1s/^/[\n/' -e '$s/,$/\n]/' *.o.json > compile_commands.json

This sed invocation does the following:

• insert the opening bracket: [
• concatenate the entries
• remove the trailing comma of the last entry (to be JSON compliant)
• insert the closing bracket: ]

Note: This will not work when merging multiple compile_commands.json together (which already start and end with []). For such case, a possible workaround is:

cat ./**/compile_commands.json > compile_commands.json && sed -i -e ':a;N;$!ba;s/\]\n\n\[/,/g' compile_commands.json

CMake

mkdir build
(cd build; cmake -DCMAKE_EXPORT_COMPILE_COMMANDS=YES ..)
ln -s build/compile_commands.json

[Windows] MSBuild does not support -DCMAKE_EXPORT_COMPILE_COMMANDS=ON. Instead, Ninja should be used to generate compile_commands.json #663

Build EAR

Bear is a tool that generates a compilation database for clang tooling. It can be used for any project based on Makefile.

bear make
# generates compile_commands.json

scan-build

scan-build is a python package that can generate a compilation database for clang tooling (uses Bear as a backend). This too can be used for any project based on a Makefile.

intercept-build make all # generates compile_commands.json from the `make all` ruleset

compiledb-generator

Tool for generating Clang's JSON Compilation Database files for make-based build systems. Implemented as a Python package + CLI Tool, focused on make-based build systems and designed to be fast (doesn't need clean builds) and cross-compiling friendly, also supporting incremental update of the compile_commands.json file.

compiledb -n make # extract compile commands without executing the actual build

make -w | tee build_log.txt 
compiledb < build_log.txt # extract compile commands from the build log previously generated

Ninja

# Format: ninja -t compdb rule_names... > compile_commands.json
ninja -C out/Release -t compdb cxx cc > compile_commands.json

Waf

Load the clang_compilation_database tool in your wscript:

def configure(conf):
    conf.load('clang_compilation_database')

./waf configure build
ln -s build/compile_commands.json

buck

buck build :helloworld#compilation-database
ln -s $(buck targets --show-output :helloworld#compilation-database | cut -d ' ' -f 2)

stdout of an external command

You may use the initialization option "compilationDatabaseCommand" to provide the JSON compilation database. cquery will read its stdout rather than read compile_commands.json. This may be useful when cquery cannot parse the compile_commands.json correctly (e.g. MSVC cl.exe, Intel C++ Compiler options)

# extra command line option
'--init={"compilationDatabaseCommand":"mycompdb"}'

(setq cquery-extra-init-params '(:compilationDatabaseCommand "mycompdb"))

Suppose the project is at /tmp/c, mycompdb /tmp/c will be executed with stdin=initializationOptions and the stdout should be a JSON compilation database.

You may use this shell script as a starting point:

#!/bin/zsh
# mycompdb /tmp/c
cat >> /tmp/initialization-options
cat <<e
[ { "arguments": [ "c++", "a.o", "a.cc" ],
    "directory": "/tmp/c", "file": "a.cc" } ]
e

An example to scrub Intel C++ Compiler options:

#!/usr/bin/env python3
import json
import os
import sys
with open(os.path.join(sys.argv[1], 'compile_commands.json')) as f:
    db = json.load(f)
    for entry in db:
        args = entry['arguments']
        try:
            # Intel C++ Compiler option that is unknown to clang
            args.remove('-xHOST')
        except ValueError:
            pass
    json.dump(db, sys.stdout)

Examples

Linux kernel

wget 'https://git.archlinux.org/svntogit/packages.git/plain/trunk/config?h=packages/linux' -O .config
yes '' | make config
bear make -j bzImage modules

Misc