Compilation database support - clang, gcc, mingw

[RevoluPowered]

This file is from mongodb and the tooling Automatically generates the compilation database across all platforms.

Thanks to acmarrow for a simple solution

SCons/scons#3355

For the uninitiated this supports IDE symbol lookup, refactoring tools, documentation tools:
This means we can support intelisense style technologies with reliable type lookup.

For example:

• CLion symbols (https://www.jetbrains.com/help/clion/compilation-database.html)
• VSCode symbols / refactoring (https://devblogs.microsoft.com/cppblog/visual-studio-code-cc-extension-october-2017-update/)
• QTcreator (https://doc.qt.io/qtcreator/creator-clang-codemodel.html#using-compilation-databases)
• atom (https://atom.io/packages/linter-clang)
• neovim
• vim https://github.com/ycm-core/YouCompleteMe
• anything which supports compilation db
• etc

Known issue:

• no support for msvc compiler

Co-authored-by: K. S. Ernest (iFire) Lee [email protected]

[RevoluPowered]

Python 2.7 justification for upgrade:

• Python 2.7 is going to be depreciated very soon in scons. (next release)

Scons would not build without this update with the compilation db on windows, due to a bug they have not fixed.

Simplifies osx path code.

Updated platform install to use sudo to install scons to prevent us having to modify path.

Since the VM image is thrown away this doesn't matter.

[IAmActuallyCthulhu]

I have tested it on macOS (10.14.6) with CLion (2019.2.4).

While testing I found one issue, Objective-C files found in /platform/osx are not recorded to the compilation database.

I was able to fix it (quick and dirty) by adding , '.mm' to the _CXXSuffixes array.

https://github.com/godotengine/godot/blob/fb8bb2dcd4db1ece6bb25aabd52837b2f363b984/site_scons/site_tools/compilation_db.py#L39

---

P.S. Thank you @RevoluPowered and @fire for making this PR. I am incredibly grateful, happy, and overall ecstatic to have this.

[IAmActuallyCthulhu]

I have tested it on macOS (10.14.6) with VSCode.

VSCode detected the compile_commands.json and generated the config c_cpp_properties.json. There are no false positives anymore!

However, same Objective-C issues as explained in #32848 (comment).

Side Note: I do not know if the generated c_cpp_properties.json file is correct or needs manual changes.

[RevoluPowered]

Hello, I will update the PR tomorrow as we are all returning from GodotCon

Thanks everyone for testing I will merge the objective c support too :)

[RevoluPowered]

not got access to apple hw, will delay this until tomorrow

[RevoluPowered]

@IAmActuallyCthulhu updated the suffixes properly so should be platform agnostic now

Adding the method to the cxxsuffix as requested would have broken it on windows, so I have made it retrieve this directly from scons, on osx it will make it available and linux, but on windows it will not as this made appveyor break.

[Xrayez]

Doesn't seem to work on Windows, VS2017, Python 3.6.4, Scons v3.0.5.

[ 96%] Building compilation database compile_commands.json
...
[100%] scons: done building targets.

It simply produces an empty array in compile_commands.json.

scons platform=windows tools=yes target=release_debug bits=64 -j8

Same thing with target=debug debug_symbols=yes.

[RevoluPowered]

Doesn't seem to work on Windows, VS2017, Python 3.6.4, Scons v3.0.5.

[ 96%] Building compilation database compile_commands.json
...
[100%] scons: done building targets.

It simply produces an empty array in compile_commands.json.

scons platform=windows tools=yes target=release_debug bits=64 -j8

Same thing with target=debug debug_symbols=yes.

This will only work with scons 3.1.1 the stable release

[RevoluPowered]

vscode currently requires us to reload this compilation_db file manually, I think I can update the behaviour so that vscode just sees this as a file modification rather than the file being unreadable during write.

temp solution is that you can just do Ctrl + Shift + P then run 'Reload window' after a build.

This happens instantly on my computer so it's not a huge priority IMO.

CLion has no issues with this which I can see.

[Thorhian]

Hi RevoluPowered! I also wanted to suggest adding "compile_commands.json" to the .gitignore file for the project as well. That wouldn't cause any unwanted side effects, would it?

[RevoluPowered]

Hi RevoluPowered! I also wanted to suggest adding "compile_commands.json" to the .gitignore file for the project as well. That wouldn't cause any unwanted side effects, would it?

Good suggestion I will test this and see what happens

[clayjohn]

I have the same issue as Xrayez. I am using Windows 10 and I have scons version 3.1.1 installed. The compilation database just contains an empty array.

[clayjohn]

Thanks for all your work :)

I plan on running MinGW periodically to update my compilation database in the meantime.

[RevoluPowered]

Thanks for all your work :)

I plan on running MinGW periodically to update my compilation database in the meantime.

Ach, can only take a minor part of credit, was team effort between @fire and I :)

also, acmarrow from mongodb :) and mongodb themselves

[akien-mga]

Force pushed an amend that fixes Python style issues (we now use black for formatting), and moves the script to misc/scons/site_tools as I found a workaround to override the default site_scons path. (Basically reimplementing what SCons' --site-dir command line option would do by using a private method - Thanks Python for not having real private methods :P).

[akien-mga]

Thanks!

[RevoluPowered]

Thanks for your help merging I had little time to pull away from FBX. That's great I am so happy, I use this regularly! <3

[Faless]

Getting the following error after this merge:

[ 98%] scons: done building targets.
TypeError: changed_since_last_build_node() missing 1 required positional argument: 'node':
  File "/usr/lib/scons/SCons/Script/Main.py", line 1376:
    _exec_main(parser, values)
  File "/usr/lib/scons/SCons/Script/Main.py", line 1339:
    _main(parser)
  File "/usr/lib/scons/SCons/Script/Main.py", line 1103:
    nodes = _build_targets(fs, options, targets, target_top)
  File "/usr/lib/scons/SCons/Script/Main.py", line 1313:
    jobs.run(postfunc = jobs_postfunc)
  File "/usr/lib/scons/SCons/Job.py", line 111:
    self.job.start()
  File "/usr/lib/scons/SCons/Job.py", line 423:
    task.executed()
  File "/usr/lib/scons/SCons/Script/Main.py", line 237:
    SCons.Taskmaster.OutOfDateTask.executed(self)
  File "/usr/lib/scons/SCons/Taskmaster.py", line 317:
    t.release_target_info()
  File "/usr/lib/scons/SCons/Node/FS.py", line 2980:
    self.changed(allowcache=True)
  File "/usr/lib/scons/SCons/Node/FS.py", line 3223:
    has_changed = SCons.Node.Node.changed(self, node)
  File "/usr/lib/scons/SCons/Node/__init__.py", line 1455:
    if _decider_map[child.changed_since_last_build](child, self, prev_ni):

Tested with both pip version:

SCons by Steven Knight et al.:
	script: v3.1.2.bee7caf9defd6e108fc2998a2520ddb36a967691, 2019-12-17 02:07:09, by bdeegan on octodog
	engine: v3.1.2.bee7caf9defd6e108fc2998a2520ddb36a967691, 2019-12-17 02:07:09, by bdeegan on octodog

and distro version:

SCons by Steven Knight et al.:
	script: v3.0.1.74b2c53bc42290e911b334a6b44f187da698a668, 2017/11/14 13:16:53, by bdbaddog on hpmicrodog
	engine: v3.0.1.74b2c53bc42290e911b334a6b44f187da698a668, 2017/11/14 13:16:53, by bdbaddog on hpmicrodog

[akien-mga]

@Faless The backtrace is from your distro version (/usr/lib/scons), so make sure that the pip version is not using the wrong SCons lib from the distro.

[akien-mga]

We should likely increase EnsureSConsVersion to the version needed for the compile DB, or alternatively disable generation of the compile DB on older SCons versions (might be more user-friendly).

[Faless]

@akien-mga right, my bash alias tricked me -.- . It works with the pip version, so yeah, scons 3.0.1 doesn't seem to work, scons 3.1.2 is fine.

[akien-mga]

Cherry-picked for 3.2.2. (Together with #38830 to fix build on older SCons versions.)