Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor modules support #4673

Merged
merged 149 commits into from
Feb 18, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
149 commits
Select commit Hold shift + click to select a range
e20c6ca
improve module tests
Arthapz Jan 23, 2024
d52291f
refactor module common infrastructure
Arthapz Jan 23, 2024
2148980
refactor clang module infrastructure
Arthapz Jan 23, 2024
da5f589
refactor gcc module infrastructure
Arthapz Jan 23, 2024
82714f7
refactor msvc module infrastructure
Arthapz Jan 23, 2024
7dddd80
optimise clang fallback dependency scanner
Arthapz Jan 23, 2024
a6f1cc3
add a policy to toggle C++23 std module
Arthapz Jan 23, 2024
d914662
fix headerunit compilation on clang
Arthapz Jan 23, 2024
35f41b6
fix gcc module support
Arthapz Jan 23, 2024
a165624
use execv instead of iorunv
Arthapz Jan 23, 2024
713e8a5
Revert "use execv instead of iorunv"
Arthapz Jan 23, 2024
931e8c6
reenable build across targets in parallel policy
Arthapz Jan 25, 2024
635f106
Revert "reenable build across targets in parallel policy"
Arthapz Jan 26, 2024
300600d
take advantage of XMake runtimes support
Arthapz Jan 26, 2024
cb8b45b
update tests to use XMake runtimes support
Arthapz Jan 26, 2024
d50a83f
apply PR suggestions
Arthapz Jan 29, 2024
952b813
populate module map before generating batchjobs
Arthapz Jan 29, 2024
e8c63dc
fix space indent of build.c++.modules.std policy
Arthapz Jan 30, 2024
0ff527a
remove removed build.c++.clang.stdmodules policy from tests
Arthapz Jan 30, 2024
a9f3040
use -fkeep-system-includes only if supported by underlying clang
Arthapz Jan 30, 2024
7e1cf4f
remove now useless before_link on C++ modules rule
Arthapz Jan 30, 2024
a2ccc58
use try{} instead of assert to preserve iorunv error colors
Arthapz Jan 30, 2024
8e10652
revert last commit
Arthapz Jan 30, 2024
98646d9
update comment
Arthapz Jan 31, 2024
cc88697
remove dead code
Arthapz Jan 31, 2024
95996d7
improve _should_build
Arthapz Jan 31, 2024
f3ec9cc
implement missing headerunit aliasing
Arthapz Jan 31, 2024
89f151a
add a test for aliased headerunits
Arthapz Jan 31, 2024
0b37f42
use os.vrunv for dependency scanning
Arthapz Jan 31, 2024
a5fb58f
improve module cmdline print
Arthapz Jan 31, 2024
2465d8a
use table.orderpair to ensure dependency order stay the same
Arthapz Jan 31, 2024
b9b078d
code cleanup
Arthapz Jan 31, 2024
e339cb5
fix inconsistent print format for gcc module mapper
Arthapz Jan 31, 2024
15f2678
use table.orderpair to ensure dependency order stay the same
Arthapz Jan 31, 2024
77d7dc5
improve std c++ lib handling for clang
Arthapz Jan 31, 2024
47afe0c
cleanup
Arthapz Jan 31, 2024
adee617
fix progress for clang
Arthapz Jan 31, 2024
9cb5dba
fix -fkeep-system-includes detection
Arthapz Jan 31, 2024
0bb30a6
fix potentially nil runtime_flag for clang toolchain include detection
Arthapz Jan 31, 2024
e219f8a
cleanup
Arthapz Jan 31, 2024
f17ccef
use ipairs + orderpkgs
Arthapz Jan 31, 2024
9b378dd
cleanup
Arthapz Jan 31, 2024
7dbdcf9
remove useless path.translate
Arthapz Jan 31, 2024
69e66b7
cleanup
Arthapz Jan 31, 2024
7d5573a
optimise header duplication detection
Arthapz Jan 31, 2024
c7a2537
fix assert in gcc module dependency mapper
Arthapz Feb 1, 2024
b1005e8
use compinst:compile when possible
Arthapz Feb 1, 2024
b81eed5
Merge pull request #4321 from Arthapz/improve-modules-support
waruqi Feb 2, 2024
b64836f
improve module circular dependency detection
Arthapz Feb 2, 2024
b4d3167
parallelise dependency scanning
Arthapz Feb 2, 2024
99197a6
support libc++ std module
Arthapz Feb 2, 2024
c5e4d30
use compinst for msvc when possible
Arthapz Feb 2, 2024
bb4f92b
Merge pull request #4676 from Arthapz/compinst-msvc
waruqi Feb 2, 2024
6d4f138
use find_file instead of os.files
Arthapz Feb 2, 2024
5c8d2cb
remove unnecessary path.normalize
Arthapz Feb 2, 2024
55d933b
cleanup
Arthapz Feb 2, 2024
6f7cc07
add some comments
Arthapz Feb 2, 2024
a204e49
format some codes
waruqi Feb 2, 2024
0e13da2
rename apis
waruqi Feb 2, 2024
5158579
format more codes
waruqi Feb 2, 2024
e449836
improve _is_duplicated_headerunit
waruqi Feb 2, 2024
c630a34
Merge pull request #4678 from Arthapz/parallelise-dependency-scanning
waruqi Feb 2, 2024
48be7cd
Merge pull request #4679 from Arthapz/support-clang-stdmodule
waruqi Feb 2, 2024
22d46b0
add a test for circular dependency detection
Arthapz Feb 2, 2024
aedc8fa
use core.base.graph instead of rolling out a custom DAG
Arthapz Feb 2, 2024
05deea2
Merge branch 'modules' into optimise-dag
Arthapz Feb 2, 2024
52d8594
cleanup
Arthapz Feb 2, 2024
6061ff6
Merge branch 'optimise-dag' of github.com:Arthapz/xmake into optimise…
Arthapz Feb 2, 2024
9ccf155
improve circular_dependency test
Arthapz Feb 2, 2024
07c0af9
use hashset to improve speed
Arthapz Feb 3, 2024
b40cc42
improve tests
Arthapz Feb 3, 2024
3bdb07d
use raise instead of os.raise
Arthapz Feb 3, 2024
2d59e22
fix tests
Arthapz Feb 3, 2024
c5a17b3
fix dependency_scanner hashset values
Arthapz Feb 3, 2024
a393c0a
fix recompilation of .cpp files
Arthapz Feb 3, 2024
fa0d883
fix recompilation for gcc
Arthapz Feb 3, 2024
ce79b0d
fix batchcmds
Arthapz Feb 3, 2024
ca775e5
fix unnecessary recompilation of modules
Arthapz Feb 3, 2024
794f6db
Update test_base.lua
waruqi Feb 4, 2024
0d72b2e
Update test.lua
waruqi Feb 4, 2024
7f4af3b
Update dependency_scanner.lua
waruqi Feb 4, 2024
650d8f2
Merge pull request #4677 from Arthapz/optimise-dag
waruqi Feb 4, 2024
e5f820e
make should_build and mark_build public
Arthapz Feb 4, 2024
2728a2c
fix gcc module compilation
Arthapz Feb 4, 2024
b9f8cab
fix finding of libc++.modules.json when it is not directly in lib folder
Arthapz Feb 4, 2024
ffe0768
fix missing build variable declaration
Arthapz Feb 4, 2024
d375ab0
Merge pull request #4687 from Arthapz/fix-.cpp-objectfiles-update
waruqi Feb 5, 2024
05b80a3
Merge pull request #4695 from Arthapz/fix-libc++.modules.json-finding
waruqi Feb 5, 2024
4a6acaf
fix objectfile handling
Arthapz Feb 4, 2024
2328ca4
fix missing objectfiles for private modules
Arthapz Feb 4, 2024
538cf30
fix merge
Arthapz Feb 5, 2024
4d3ad3f
fix format
Arthapz Feb 5, 2024
750aa4c
fix gcc
Arthapz Feb 5, 2024
0b8ea93
cull unreferenced modules and fix gcc module mapper
Arthapz Feb 5, 2024
cf0e793
fix tests
Arthapz Feb 5, 2024
6739478
fix gcc
Arthapz Feb 5, 2024
3a7d68d
update relevent tests to use moduleonly target rule
Arthapz Feb 5, 2024
3280d7f
add support of moduleonly targets with c++.moduleonly rule
Arthapz Feb 5, 2024
e8b4daf
fix packages test
Arthapz Feb 5, 2024
effa81f
remove old file
Arthapz Feb 5, 2024
359ad44
fix msvc
Arthapz Feb 5, 2024
cf5c45b
cleanup
Arthapz Feb 5, 2024
3266f5d
fixe module installation
Arthapz Feb 5, 2024
c11559e
fix stdmodule find_file and update test (std module doesn't export
Arthapz Feb 6, 2024
0c8646d
Merge pull request #4714 from Arthapz/fix-stdmodule-clang
waruqi Feb 6, 2024
8929c38
implement target kind moduleonly
Arthapz Feb 6, 2024
eb1fd6f
Update xmake.lua
waruqi Feb 7, 2024
50ecef5
Update bar.mpp
waruqi Feb 7, 2024
660d40f
Update xmake.lua
waruqi Feb 7, 2024
ec17990
Update xmake.lua
waruqi Feb 7, 2024
c732c99
Update xmake.lua
waruqi Feb 7, 2024
384256c
improve moduleonly.lua
Arthapz Feb 7, 2024
d98db4c
use table.orderpairs for generate_metadata
Arthapz Feb 7, 2024
618f751
use target:is_binary instead of target:kind() == "binary"
Arthapz Feb 7, 2024
dee7608
fix test
Arthapz Feb 7, 2024
348a74a
update vsxmake to handle moduleonly target kind
Arthapz Feb 7, 2024
56ba14a
fix msvc batchcmds
Arthapz Feb 7, 2024
6f520cf
improve moduleonly vsxmake
Arthapz Feb 7, 2024
6e3d6c4
update vs project generator to support moduleonly target kind
Arthapz Feb 7, 2024
5d08871
cleanup
Arthapz Feb 7, 2024
e721aae
implement basic cmake support for moduleonly
Arthapz Feb 7, 2024
955f2bb
support ProjectReferences for VS project generator
Arthapz Feb 8, 2024
b23959e
fix msvc batchcmds
Arthapz Feb 8, 2024
9354550
fix module compilation for CMake
Arthapz Feb 8, 2024
14a186c
fix CMake moduleonly support
Arthapz Feb 8, 2024
7b6d069
cleanup
Arthapz Feb 8, 2024
b0819c9
support uninstalling modules
Arthapz Feb 8, 2024
3af5f89
add support of xmake package for modules
Arthapz Feb 8, 2024
0e9efa8
fix find_file for libc++.modules.json when in a subdir
Arthapz Feb 8, 2024
37c10d6
fix libc++.modules.json finding
Arthapz Feb 9, 2024
7267466
Update compiler_support.lua
waruqi Feb 9, 2024
e6fd7ca
Merge pull request #4721 from Arthapz/fix-stdmodule-clang
waruqi Feb 9, 2024
6015b11
test modules incremental compilation
waruqi Feb 10, 2024
c68dad9
Merge pull request #4727 from xmake-io/inctest
waruqi Feb 11, 2024
0550446
Update cmakelists.lua
waruqi Feb 11, 2024
b6983ac
Update main.lua
waruqi Feb 11, 2024
a739684
Merge pull request #4707 from Arthapz/support-moduleonly-libraries
waruqi Feb 11, 2024
1a6c4c2
implement module reusage based on flag comparison
Arthapz Feb 10, 2024
eca1b25
fix incremental compilation for msvc
Arthapz Feb 12, 2024
04cee16
try to fix incremental build
Arthapz Feb 12, 2024
be329f0
cleanup
Arthapz Feb 14, 2024
45b8fa3
improve incremental compilation fix
Arthapz Feb 14, 2024
b1d3ab4
Merge pull request #4728 from Arthapz/reuse-modules
waruqi Feb 14, 2024
4cb6ecc
add more tests
waruqi Feb 15, 2024
0e54526
rename mapper path varname
waruqi Feb 18, 2024
b92d9b0
format code
waruqi Feb 18, 2024
c3635e4
fix incremental compilation for tests/ modules/package
waruqi Feb 18, 2024
af3461d
fix incremental compilation for tests/ modules/package
waruqi Feb 18, 2024
bc1a30d
improve should_build for modules
waruqi Feb 18, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions tests/projects/c++/modules/aliased_headerunit/src/foo/hello.mpp
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
module;
#include <cstdio>

export module hello;

import "../header.hpp";

export namespace hello {
void say(const char *arg) {
printf("%s: %s\n", FOO, arg);
}
}
5 changes: 5 additions & 0 deletions tests/projects/c++/modules/aliased_headerunit/src/header.hpp
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
#pragma once

namespace hello {
inline constexpr auto FOO = "Hello";
}
7 changes: 7 additions & 0 deletions tests/projects/c++/modules/aliased_headerunit/src/main.cpp
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
import hello;
import "header.hpp";

int main() {
hello::say(hello::FOO);
return 0;
}
1 change: 1 addition & 0 deletions tests/projects/c++/modules/aliased_headerunit/test.lua
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
inherit(".test_headerunits")
8 changes: 8 additions & 0 deletions tests/projects/c++/modules/aliased_headerunit/xmake.lua
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
add_rules("mode.release", "mode.debug")
set_languages("c++20")

-- header.hpp should be built only one time
target("aliased_headerunit")
set_kind("binary")
add_headerfiles("src/*.hpp")
add_files("src/*.cpp", "src/foo/*.mpp")
13 changes: 13 additions & 0 deletions tests/projects/c++/modules/circular_dependency/src/hello.mpp
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
export module hello;

import hello3;

export namespace hello {
class say {
public:
say(int data);
void hello();
private:
int data_;
};
}
3 changes: 3 additions & 0 deletions tests/projects/c++/modules/circular_dependency/src/hello2.mpp
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
export module hello2;

import hello;
3 changes: 3 additions & 0 deletions tests/projects/c++/modules/circular_dependency/src/hello3.mpp
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
export module hello3;

import hello2;
7 changes: 7 additions & 0 deletions tests/projects/c++/modules/circular_dependency/src/main.cpp
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
import hello;

int main() {
hello::say s(sizeof(hello::say));
s.hello();
return 0;
}
7 changes: 7 additions & 0 deletions tests/projects/c++/modules/circular_dependency/test.lua
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
import(".test_base", {alias = "test_build"})

function main(t)
if test_build.can_build() then
t:will_raise(test_build, "circular modules dependency detected")
end
end
8 changes: 8 additions & 0 deletions tests/projects/c++/modules/circular_dependency/xmake.lua
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
add_rules("mode.release", "mode.debug")
set_languages("c++20")

target("circular_dependency")
set_kind("binary")
add_files("src/*.cpp", "src/*.mpp")


3 changes: 2 additions & 1 deletion tests/projects/c++/modules/cpp_with_moduledeps/xmake.lua
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,8 @@ set_languages("c++20")

target("mod")
set_kind("static")
add_files("src/mod.mpp", "src/mod.cpp")
add_files("src/mod.mpp", {public = true})
add_files("src/mod.cpp")

target("cpp_with_moduledeps")
set_kind("binary")
Expand Down
6 changes: 3 additions & 3 deletions tests/projects/c++/modules/inline_and_template/src/main.cpp
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@
#include <iostream>

import hello;
import say;
import foo;

#include <iostream>

int main() {
hello::say_hello();
say{}.hello<sizeof(say)>();
std::cout << foo<int>{}.hello() << std::endl;
return 0;
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -7,5 +7,5 @@ module hello:part_internal;
namespace hello {
void say_internal(const char *str) {
printf("%s\n", str);
}
}
}
}
6 changes: 2 additions & 4 deletions tests/projects/c++/modules/link_order/xmake.lua
Original file line number Diff line number Diff line change
Expand Up @@ -2,13 +2,11 @@ add_rules("mode.release", "mode.debug")
set_languages("c++20")

target("foo")
add_rules("c++")
set_kind("static")
set_kind("moduleonly")
add_files("src/foo.mpp")

target("bar")
add_rules("c++")
set_kind("static")
set_kind("moduleonly")
add_files("src/bar.mpp")

target("link_order_1")
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,5 +2,5 @@ add_rules("mode.release", "mode.debug")
set_languages("c++20")

target("mod")
set_kind("static")
set_kind("moduleonly")
add_files("src/mod.mpp")
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
module bar;
#include <a.hpp>

const char *bar() {
const char *hello() {
return "Hello world";
}
}
Original file line number Diff line number Diff line change
@@ -1,3 +1,12 @@
module;

#include <a.hpp>

export module bar;

export const char *bar();
export namespace bar {
const char *hello() {
return ::hello();
}
}

Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
#ifndef A_HPP
#define A_HPP

[[gnu::visibility("default")]] const char *hello();

#endif
Original file line number Diff line number Diff line change
Expand Up @@ -3,5 +3,7 @@ set_languages("c++20")

target("bar")
set_kind("static")
add_headerfiles("include/(**.hpp)")
add_includedirs("include")
add_files("*.cpp")
add_files("*.mpp", { install = true })
add_files("*.mpp", { public = true })
Original file line number Diff line number Diff line change
Expand Up @@ -3,4 +3,4 @@ package("bar")

on_install(function(package)
import("package.tools.xmake").install(package, {})
end)
end)
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
export module bar2;

export namespace bar {
const char *hello2() {
return "Hello world";
}
}

Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
add_rules("mode.release", "mode.debug")
set_languages("c++20")

target("bar2")
set_kind("moduleonly")
add_files("*.mpp")
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
package("bar2")
set_kind("library")
set_sourcedir(path.join(os.scriptdir(), "src"))

on_install(function(package)
import("package.tools.xmake").install(package, {})
end)
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
export module foo;

export namespace foo {
#ifdef FOO_EXPORT
void say(const char *);
}
#endif
}
Original file line number Diff line number Diff line change
Expand Up @@ -4,4 +4,4 @@ set_languages("c++20")
target("foo")
set_kind("static")
add_files("*.cpp")
add_files("*.mpp", { install = true })
add_files("*.mpp", {defines = "FOO_EXPORT", public = true})
4 changes: 3 additions & 1 deletion tests/projects/c++/modules/packages/src/main.cpp
Original file line number Diff line number Diff line change
@@ -1,7 +1,9 @@
import foo;
import bar;
import bar2;

int main() {
foo::say(bar());
foo::say(bar::hello());
foo::say(bar::hello2());
return 0;
}
6 changes: 3 additions & 3 deletions tests/projects/c++/modules/packages/xmake.lua
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
add_rules("mode.release", "mode.debug")
set_languages("c++20")
set_languages("c++2b")

add_repositories("my-repo my-repo")
add_requires("foo", "bar")
add_requires("foo", "bar", "bar2")

target("packages")
set_kind("binary")
add_files("src/*.cpp")
add_packages("foo", "bar")
add_packages("foo", "bar", "bar2")
set_policy("build.c++.modules", true)
8 changes: 8 additions & 0 deletions tests/projects/c++/modules/private_module/src/use.cpp
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
module use;

import dep1;
import dep2;

int lib() {
return m() + i();
}
7 changes: 2 additions & 5 deletions tests/projects/c++/modules/private_module/src/use.mpp
Original file line number Diff line number Diff line change
@@ -1,6 +1,3 @@
import dep1;
import dep2;
export module use;

int lib() {
return m() + i();
}
export int lib();
1 change: 1 addition & 0 deletions tests/projects/c++/modules/private_module/xmake.lua
Original file line number Diff line number Diff line change
Expand Up @@ -5,3 +5,4 @@ target("private_module")
add_rules("c++")
set_kind("$(kind)")
add_files("src/*.mpp")
add_files("src/*.cpp")
3 changes: 2 additions & 1 deletion tests/projects/c++/modules/staticlib/xmake.lua
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,8 @@ set_languages("c++20")

target("mod")
set_kind("static")
add_files("src/mod.mpp", "src/mod.cpp")
add_files("src/mod.mpp", {public = true})
add_files("src/mod.cpp")

target("hello")
set_kind("binary")
Expand Down
2 changes: 1 addition & 1 deletion tests/projects/c++/modules/stdmodules/src/my_module.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -2,4 +2,4 @@ module my_module;

import std;

auto my_sum(size_t a, size_t b) -> size_t { return a + b; }
auto my_sum(std::size_t a, std::size_t b) -> std::size_t { return a + b; }
2 changes: 1 addition & 1 deletion tests/projects/c++/modules/stdmodules/src/my_module.mpp
Original file line number Diff line number Diff line change
Expand Up @@ -2,4 +2,4 @@ export module my_module;

import std;

export auto my_sum(size_t a, size_t b) -> size_t;
export auto my_sum(std::size_t a, std::size_t b) -> std::size_t;
5 changes: 2 additions & 3 deletions tests/projects/c++/modules/stdmodules/xmake.lua
Original file line number Diff line number Diff line change
Expand Up @@ -3,11 +3,10 @@ set_languages("c++latest")

target("mod")
set_kind("static")
add_files("src/*.cpp", "src/*.mpp")
set_policy("build.c++.clang.stdmodules", true)
add_files("src/*.cpp")
add_files("src/*.mpp", {public = true})

target("stdmodules")
set_kind("binary")
add_files("test/*.cpp")
add_deps("mod")
set_policy("build.c++.clang.stdmodules", true)
1 change: 0 additions & 1 deletion tests/projects/c++/modules/stdmodules_cpp_only/xmake.lua
Original file line number Diff line number Diff line change
Expand Up @@ -6,5 +6,4 @@ target("stdmodules_cpp_only")
set_kind("binary")
add_files("src/*.cpp")
set_policy("build.c++.modules", true)
set_policy("build.c++.clang.stdmodules", true)

Original file line number Diff line number Diff line change
Expand Up @@ -5,10 +5,6 @@ target("mod")
set_kind("static")
add_files("src/*.cpp", "src/*.mpp")

set_policy("build.c++.clang.stdmodules", true)

target("mod2")
set_kind("static")
add_files("src/*.cpp", "src/*.mpp")

set_policy("build.c++.clang.stdmodules", true)
Loading
Loading