-
Notifications
You must be signed in to change notification settings - Fork 4.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add exec_group_compatible_with
attribute
#24964
Add exec_group_compatible_with
attribute
#24964
Conversation
3df50ad
to
b54072a
Compare
on this reproducer: #23202 (comment) w/ this change (and the exec_properties one) I get this crash:
i looked into this a bit last week and I believe it's because this function
|
8528676
to
c943de3
Compare
verified on the latest commits here it works with this repro case now! |
Currently stacked on #24979 |
for (Map.Entry<?, ?> e : map.entrySet()) { | ||
dict.put(e.getKey(), Starlark.fromJava(e.getValue(), null)); | ||
} | ||
map.forEach((key, value) -> dict.put(key, Starlark.fromJava(value, null))); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is a drive-by improvement to avoid creating and retaining an entry set.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I've reviewed the documentation changes (great!) and the non-test code changes. I'd like to ask you to split those further, to make import and history easier to read, but overall everything looks good.
I'll review the test changes once those are split out.
src/main/java/com/google/devtools/build/lib/skyframe/DependencyResolver.java
Show resolved
Hide resolved
This makes `exec_properties` such as `no-remote-exec` effective for the main test spawn, where it previously only affected post-processing spawns such as test XML generation.
5b96980
to
b206ce2
Compare
b206ce2
to
91a43af
Compare
I split up all changes above #24979 into five commits that can be reviewed and merged independently (in order). Since stacked CLs are much easier than stacking PRs on GitHub, could you convert the commits into separate CLs during import? If not, let me know and I can send five separate stacked PRs. |
Sounds good, I'll try to import separately. I agree that Github's workflow for stacked PRs is terrible. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
One small nit, let me know if you want to fix it or if I should handle on import.
src/main/java/com/google/devtools/build/lib/analysis/starlark/StarlarkAttributesCollection.java
Outdated
Show resolved
Hide resolved
This change implements the [Per-target execution platform constraints for exec groups](https://docs.google.com/document/d/1u4zP5MLU3HOt-qlyiNnlxcu4SqOB5tnm32v02vLnd5U/edit) proposal by adding the `exec_group_compatible_with` attribute. RELNOTES: The new exec_group_compatible_with attribute on all rules accepts a dictionary mapping exec group names to lists of additional constraints to request from the exec group's execution platform.
RELNOTES[INC]: Constraints and toolchain requirements added to the default exec group, for example via the toolchains parameter of the rule function or the exec_compatible_with attribute on all rules, no longer apply to the test exec group, which contains the test action on test rules. Instead, use exec_group_compatible_with to apply constraints and/or define an explicit test exec group with toolchain requirements on test rules that require it.
In addition to documenting the new attribute, existing documented best practices around exec constraints also need to change.
91a43af
to
5527039
Compare
Once tests pass I'll start importing these: @gregestren and @fweikert, if you have any comments please add them. |
Part of #24964. PiperOrigin-RevId: 721429916 Change-Id: Ia473061bf78f283a35eb575097d7f6af5edf81f7
This change implements the [Per-target execution platform constraints for exec groups](https://docs.google.com/document/d/1u4zP5MLU3HOt-qlyiNnlxcu4SqOB5tnm32v02vLnd5U/edit) proposal by adding the `exec_group_compatible_with` attribute. Part of #24964. RELNOTES: The new exec_group_compatible_with attribute on all rules accepts a dictionary mapping exec group names to lists of additional constraints to request from the exec group's execution platform. PiperOrigin-RevId: 721529975 Change-Id: Ibcd6e9772684272d48a39907eaa9f7503fbaaa6c
Part of #24964. RELNOTES[INC]: Constraints and toolchain requirements added to the default exec group, for example via the toolchains parameter of the rule function or the exec_compatible_with attribute on all rules, no longer apply to the test exec group, which contains the test action on test rules. Instead, use exec_group_compatible_with to apply constraints and/or define an explicit test exec group with toolchain requirements on test rules that require it. PiperOrigin-RevId: 721834175 Change-Id: I8504ff1a08a643ed35a7282f09f6d6faf75d574c
Looks like this change is breaking Stardoc in downstream pipeline: |
I can work on a fix for Stardoc @tetromino |
I sent bazelbuild/stardoc#279 |
This change implements the Per-target execution platform constraints for exec groups proposal by adding the
exec_group_compatible_with
attribute. It differs from the proposal in its accepted from in two ways:test
exec group always inherits toolchain requirements and constraints from the default exec group. This behavior makes it impossible to apply additional constraints to the default exec group viaexec_compatible_with
without also applying them to thetest
exec group. This preexisting behavior has mostly undesirable consequences: In a multi- or cross-platform build, where exec constraints matter, tests actions and build actions are unlikely to ever be bound to the same toolchain requirements as tests actions don't consume toolchains directly and their intended execution platform is usually closer to the target platform.exec_compatible_with
apply to all non-test
exec groups and renaming the default execution group todefault
. This PR does not implement this change as it makes it easier to accidentally apply execution constraints too broadly as can currently happen withtags
with--incompatible_allow_tags_propagation
(e.g.tags = ["no-remote"]
on a test also applies this restriction to the action compiling the test). More fine-grained primitives for adding constraints pose less of a risk.Fixes #23202
Fixes #23802
RELNOTES: The new
exec_group_compatible_with
attribute on all rules accepts a dictionary mapping exec group names to lists of additional constraints to request from the exec group's execution platform.RELNOTES[INC]: Constraints and toolchain requirements added to the default exec group, for example via the
toolchains
parameter of therule
function or theexec_compatible_with
attribute on all rules, no longer apply to thetest
exec group, which contains the test action on test rules. Instead, useexec_group_compatible_with
to apply constraints and/or define an explicittest
exec group with toolchain requirements on test rules that require it.