Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Move 'can_go_in_shared_cache' from Action to Rule, with the same default #10789

Closed
wants to merge 1 commit into from

Conversation

ElectreAAS
Copy link
Collaborator

This is my second attempt at going in the direction of caching by default, and caching downloads, following the discussion over at #10729.
I realized that the field can_go_in_shared_cache in Dune_Engine.Action.Full.t already existed, had a default to true, and was never set to false anywhere. I then hijack this by moving the field to Rule.t instead. This makes it so that all calls to Rule.make can now set this variable, and the goal is to change the call in pkg_rule to true when the PR is ready.

cc @Leonidas-from-XIV

@@ -273,7 +272,7 @@ end = struct
, Dep.Facts.digest facts ~env
, target_paths rule.targets.files @ target_paths rule.targets.dirs
, Action.for_shell action
, can_go_in_shared_cache
, true (* TODO: don't forget to remove this *)
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TODO comment intentionnaly left here, as removing the true in the tuple changes the digest, and trips the tests. I wanted to make a separate commit that only removes this line and changes the hash to make it clean for the CI.

@rgrinberg
Copy link
Member

How does it help you to move the flag from the rule to the action? Every rule has a corresponding action, so I think you can just set the flag on the action in the rule.

@ElectreAAS
Copy link
Collaborator Author

How does it help you to move the flag from the rule to the action? Every rule has a corresponding action, so I think you can just set the flag on the action in the rule.

As you can probably guess I'm a little bit confused by the situation.
The field can_go_in_shared_cache in Action.t sounds like it should do the trick, and I can just set this flag to true in more places than currently. However, it is set by default to true, and I can't find a place in the code where that default value is not used. Is this field really doing the thing it should?
Moreover, if I want to enable caching package downloads, can I trust that by adding a variable in dune_rules/pkg_rules it's going to make its way to dune_engine/build_system? That file is the one place where the field is actually used, but I don't know if package rules actually make it there...

@rgrinberg
Copy link
Member

You're on the right track. To recap:

  1. We want to enable the dune cache by default
  2. But doing so will break a lot of existing code that might not be cacheable
  3. So we flip the default, but set this flag for all rules to say that they're incompatible with the dune cache.
  4. We make an exception for the rules defined by pkg_rules.ml. Those should now be explicitly compatible.
  5. But flipping 3, we've made it impossible for users to get back the old behavior of caching everything. So we can introduce a dune flag, say DUNE_CACHE_RULE_DEFAULT=false|true to allow user to get back to the old world where everything would be cached.

Hope that makes sense. Feel free to ask more.

@ElectreAAS
Copy link
Collaborator Author

ElectreAAS commented Aug 2, 2024

Thanks a lot for the explanation! I think I made some progress back on #10710.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants