-
-
Notifications
You must be signed in to change notification settings - Fork 174
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Questions about theseus_cargo #852
Comments
Great question! If you do find a solution, definitely let us know! |
Thanks for the hint @kevinaboos and quick response! Carefully ensuring that only one version of each dependency is used definitely makes sense to me. That's one direction I can look into. If all we need is to check the semantic version and feature compatibility (which both are high-level concepts in cargo rather than what rustc understands, leading to the absense of the package version in the rmeta) of two crates. I guess what we need is probably a past history of cargo log, which fortunately displays the package version in the env part of a command and the features after
Which sounds even more hacky... The unit-graph feature might also be useful and more structured. I hope this might be also useful for you (or maybe in the future). Anyway all I want to know is the situation in theseus and hopefully getting some comments on how it sounds (how likely it's gonna work) to proceed with the more hacky way which is to maintain a compile history. Feel free to close the issue if you don't have any other comment 😋. Thanks! |
I actually remember exploring But I hadn't yet considered whether they would provide information on crate version --> crate output file. That's not a bad idea, hope it works out for you. One problem with maintaining a compile "history" is that rust flags and many other aspects of a compilation target will affect the metadata/hashes appended to an outputted crate library file, so I don't think that approach is likely to work. I could be wrong though, having not tried it myself. |
I see. Thanks for the comments! I guess it should be fine that many aspects would affect the metadata/hashes appeded to a crate name. As long as there's a log of how each crate library file was compiled, I can probably maintain a mapping of each crate-name-hash to its version and features. I can imagine there will be many potential pitfalls that will make the rustc compiler unhappy by messing with this thing, but I guess l have to give it a try and will let you know the result. |
Hey @kevinaboos, just want to let you know that I have implemented and tested the feature I mentioned for my project. As a PoC, I think it generally works as expected! You can find the code here: https://github.com/phoenix-dataplane/phoenix/blob/21a9733ffd4d3f813ad46db50da9e3678fc6733b/tools/phoenix_cargo/src/main.rs#L188 I basically made a few changes based on your code: different than theseus_cargo, phoenix_cargo I recently also have taken another look at My next step will be to parallelize the compilation and bring incremental compilation back. Is there anything particular you want me to know before proceeding? Thanks! |
It looks like there are many issue with |
This is the general question about how theseus_cargo works. I am facing an almost identical issue mentioned in this doc https://www.theseus-os.com/Theseus/book/building/rust_builds_out_of_tree.html, and I find it quite inspiring. I am currently implementing a similar cargo wrapper that captures cargo's output commands and replaces the arguments (such as
--extern
) and rerun the recreated command. I'm thinking maybe there is a better way to handle dependency replacement (or more likely it is not a problem in Theseus).In
theseus_cargo
, there is aprebuilt_crates_set
that maps a crate name to a crate name with a hash suffix. In reality (maybe just in my more general case), there could be multiple crates underinput_dir_path
sharing the same crate name but with different hash suffixes. They may be compiled from different semantic versions of a package or different features (not sure if this could be the case in the presence of feature unification). Now given a rustc command, how do I know whether I should replace one of its--extern
argument with one of the multiple crates that has the same crate-name but different suffixes underinput_dir_path
?(You could skip reading the following example if you already understand the problem.) Let's say I have a original command (environment variables are omitted)
After the passing through the cargo wrapper, this compile command turns into
In short, the dependency
hashbrown
forindexmap
has been changed tolibhashbrown-b539cb8f16cbe55b.rmeta/rlib
under target/release. However, under target/release, I have more than one versions of libhashbrown (0.11.2, 0.12.3, 0.13.1).My current takeway is that I need someway to check whether the sementic version and features of an extern argument is compatible with the one in
prebuilt_crates_set
. This check obviously further complicates this wrapper but seems doable.I don't think the situation is handled in Theseus's cargo probably because it's not happening in your case. @kevinaboos Could you give a hint on whether it is a real issue or how theseus avoid this situation? Thanks a lot!
The text was updated successfully, but these errors were encountered: