Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Single-jar pack_sources should conform to Bazel behavior #1378

Merged
merged 10 commits into from
Apr 6, 2022

Conversation

wiwa
Copy link
Contributor

@wiwa wiwa commented Apr 4, 2022

Description

Fix _pack_source_jar in phase_compile.bzl to return single input source jars if there are no source files and only one source jar.
This conforms to the expected behavior of java_common.pack_sources (at least as of Bazel 5.0.0).

The previous PR #1314 did not take into account the behavior of pack_sources with only one input jar.
The field output_source_jar is actually checked in pack_sources, which will only
output the input jar directly if output_source_jar is null.
We also remove any additional . in output_source_jars, i.e. if they look like my-jars.-src.jar,
our change will instead name them my-jars-src.jar.

Motivation

We have a macro which exposes jars with .srcjar extensions. These are in fact the input jars to -src.jar (bundled) output jars. However, we expect that singular input jars will maintain the .srcjar extension and not be bundled.
This is the previous behavior of rules_scala, and we would like to maintain this for our codebase,
since it is also the behavior of Bazel's java_common.pack_sources.

It should be noted that #1314 does not correctly remove the entire extension, which we fix here.
i.e.: my-jars.extension will become my-jars.-src.jar.. Presumably, we should be naming it my-jars-src.jar, without the extra . before the -src.jar suffix. This is likely due to an off-by-one error in the code:
source_jar_name = output_jar.basename[:-len(output_jar.extension)] + "-src.jar"
Thus, we remove any trailing .s before appending -src.jar.

@wiwa wiwa requested review from liucijus and simuons as code owners April 4, 2022 21:12
Copy link
Contributor

@wisechengyi wisechengyi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, Win!

@@ -309,16 +309,26 @@ def _create_scala_compilation_provider(ctx, ijar, source_jar, deps_providers):
)

def _pack_source_jar(ctx, scala_srcs, in_srcjars):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

was confused a bit at the beginning as in_srcjars implies a bool, but looks like it's meant to be a list of src jars.

so nit possibly to rename it scala_src_jars. feel free to do so as a follow up.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed, though we should rename to input_srcjars imo (as followup).

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sgtm. thanks!

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Too lazy for followup. Renamed it and also changed the .-src.jar naming into -src.jar.


# In line with Bazel's java_common.pack_sources,
# We return the initial .srcjar file since there are no source files
# Not sure where the second -src.jar comes from, maybe due to java rules
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you check if adding expect_java_output = False to :source_jar target makes this go away?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It does! Thanks for the test case. I added one more to test multiple source jars, adding SourceJar2.srcjar to the test resources. Also refactored the test logic since it was insanely repetitive.

return input_srcjars[0]
else:
output_jar = ctx.outputs.jar
without_ext = output_jar.basename[:-len(output_jar.extension)]
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure if this would make code look better and probably it's a matter of taste but without_ext could be calculated as output_jar.basename[:-len(output_jar.extension) - 1] if output_jar.extension else output_jar.basename
Please feel free to ignore this comment.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good suggestion! I ended up with (ironically) a longer implementation. Your suggestion made me realize that we shouldn't expect that output_jar has an extension. Personally, I think that the ternary if is too long for one line, so a mutating if is fine.

Copy link
Collaborator

@simuons simuons left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @wiwa . LGTM. Just a couple of minors comments which can be ignored.

@wiwa wiwa requested a review from simuons April 5, 2022 20:24
@wiwa wiwa force-pushed the f/pack-source-1-jar branch 2 times, most recently from 0cacb4f to a319123 Compare April 5, 2022 20:41
@wiwa wiwa force-pushed the f/pack-source-1-jar branch 2 times, most recently from 0cacb4f to d0ebff4 Compare April 5, 2022 20:50
Co-authored-by: Shane Delmore <[email protected]>
@wiwa wiwa force-pushed the f/pack-source-1-jar branch from d0ebff4 to 151cf8b Compare April 5, 2022 21:11
Copy link
Collaborator

@simuons simuons left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @wiwa

@simuons simuons merged commit 0b2bd39 into bazelbuild:master Apr 6, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants