Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add COPYRIGHT-*.html files to distribution and update COPYRIGHT #133461

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

jonathanpallant
Copy link
Contributor

@jonathanpallant jonathanpallant commented Nov 25, 2024

  • Updates the COPYRIGHT file to describe how we actually do things now, and removes the licence text from it as they are stored elsewhere.
  • dist tarballs get all of the files in LICENSES/*.
    • This folder is managed by reuse and each file exists because we refer to the licence somewhere in our tree. We should be supplying these licence texts to anyone who obtains a copy of the source code and now we do.
  • The binary rust tarball gets COPYRIGHT.html and COPYRIGHT-library.html, which are auto-generated files that describe the licence information for both the in-tree source files used to build the Rust toolchain, and the out-of-tree dependencies we used to build the toolchain.
    • The other binary tarballs are unchanged, for now. In future you need to make a call whether to ship multiple version of COPYRIGHT.html, or whether to try and make, for example, a cargo-specific COPYRIGHT.html file.
  • The LICENSE-MIT file now includes a blanket copyright statement, as the text indicates that it should and because users will expect to know who owns the copyright of the material they have been given (even if the answer is 'lots of people').

try-job: x86_64-fuchsia

@rustbot
Copy link
Collaborator

rustbot commented Nov 25, 2024

r? @albertlarsan68

rustbot has assigned @albertlarsan68.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

@rustbot rustbot added A-meta Area: Issues & PRs about the rust-lang/rust repository itself A-testsuite Area: The testsuite used to check the correctness of rustc S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap) T-infra Relevant to the infrastructure team, which will review and decide on the PR/issue. labels Nov 25, 2024
@rust-log-analyzer

This comment has been minimized.

@Kobzol
Copy link
Contributor

Kobzol commented Nov 26, 2024

r? @Kobzol

(although as JP mentioned, this will need some opinions from the council, probably)

@rustbot rustbot assigned Kobzol and unassigned albertlarsan68 Nov 26, 2024
@Kobzol
Copy link
Contributor

Kobzol commented Nov 28, 2024

Can you please rebase this to make it a bit easier to review? Thanks!

@jonathanpallant jonathanpallant force-pushed the add-copyright-files-to-dist branch from c78c7ba to 249479a Compare November 28, 2024 16:19
@jonathanpallant
Copy link
Contributor Author

Done

Copy link
Contributor

@Kobzol Kobzol left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left a few nits/comments, but otherwise looks good, from the technical side.

COPYRIGHT Outdated Show resolved Hide resolved
COPYRIGHT Outdated Show resolved Hide resolved
COPYRIGHT Outdated Show resolved Hide resolved
src/bootstrap/src/core/build_steps/dist.rs Show resolved Hide resolved
src/bootstrap/src/utils/tarball.rs Outdated Show resolved Hide resolved
@Kobzol
Copy link
Contributor

Kobzol commented Dec 3, 2024

I thought that we would copy the MIT/Apache licenses from the LICENSES directory to the root of the dist archives, but I suppose that having the MIT/Apache licenses duplicated is not an issue, and at least they are immediately visible in the repo, and e.g. GitHub will probably have an easier time autodetecting what license we use.

From the technical point of view, I'm fine with this PR (well, a squash of the commits would be nice, but that's a detail). For the legal side, we should probably wait for the council meeting (?).

@jonathanpallant
Copy link
Contributor Author

I thought that we would copy the MIT/Apache licenses from the LICENSES directory to the root of the dist archives

  1. The MIT file in the LICENSES directory is a vanilla version with a template copyright statement
  2. The COPYRIGHT file has to refer to ./LICENCES/MIT.txt when read from a git checkout, but ./MIT.txt when in a distribution tarball and I couldn't workout how to resolve that. So I just duplicated the file, so the relative path between the MIT file and the COPYRIGHT file is the same in the git repo and in the tarball.

@traviscross
Copy link
Contributor

Have we tested, does GitHub still autodetect the license with these changes? It would seem unfortunate to lose that.

@Kobzol
Copy link
Contributor

Kobzol commented Dec 6, 2024

This PR (at least currently) keeps the original LICENSE-MIT and LICENSE-APACHE files, so the autodetection should keep working.

@ehuss
Copy link
Contributor

ehuss commented Dec 6, 2024

Thanks @jonathanpallant for moving this forward, this looks great! The Council did briefly touch on this today, and there should be some more followup beyond my personal comment here.

I had some questions:

The source tarballs get all of the files in LICENSES/*.

This seems good to me, but I'm curious how someone is to interpret this directory. Is this directory intended for a user to look at when they download the source? Or is it primarily used for building/installing to generate the output?

The other binary tarballs are unchanged, for now.

Curious about your thoughts here. Is the intent to add more to those in the future?

The LICENSE.rtf file used in Windows builds now pulls text from LICENSES.

Any particular reason to do this? Those files look like generic copies with placeholders. Shouldn't it be using the license files from the root that are the actual license we are asserting?

  • The top-level LICENSE-APACHE and LICENSE-MIT files are now gone.

It looks like this has been reverted. Can you update the PR comment for whatever is now current?


It looks like one of the few substantive changes is 683dbe4, can you include that in the PR description? I am recommending that we double-check with our legal counsel on this particular thing, because there has been some disagreement on which form it should take, and I think it would be good to have an official answer to settle that.1


It was helpful to me to actually see what this looks like, so I created a copy of the relevant files now in the binary distribution with the changes from this PR: copyright.zip

Footnotes

  1. Some more context on the commit with comments: https://github.com/rust-lang/rust/commit/2a8807e889a43c6b89eb6f2736907afa87ae592f

@traviscross
Copy link
Contributor

As @ehuss mentioned, we discussed this in our council call today, and given that we had some questions here, including those that Eric mentioned above, we tabled this matter to our next call on 20 December.

@jonathanpallant
Copy link
Contributor Author

Any particular reason to do this? Those files look like generic copies with placeholders. Shouldn't it be using the license files from the root that are the actual license we are asserting?

Good point. Now we're leaving those two files in, the Windows installer can use them.

) add COPYRIGHT*.html files to the rustc binary distribution
) add contents of LICENSE folder to dist tarballs, because some of our in-tree licences will require that the license text is reproduced.
) The wording of COPYRIGHT is adjusted to not include license text (`reuse` ensures that it's in the LICENSE folder)
) A blanket copyright notice is added to LICENCE-MIT as required by the text.

The general approach is that the license statements are now compiled using a tool in CI (generate-copyright), and you get either:

* the source code (COPYRIGHT, LICENCE-APACHE, LICENCE-MIT, REUSE.toml and the LICENCES folder), or
* the compiled version (COPYRIGHT.html, COPYRIGHT-library.html and the LICENCES folder).
@jonathanpallant jonathanpallant force-pushed the add-copyright-files-to-dist branch from 683dbe4 to f9c1699 Compare December 9, 2024 10:20
@jonathanpallant
Copy link
Contributor Author

Apologies for the messy PR - it has required an iterative approach. I've now squashed everything and re-worded the commit message and the opening message in this PR.

@Kobzol
Copy link
Contributor

Kobzol commented Dec 9, 2024

LGTM from my side, I'll let the council take another look, but otherwise you can r=me.

@ehuss
Copy link
Contributor

ehuss commented Dec 9, 2024

I sent an email to the Foundation to get a follow up on the MIT copyright notice.

@ehuss ehuss removed the T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap) label Dec 19, 2024
@bors
Copy link
Contributor

bors commented Jan 7, 2025

⌛ Testing commit f7f2fa5 with merge 14ef16e...

@rust-log-analyzer
Copy link
Collaborator

A job failed! Check out the build log: (web) (plain)

Click to see the possible cause of the failure (guessed by this bot)

@bors
Copy link
Contributor

bors commented Jan 7, 2025

💔 Test failed - checks-actions

@bors bors added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels Jan 7, 2025
@ehuss
Copy link
Contributor

ehuss commented Jan 7, 2025

@bors retry

dist-aarch64-linux died somewhere during the upload stage

The hosted runner: GitHub Actions 55 lost communication with the server. Anything in your workflow that terminates the runner process, starves it for CPU/Memory, or blocks its network access can cause this error.

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Jan 7, 2025
bors added a commit to rust-lang-ci/rust that referenced this pull request Jan 7, 2025
…t, r=Kobzol,traviscross,ehuss

Add COPYRIGHT-*.html files to distribution and update `COPYRIGHT`

* Updates the `COPYRIGHT` file to describe how we actually do things now, and removes the licence text from it as they are stored elsewhere.
* dist tarballs get all of the files in `LICENSES/*`.
  * This folder is managed by `reuse` and each file exists because we refer to the licence somewhere in our tree. We should be supplying these licence texts to anyone who obtains a copy of the source code and now we do.
* The binary rust tarball gets `COPYRIGHT.html` and `COPYRIGHT-library.html`, which are auto-generated files that describe the licence information for both the in-tree source files used to build the Rust toolchain, and the out-of-tree dependencies we used to build the toolchain.
   * The other binary tarballs are unchanged, for now. In future you need to make a call whether to ship multiple version of COPYRIGHT.html, or whether to try and make, for example, a cargo-specific COPYRIGHT.html file.
* The `LICENSE-MIT` file now includes a blanket copyright statement, as the text indicates that it should and because users will expect to know who owns the copyright of the material they have been given (even if the answer is 'lots of people').

try-job: x86_64-fuchsia
@bors
Copy link
Contributor

bors commented Jan 7, 2025

⌛ Testing commit f7f2fa5 with merge 8d787c3...

@rust-log-analyzer
Copy link
Collaborator

A job failed! Check out the build log: (web) (plain)

Click to see the possible cause of the failure (guessed by this bot)

@bors
Copy link
Contributor

bors commented Jan 7, 2025

💔 Test failed - checks-actions

@bors bors added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels Jan 7, 2025
@ehuss
Copy link
Contributor

ehuss commented Jan 7, 2025

@bors retry

dist-aarch64-linux died in the exact same place? Maybe related to the ongoing incident? I'm really confused.
https://www.githubstatus.com/incidents/dk61qxd21mtl

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Jan 7, 2025
bors added a commit to rust-lang-ci/rust that referenced this pull request Jan 8, 2025
…t, r=Kobzol,traviscross,ehuss

Add COPYRIGHT-*.html files to distribution and update `COPYRIGHT`

* Updates the `COPYRIGHT` file to describe how we actually do things now, and removes the licence text from it as they are stored elsewhere.
* dist tarballs get all of the files in `LICENSES/*`.
  * This folder is managed by `reuse` and each file exists because we refer to the licence somewhere in our tree. We should be supplying these licence texts to anyone who obtains a copy of the source code and now we do.
* The binary rust tarball gets `COPYRIGHT.html` and `COPYRIGHT-library.html`, which are auto-generated files that describe the licence information for both the in-tree source files used to build the Rust toolchain, and the out-of-tree dependencies we used to build the toolchain.
   * The other binary tarballs are unchanged, for now. In future you need to make a call whether to ship multiple version of COPYRIGHT.html, or whether to try and make, for example, a cargo-specific COPYRIGHT.html file.
* The `LICENSE-MIT` file now includes a blanket copyright statement, as the text indicates that it should and because users will expect to know who owns the copyright of the material they have been given (even if the answer is 'lots of people').

try-job: x86_64-fuchsia
@bors
Copy link
Contributor

bors commented Jan 8, 2025

⌛ Testing commit f7f2fa5 with merge f86e690...

@rust-log-analyzer
Copy link
Collaborator

A job failed! Check out the build log: (web) (plain)

Click to see the possible cause of the failure (guessed by this bot)

@bors
Copy link
Contributor

bors commented Jan 8, 2025

💔 Test failed - checks-actions

@bors bors added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels Jan 8, 2025
@ehuss
Copy link
Contributor

ehuss commented Jan 8, 2025

@bors r-

Looks like dist-aarch64-linux is running out of disk space. It's not immediately obvious to me looking at the diff here why that might be, but it looks like it is using over 2GB more space than a normal build? That doesn't seem right.

@bors bors added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Jan 8, 2025
@ehuss
Copy link
Contributor

ehuss commented Jan 8, 2025

Oh, I didn't realize this was using cargo vendor. That would definitely use a few extra GB of space.

#135262 is going to be moving the job to a much larger runner.

@jonathanpallant
Copy link
Contributor Author

Yeah, sadly. I don't know any other way to trawl the source code for every dependency looking for the relevant notice and licence files that we are required to reproduce.

@jonathanpallant
Copy link
Contributor Author

I also note that the process added here is independent of the target being built (it's the same source code, regardless of target). So in theory, you could have a job that builds the COPYRIGHT.html and COPYRIGHT-library.html files once, and you can then just include those files alongside every binary artefact, without rebuilding the files N times. I don't know how to drive your CI pipeline to do that though.

@Kobzol
Copy link
Contributor

Kobzol commented Jan 9, 2025

We will probably experiment with something like this (fan-off) to improve CI times, so ee can include copyright in that. But that's quite complex, so no need to gate this on these experiments.

I'll reapprove this once the ARM job is moved to an ARM runner.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-meta Area: Issues & PRs about the rust-lang/rust repository itself A-testsuite Area: The testsuite used to check the correctness of rustc S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap) T-leadership-council Relevant to the leadership council.
Projects
None yet
Development

Successfully merging this pull request may close these issues.