Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Contributing Guidelines] Add script to check that versions are present #75328

Merged
merged 3 commits into from
Jan 8, 2023
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
64 changes: 63 additions & 1 deletion CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -68,7 +68,8 @@ You can review such a PR by checking that the old URL redirects to the new one.
* If it does not, you can ask the author why. This should be handled on a case-by-case basis. Be sure to check that:
1. The package is not being hijacked; check for example that the person making the PR has registered a version of the package before, indicating they are authorized to do so.
2. All the registered revisions of the package are accessible in the new repository.
Specifically, this means checking that all the git-tree-shas can be found in the new repository ([example](https://github.com/JuliaRegistries/General/pull/35965#issuecomment-832721704)).
Specifically, this means checking that all the git-tree-shas can be found in the new repository.
See [the appendix](#appendix-checking-if-a-repository-contains-all-registered-versions-of-a-package) below for a script to automate this checking.

### Other ways to help

Expand All @@ -89,6 +90,67 @@ You generally should not merge your own registrations (though you can make reque
* [write] You can merge improvements to the README, these guidelines, or our workflows.
* [admin] You can give other contributors triage-level access so they can apply labels to PRs, or write-level permissions to merge PRs.

## Appendix: Checking if a repository contains all registered versions of a package

When someone wishes to move a package from one repo to another, it is important that the new repo contains all of the tree hashes corresponding to registered versions of a package. That way these old versions of the package can continue to be installed from the new repository. In order to check if a given repository contains all of the registered versions of a package, the following script can be used:

```julia
using RegistryInstances, UUIDs, Git

const GENERAL_UUID = UUID("23338594-aafe-5451-b93e-139f81909106")

pretty_print_row(row) = println(row.pkg_name, ": v", row.version, " ", row.found ? "found" : "is missing")
pretty_print_table(table) = foreach(pretty_print_row, table)

function check_all_found(table)
idx = findfirst(row -> !row.found, table)
idx === nothing && return nothing
row = table[idx]
error(string("Repository missing v", row.version, " of package $(row.pkg_name)"))
end

function check_packages_versions(pkg_names, repo_url; registry_uuid=GENERAL_UUID, verbose=true, throw=true)
dir = mktempdir()
run(`$(git()) clone $(repo_url) $dir`)

registry = only(filter!(r -> r.uuid == registry_uuid, reachable_registries()))

table = @NamedTuple{pkg_name::String, version::VersionNumber, found::Bool}[]

for pkg_name in pkg_names
pkg = registry.pkgs[only(uuids_from_name(registry, pkg_name))]
versions = registry_info(pkg).version_info
for version in sort(collect(keys(versions)))
tree_sha = versions[version].git_tree_sha1
found = success(`$(git()) -C $dir rev-parse -q --verify "$(tree_sha)^{tree}"`)

push!(table, (; pkg_name, version, found))
end
end
verbose && pretty_print_table(table)
throw && check_all_found(table)
return table
end

check_package_versions(pkg_name, repo_url; kw...) = check_packages_versions([pkg_name], repo_url; kw...)
```

For example, in [General#75319](https://github.com/JuliaRegistries/General/pull/75319), a package author wanted to update the URL associated
to their package "FastParzenWindows". At the time, the package had 1 registered version. We can check that it is present in the new repository via:
```julia
julia> check_package_versions("FastParzenWindows", "https://github.com/ngiann/FastParzenWindows.jl.git");
Cloning into '/var/folders/jb/plyyfc_d2bz195_0rc0n_zcw0000gp/T/jl_ke9E8C'...
...text omitted...
FastParzenWindows: v0.1.2 found
```
We see that this version was found in the new repository. This script was based on [this comment from General#35965](https://github.com/JuliaRegistries/General/pull/35965#issuecomment-832721704),
which involved checking if 4 packages in the same repository all had their versions present in the new repository. That example can be handled as follows:

```julia
pkg_names = ["ReinforcementLearningBase", "ReinforcementLearningCore",
"ReinforcementLearningEnvironments", "ReinforcementLearningZoo"]
check_packages_versions(pkg_names, "https://github.com/JuliaReinforcementLearning/ReinforcementLearning.jl.git")
```

[FAQ]: https://github.com/JuliaRegistries/General#faq]
[naming-guidelines]: https://pkgdocs.julialang.org/dev/creating-packages/#Package-naming-guidelines-1
Expand Down