-
Notifications
You must be signed in to change notification settings - Fork 140
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
docs releases #766
Merged
+66
−53
Merged
docs releases #766
Changes from all commits
Commits
Show all changes
14 commits
Select commit
Hold shift + click to select a range
5295c0c
docs releases
plebhash 0427297
add some info
plebhash 6116342
Merge branch 'main' into docs-releases
plebhash d64363f
add git branching; fix some docs
plebhash b5491e7
add paragraph about patches
plebhash 24f7648
add branch naming conventions
plebhash bb039a4
specify main branch
plebhash 22e196e
specify release cycle
plebhash ec1e552
lint
plebhash 0e3de0b
Update RELEASE.md
plebhash c114f16
Merge branch 'main' into docs-releases
plebhash 4bc7e96
release every 4 weeks
plebhash 985c438
lint
plebhash 2c4cc3f
leave release cycle timelines as tbd
plebhash File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,66 +1,79 @@ | ||
# Releasing Roles Binaries and Publishing | ||
# Versioning | ||
|
||
The github binary releases of the roles and the publishing of the SRI crates are both handled in the `release.yaml` workflow. This workflow must be manually | ||
started by navigating to the "Actions" tab in the SRI repo, then navigating to the Release workflow section, and clicking "Run Workflow". | ||
SRI releases follow [SemVer 2.0.0](https://semver.org/). | ||
|
||
## Crates Publishing | ||
Given a version number `MAJOR.MINOR.PATCH`, we increment the: | ||
- `MAJOR` version when incompatible API changes are introduced | ||
- `MINOR` version when functionality is added in a backward compatible manner | ||
- `PATCH` version when backward compatible bug fixes are introduced | ||
|
||
For publishing crates we use the `cargo-release` tools because it easily handles the order in which crates can be published. Crates are published | ||
in the `crates_publish` job. Currently, crates.io rate limits the publishing of more than 5 NEW crates. This means that if we have more than 5 crates | ||
to publish, that are not currently published to crates.io, this workflow will not publish anything. To bypass this we will need to add the `--exclude [crate_name]` | ||
flag for as many crates as needed to the `cargo-release` command in the "Publish" step. For example, if there are 7 new crates to publish, we will need to add | ||
`--exclude [crate_1] --exclude [crate_2]` to the command. This is not an issue for publishing version updates since the crates are not NEW publishes. | ||
SRI has a global version, which uses git tags and keeps track of how the codebase evolves across time as a whole. | ||
|
||
### Current Successful Publishes | ||
Every internal SRI crate also follows SemVer 2.0.0, but each crate version is only set on the respective `Cargo.toml`, | ||
(no git tags), and it evolves independently of the global release version. | ||
|
||
Due to either github dependencies or a crate failing the build stage during publish not all crates are being published. | ||
Whenever a `PATCH` is introduced, it is applied to all the latest `MAJOR` releases. | ||
For example: imagine there's releases `v1.0.0`, `v1.1.0`, and `v2.0.0`. A bug is found, dating back all the way to `v1.0.0`. | ||
A patch is applied such that the following new tags are introduced: `v1.1.1` and `v2.0.1`. | ||
|
||
- [x] buffer_sv2 | ||
- [x] binary_codec_sv2 | ||
- [x] binary_sv2 | ||
- [x] common_messages_sv2 | ||
- [x] const_sv2 | ||
- [x] derive_codec_sv2 | ||
- [x] framing_sv2 | ||
- [x] serde_sv2 | ||
- [x] sv1_api | ||
- [] noise_sv2 - github depependency (ed25519-dalek) | ||
- [] codec_sv2 - noise dependency | ||
- [] template_distribution_sv2 - build failure | ||
- [] mining_sv2 - publishes after template_distribution_sv2 | ||
- [] job-declaration_sv2 - publishes after template_distribution_sv2 | ||
- [] roles_logic_sv2 - publishes after template_distribution_sv2 | ||
- [] network_helpers - publishes after template_distribution_sv2 | ||
- [] error_handling - had to exclude to bypass rate limit but will probably publish now | ||
- [] sv2_ffi - had to exclude to bypass rate limit but will probably publish now | ||
- [] all roles - noise dependency | ||
# Git Branching Strategy | ||
|
||
## Github Roles Releases | ||
We follow a simplified [gitflow](https://nvie.com/posts/a-successful-git-branching-model/) branching strategy. | ||
|
||
The roles binaries are released in github in all jobs other than the `crates_publish` job. To be able to publish, | ||
the job must be able to find a new tag (one that hasnt already been released), matching the crate name of the role to be | ||
released. For example, imagine there is currently a release for pool_sv2-v1.0.0. For the jobs releasing a new pool version to succeed, | ||
it must be able to find a tag containing the string "pool_sv2" with a version greater than 1.0.0. This should not be an issue since the tags | ||
are automatically created by the `autoversion.yaml` workflow, and moved into main when the generated PR is merged. | ||
Although our strategy is very similar to the classic gitflow model, we do not keep release branches. | ||
|
||
### Github Release Issue | ||
![](git-branching.png) | ||
|
||
Currently we do not support windows releases because we are unable to run the step to return the last tag for a given crate name. See the section above | ||
for an explaination of the process. | ||
## Principal Branches | ||
|
||
The SRI repo holds two principal branches with an infinite lifetime: | ||
- `main` | ||
- `dev` | ||
|
||
We consider `main` to be the branch where the source code of `HEAD` always reflects a production-ready state. | ||
|
||
## Github Binary Releases | ||
We consider `dev` to be the branch where the source code of `HEAD` always reflects a state with the latest delivered development changes for the next release. | ||
|
||
# Versioning | ||
The SRI team will decide the appropriate time when the changes to `dev` are merged back into `main` and then tagged with a release number while bumping `MAJOR` and/or `MINOR`. | ||
|
||
## Feature Branches | ||
|
||
New features are developed into separate branches that only live in the contributor's forks. | ||
|
||
- branch off from: `dev` | ||
- merge back into: `dev` | ||
- naming convention: `feat-x`, where `x` describes the feature | ||
|
||
## Patch Branches | ||
|
||
Bugs are patched into separate branches that only live in the contributor's forks. | ||
|
||
- branch off from: `main` | ||
- merge back into: `main` + `dev` | ||
- naming convention: `patch-x`, where `x` describes the bug/patch | ||
|
||
# Releasing Roles Binaries | ||
|
||
The [release page of SRI repo](https://github.com/stratum-mining/stratum/releases) provides executable binaries for all SRI roles, targeting popular hardware architectures. | ||
|
||
The GitHub binary releases of the roles are handled in the `release-bin.yaml` workflow. | ||
|
||
This workflow is manually started by navigating to the "Actions" tab in the SRI repo, then navigating to the Release workflow section, and clicking "Run Workflow". | ||
|
||
Note: in order to be able to manually trigger the "Run Workflow" button, the user needs to have "Write" permissions on the repository, otherwise the button will not show up on the UI. | ||
|
||
# Publishing Library Crates | ||
|
||
Although SRI has a global release cycle, which is attached to the binaries, each internal crate also has its own versioning history. | ||
|
||
Lib crates are published to crates.io in the `release-lib.yaml` workflow. The workflow tries to update all the library crates. | ||
|
||
If a crate is not updated successfully, the step will fail, but since all steps have `continue-on-error` set to true, the workflow will continue. | ||
|
||
Since steps can fail, the output of the action must be manually checked. | ||
|
||
Versioning is handled by the `autorelease.yaml` workflow. The workflow will will auto detect changes in crates and auto bump the patch versions | ||
in the target crate, as well as any crate that uses the target crate as a dependency, if changes are detected. It will then push a new commit with the | ||
versioning changes to the `bot/versioning` branch, and Lastly, it will auto create a PR (bot/versioning -> main). Since cargo-smart-release does not yet | ||
support autodetection of MAJOR and MINOR changes with regards to Semver, any MAJOR/MINOR versioning will need to be manually changed in the package Cargo.toml. | ||
Every PR to `main` (either coming from `dev`, or a patch) needs to increase the version of whatever crate it is touching. Otherwise, we will mess up the dependency chain of whoever is fetching from crates.io | ||
|
||
## Versioning Notes | ||
Manually running `cargo release` in the various workspaces helps to prepare the version number before the releases are published. | ||
|
||
This workflow should not run if the push is resulting from this workflow, but this not necessarily guaranteed since the | ||
a developer could make a minor change to the bot/versioning PR, and the latest author would no longer be "Github Actions Bot". | ||
If the workflow does run again, there will just be no changes, and the PR can be closed. | ||
Every time we bump some crate's version, `release-libs.yaml` needs to be manually triggered in order to update crates.io. |
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
How do we do it? I guess that we should have a branch (or a tag?) for all the last x majior version of each libs.
This seems complicate. Another way could be to create the branch only when needed. We fix a bug for
lib1-2.0.0
we find that alsolib1-1.1.0
is affected and we want to backport the fix. We create a new branch from whatever was the state of the repo when lib1 have been go from v1 ti v2 and we apply the fix. If we do that only for last majior version, only when we have a bug that affect also the last majior version I do not think that we will end up with 'too many' branches, considering that most library in this repo will do not get a new majior version in many years. If this is a concern we can just say that we support the last majior version and that is not safe to use any version that is not the majior, or we can think about it when we will have more then one majior version (0 wont be maintained cause is unstable). Maybe we can just leave the various possibility here, for now we can say that we support only last majior so we do not make promise and when we will have more majior we will think about it.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
On lines 10 and 12 of my proposal, there is a very important distinction we need to keep in mind:
Let's imagine this scenario, where both
crate-A
andcrate-B
are affected by a bug on the following versions:Here, Global Release Versions would be indicated as git tags.
Crate versions however, would only be indicated by
Cargo.toml
andcrates.io
. Otherwise, we would end up with too many tags.After we patch the bug, we would have the following:
Cargo.toml
)Cargo.toml
)Cargo.toml
)Cargo.toml
)There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ok make sense. Only thing I wouldn't maintain version 0 and 1. I would just bump majior version of everyting to 2. And start from there 2 reasons here:
An example of that is the noise crate.
Also is startum v2 so it can make sense to start from 2. Otherwise we should remove the old library from crates.io but I dont' know if is something possible or not.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I agree with what you're saying @Fi3, the only thing that bugs me is starting with 2.0.0, afaik we didn't really have an official release yet, and in public we always referred to things as MVP and later update. 1.0.0 is really a nice clean milestone, I am worried 2.0.0 may communicate certain level of maturity and stability which I am not sure we have yet.
Doesn't sound like something impossible, I doubt there can be any consequences to this?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
With regards to
2.0.0
, we should keep the SRI vs SV2 distinction in mind.Sure, SV2 implies v2, but only for the Specifications.
That doesn't necessarily mean that as a Rust codebase, SRI should start at v2.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
With regards to crates.io removal, I found this:
rust-lang/crates.io#1506 (comment)
which is discouraging... it basically means we will not be able to remove the old libraries from crates.io, and we cannot "start from scratch".
I did understand your point on starting on
2.0.0
@Fi3 . It would be a way to put all SRI crates at the same starting point. But I still think starting on2.0.0
would be bad, for the reasons I explained in my previous comment.I wonder if we could do the same, but with all crates starting at
1.0.0
?I did a quick search on all workspaces and I didn't see any crate that is above
1.0.0
.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
from @Fi3 (over discord):
so assuming we will set all crates to
v1.0.0
(exceptnoise-sv2
), I crated this PR: #785There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sounds good. All the 0s will become 1.0.0. All the one already to 1 are yanked and we start from the very next minor. And we say that 0 is not supported anymore.