CFEP-05: conda-forge policy for providing testing/dev/rc builds on anaconda.org/conda-forge #3
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,166 @@ | ||
|
|
||
| <table> | ||
| <tr><td> Title </td><td> Providing pre-release builds on the conda-forge anaconda channel </td> | ||
| <tr><td> Status </td><td> Proposed </td></tr> | ||
| <tr><td> Author(s) </td><td> Eric Dill <[email protected]></td></tr> | ||
| <tr><td> Created </td><td> Sept 09, 2016</td></tr> | ||
| <tr><td> Updated </td><td> Oct 18, 2019</td></tr> | ||
| <tr><td> Discussion </td><td> NA </td></tr> | ||
| <tr><td> Implementation </td><td> NA </td></tr> | ||
| </table> | ||
|
|
||
| ## Abstract | ||
|
|
||
| There are a number of recipe maintainers who have asked for guidance regarding | ||
| how to make pre-release builds available with the CI infrastructure on | ||
| conda-forge. This document outlines how those builds should be labeled on | ||
| anaconda.org so that the more adventurous users can try out these pre-release | ||
| packages. | ||
|
|
||
| ## Motivation | ||
|
|
||
| PyPI allows package maintainers to upload "prerelease" builds which users can | ||
| then install via | ||
|
|
||
| ``` | ||
| pip install PACKAGE_NAME --pre | ||
| ``` | ||
|
|
||
| The same semantics are slightly more cumbersome within the Conda ecosystem but | ||
| are still manageable. The analogous command with `conda` is (for example): | ||
|
|
||
| ``` | ||
| conda install -c conda-forge/label/rc_matplotlib matplotlib | ||
|
There was a problem hiding this comment. What are anaconda.org's restrictions on valid labels?
We have to get confirmation for 1. from Anaconda before giving our blessing. If 2. is possible too, I would like to propose There was a problem hiding this comment. +1 to There was a problem hiding this comment.
I don't share that worry. However, anaconda.org does not seem to like it: I tried out some other characters that the anaconda.org web GUI spits out as valid. A single https://conda.anaconda.org/mbargull/label/rc:_label-test/noarch/repodata.json > CONDA_DEFAULT_CHANNELS= conda search 'mbargull/label/rc::_label-test::_label-test'
Loading channels: done
No match found for: mbargull/label/rc::_label-test::_label-test. Search: mbargull/label/rc::_label-test::*_label-test*
PackagesNotFoundError: The following packages are missing from the target environment:
- mbargull/label/rc::_label-test::_label-test
> CONDA_DEFAULT_CHANNELS= conda search 'mbargull/label/rc:_label-test::_label-test'
Loading channels: done
# Name Version Build Channel
_label-test 0.1.0 0 mbargull/label/rc:_label-testThis would mean We need someone from Anaconda to comment on this. My vote would be to make anaconda.org support |
||
| ``` | ||
|
|
||
| We, as the conda-forge community need to arrive at a consensus for what labels | ||
| we will affix to packages so that there is a consistent user experience. | ||
|
|
||
| ## Backwards Compatibility | ||
|
|
||
| There are only minimal concerns of backwards compatibility. | ||
| The number of feedstocks that issue pre-releases is small, mitigating | ||
| backwards compatibility issues. | ||
|
|
||
| ## Specification | ||
|
|
||
| Mark the pre-release package on anaconda.org as "dev" or "rc" by adding them to | ||
| the appropriate label. | ||
| The semantics of these labels should generally follow the | ||
| [guidelines](https://docs.python.org/devguide/devcycle.html#stages) that Python | ||
| itself follows. | ||
|
|
||
| - "rc": [Beta](https://docs.python.org/devguide/devcycle.html#beta) and [Release | ||
| Candidate](https://docs.python.org/devguide/devcycle.html#release-candidate-rc) | ||
| (RC). No new features. Bugfix only. | ||
|
|
||
| - "dev": [Pre-Alpha](https://docs.python.org/devguide/devcycle.html#pre-alpha) | ||
| and [Alpha](https://docs.python.org/devguide/devcycle.html#alpha). These are | ||
| still packages that could see substantial changes | ||
| between the dev version and the final release. | ||
|
|
||
| See the Appendix for dev and rc version order specifics. | ||
|
|
||
| To create a `dev` or `rc` package a PR can be issued into the `dev` or `rc` branch of the | ||
| feedstock. | ||
| This branch must change the `recipe/conda_build_config.yaml` to point to the `dev_<package_name>` or `rc_<package_name>` label. | ||
|
|
||
| For example, matplotlib rc releases would include: | ||
| ```yaml | ||
| ... | ||
| channel_targets: | ||
| - conda-forge rc_matplotlib | ||
|
There was a problem hiding this comment. Can you also add an example for how to use a dev package in another dev package? There was a problem hiding this comment. Is this something we want to support? Would users get the correct env if they don't have both dev channels in their .condarc/cli call? There was a problem hiding this comment. They won't, but the user should be given specific instructions either way. I think we should support it or say that we don't. There was a problem hiding this comment. I would lean toward not supporting this for now, seeing if people need it and then revisiting. This way we are more likely to provide a stable system day one. There was a problem hiding this comment. I'd need it and I saw some people asking about it in gitter, but can't find it right now. There was a problem hiding this comment. Can you please push up the language you want to see? There was a problem hiding this comment. We also need to make sure that these extra channels don't leak into downstream There was a problem hiding this comment. Shall I add a section like There was a problem hiding this comment. That would be great! Thank you! There was a problem hiding this comment. Have a look and feel free to change it. |
||
| ``` | ||
|
|
||
| If a pre-release build of B depends on a pre-release build of A, then A should have, | ||
| ```yaml | ||
| ... | ||
| channel_targets: | ||
| - conda-forge rc_A | ||
| ``` | ||
| while B should have, | ||
| ```yaml | ||
| ... | ||
| channel_sources: | ||
| - conda-forge/label/rc_A,conda-forge,defaults | ||
| channel_targets: | ||
| - conda-forge rc_B | ||
| ``` | ||
| in `recipe/conda_build_config.yaml` in their respective feedstocks. | ||
|
|
||
| NOTE: A rerender needs to happen for these changes to reflect in CI files. | ||
|
|
||
| ## Alternatives | ||
|
|
||
| * Upload pre-release packages to a different anaconda.org user, perhaps | ||
| `conda-forge-pre` | ||
| * Dump all pre-release packages into one label: "pre" | ||
|
|
||
| We understand the proposed approach does not have the same user experience | ||
| as `pip --pre`, which seems simpler and easier to use. | ||
| Currently this level of functionality is not possible with conda while | ||
| maintaining the stability of environment we currently provide. | ||
| This CFEP proposes and initial step toward supporting dev/rc releases. | ||
| As we explore this functinality more we can re-evaluate the user and maintainer | ||
| expreience and how best to provide this functionality. | ||
|
|
||
| ## FAQ | ||
|
|
||
| * What about "alpha" and "beta" labels? | ||
|
|
||
| There is not yet heavy use of labels on the conda-forge channel on anaconda.org. | ||
| Given the light usage of labels thus far it seems rather unnecessary to | ||
| introduce many labels. "dev" and "rc" seem like a nice compromise. | ||
|
Comment on lines
+109
to
+113
There was a problem hiding this comment. Just FYI for those who haven't seen it - there's two competing PEPs for increasing the release cadence of python. One of them (see PEP text and discussion here) is proposing to add a rolling release stream with alphas (ABI-breaking) and betas (stable). The PEP is not accepted yet, and this CFEP could of course be changed later if necessary, so I don't think there's anything to do, I just wanted to point this out, because it touches on this topic. There was a problem hiding this comment. Yes, we could change the policy later if needed. Personally, I'd assume future CPython alphas could go into But basically, I just wanted to say thanks for the reference! (Though, TBH, I TLDR;'d that one, haha...) There was a problem hiding this comment. Actually, it was the other PEP that ended up being accepted. This means a smaller change compared to the status quo, just a faster release cadence. |
||
|
|
||
| * How do I use labels to install these pre-release packages? | ||
|
|
||
| Use the following command, but replace `PACKAGE_NAME` with the package you want | ||
| to install and replace `LABEL` with "rc" or "dev": | ||
|
|
||
| ``` | ||
| conda install -c conda-forge/label/LABEL_PACKAGE_NAME -c conda-forge PACKAGE_NAME | ||
| ``` | ||
|
|
||
| For example, let's install matplotlib from the "rc" label: | ||
|
|
||
| ``` | ||
| conda install -c conda-forge/label/rc_matplotlib -c conda-forge matplotlib | ||
| ``` | ||
|
|
||
| ## Work Items | ||
|
|
||
| - Fix intructions to install the package in the README. | ||
|
|
||
| Currently, conda-smithy generates incorrect instructions in the README. | ||
| This is tracked in https://github.com/conda-forge/conda-smithy/issues/996 | ||
|
|
||
| - Disable uploading to conda-forge/label/main if sources include other pre-release labels | ||
|
|
||
| This is tracked in https://github.com/conda-forge/conda-forge-ci-setup-feedstock/issues/66 | ||
|
|
||
| ## Appendix | ||
|
|
||
| If you wish to add numbers to your "dev" or "rc" you should follow the | ||
| [guidelines](http://conda.pydata.org/docs/spec.html#build-version-spec) put | ||
| forth by Continuum regarding version sorting in `conda`. Also see the [source | ||
| code for conda | ||
| 4.2.13](https://github.com/conda/conda/blob/4.2.13/conda/version.py#L93-L119). | ||
| The tl;dr here is that conda sorts as follows: | ||
|
|
||
| ``` | ||
| < 1.0 | ||
| < 1.1dev1 # special case 'dev' | ||
| < 1.1.0dev1 # special case 'dev' | ||
| == 1.1.dev1 # 0 is inserted before string | ||
| < 1.1.0rc1 | ||
| < 1.1.0 | ||
| ``` | ||
|
|
||
| So make sure that you ***tag*** your package in such a way that the package name | ||
| that conda-build spits out will sort the package uploaded with an "rc" label | ||
| higher than the package uploaded with the "dev" label. | ||
|
|
||
|
|
||
| ## Copyright | ||
|
|
||
| This document has been placed in the public domain. | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.