CFEP-05: conda-forge policy for providing testing/dev/rc builds on anaconda.org/conda-forge

[ericdill]

Thoughts @conda-forge/core ?

[msarahan]

This is great, @ericdill. And I'd say you got to it very quickly. Regarding the point about tooling for how to guide packagers towards how to get on a testing label, I think we can do some kind of detection of extra version information.

For instance, the VersionOrder class in conda (https://github.com/conda/conda/blob/master/conda/version.py#L20) splits things up, and if there's extra text, you end up with a tuple for a given split rather than just the number. I have used something like this recently to filter conda build's annoying "you should update!" messages: https://github.com/conda/conda-build/blob/a3f7e20e84555884aaa414b81188933f176cb4a0/conda_build/build.py#L495-L503

I think we could use something like that in conda-forge tools somehow to disallow packages with version tags containing strings, and to help set people on the right course of a "testing" package.

[ericdill]

@msarahan Oh, wow, that's a great solution! Should we consider making the default be uploading to the testing channel unless the len(VersionOrder(pkg).version[3]) == 1?

[jakirkham]

Haven't read this in detail yet, but glad to see you are championing this, @ericdill. Before we dig in too deep, it would be nice to settle how we steer CFEP's forward. Please see PR ( #1 ). Once that is settled, would be more than happy to give this a closer look.

[jakirkham]

Minor note: The testing term in the name confused me a bit as there is a need for testing packages that are built from a stable release, but with new features or in some different way. Maybe we could tweak the title to be more explicit in noting these are pre-releases. Unless of course I'm mistaken and this is trying to resolve both issues. In which case, I would say they are two different things that we should deal with in separate CFEPs. Already have given the testing point a fair bit of thought and am planning to submit a CFEP on that. See this hackpad. Would certainly appreciate your thoughts on it.

[ericdill]

Maybe we could tweak the title to be more explicit in noting these are pre-releases

Good idea. I'll update it

Unless of course I'm mistaken and this is trying to resolve both issues

Nope. You're correct. I had not even thought of the testing case. Someone on the call had used the term "testing" and so I am just propagating it here, but this CFEP is really about pre-releases.

Already have given the testing point a fair bit of thought and am planning to submit a CFEP on that. See this hackpad. Would certainly appreciate your thoughts on it

Awesome! I'll have a look as time permits

[ericdill]

Before we dig in too deep, it would be nice to settle how we steer CFEP's forward. Please see PR ( #1 )

Yup. I definitely recognize that those need to get sorted first. I gave that a read and used the formatting and guidelines from that to inform what sort of general structure/formatting/content cfep02 should have. I'm definitely putting cfep02 forward with the full understanding that it could need to be totally revamped depending on how #1 goes.

[jakirkham]

Thanks @ericdill. Was more mentioning the latter point so as to avoid having you wasting cycles on things YTBD. As far as the content goes, it seems like that can develop contemporaneously AFAIAC.

[jjhelmus]

@msarahan Oh, wow, that's a great solution! Should we consider making the default be uploading to the testing channel unless the len(VersionOrder(pkg).version[3]) == 1

I do not think we should do this. Not all packages follow a particular versioning scheme and I think conda-forge should be as version scheme agnostic as possible. I'd rather see uploading to a testing channel be purely opt-in.

[pelson]

I'd like to see us auto-expire test/dev/rc labelled packages.

Straw-man:

• testing packages should stick around for 2 weeks
• dev and rc packages should stick around for 2 months, or as soon as a newer dev/rc package is available (whichever comes first)

[pelson]

To be completely clear, I'd like to rule "debug" python builds out of scope from this CFEP.

[ericdill]

I like the idea of auto-expiring the test/dev/rc labeled packages. Any suggestions as to how this would work? Where do we store the upload times?

[ericdill]

(It's on my todo list for this week to circle back to CFEP02)

[ericdill]

@pelson Where do we track when the package was actually uploaded? I can get the repodata.json for a given label with the following url template: "https://conda.anaconda.org/{channel}/label/{label}/{platform}/repodata.json" where channel is "conda-forge", label is "rc" and platform is "linux-64", but the records in repodata.json do not include a timestamp. Here is a code snippet to grab the rc repodata.json and some output to show the keys that they it does contain (which, afaict are not substantially different from the repodata.json for the main label):

import requests
info = requests.get('https://conda.anaconda.org/conda-forge/label/rc/linux-64/repodata.json').json()
info['packages']['matplotlib-2.0.0b1-np111py34_2.tar.bz2']

produces

{'arch': 'x86_64',
 'binstar': {'channel': 'rc',
  'owner_id': '5528f42ce1dad12974506e8d',
  'package_id': '55f96af4d3234d09d1951089'},
 'build': 'np111py34_2',
 'build_number': 2,
 'depends': ['cycler >=0.10',
  'freetype 2.6*',
  'libpng 1.6*',
  'numpy 1.11*',
  'pyparsing',
  'pyqt',
  'python 3.4*',
  'python-dateutil',
  'pytz',
  'tk'],
 'license': 'PSF-based (http://matplotlib.org/users/license.html)',
 'machine': 'x86_64',
 'md5': 'c261a0140c1ffc169f6b67c78e723b68',
 'name': 'matplotlib',
 'operatingsystem': 'linux',
 'platform': 'linux',
 'requires': [],
 'size': 9806380,
 'subdir': 'linux-64',
 'target-triplet': 'x86_64-any-linux',
 'version': '2.0.0b1'}

Anaconda server is clearly tracking this information somewhere since it shows the upload time for all packages that have been uploaded. Do we know if there is a programmatic way to access this information?

[ericdill]

I do not think we should do this. Not all packages follow a particular versioning scheme and I think conda-forge should be as version scheme agnostic as possible. I'd rather see uploading to a testing channel be purely opt-in.

Having let this sit for far too long, I am in complete agreement with you @jjhelmus . It seems like it would be a tremendous amount of work to have some sort of mechanism in place to do any sort of reasonable version-scheme validation of the package you're trying to upload and also compare it to the versions that already exist on whatever label you're trying to stick onto your conda package. And after all that work, I do not forsee tangible benefits to such a system that one or two iterations with the package maintainers would not solve.

[ericdill]

@conda-forge/core could you all give this another read through as time permits? Sorry to let this sit idle for many moons.

Also cc @tacaswell as you expressed interest in ironing out pre-releases

[tacaswell]

I am not sure of the value of dev artifact on conda-forge. @pelson made a pretty good argument about not dev frequently (that you will end up with so many 'release' if you go to one per commit, that it will become unsustainable both from a disk-space point of view and keeping-track-of-the-things point of view.

I am also not sure how I feel about the aging out policy, if dev versions are dropped re-thinking that might make sense as well.

[jakirkham]

So PR ( #6 ) ended up being called CFEP-02, which we missed. Could we renumber this somehow? It looks like 05 should be ok.

[jakirkham]

Also sorry if I overlooked this, but did we discuss having dev packages that require other dev packages? How would we like to handle those?

[jorisvandenbossche]

What is the status on this? Does this need just a decision, or further discussion?
Anyway I can help?

A pandas rc release is coming up, so it would be great to be able to use conda-forge to build the release candidate packages.

[jakirkham]

It needs some tidying as noted above to fix the conflicts.

I guess there is still some discussion on the aging of dev releases. Maybe that could be saved for a CFEP on our retention policy, which we already need anyways.

[CJ-Wright]

@ericdill since you opened this PR you can't approve it via the review. Are you on board with this?

[jasongrout]

This vote will end on Jan 22nd (2 weeks).

This is just a ping that the vote ends tomorrow. As a person in the community, I'm happy that this proposal is under consideration.

[CJ-Wright]

This vote passes!

[CJ-Wright]

Suggested change

	<tr><td> Status </td><td> Proposed </td></tr>
	<tr><td> Status </td><td> Accepted </td></tr>

[CJ-Wright]

@ericdill if you could please merge my change to mark this as accepted I'll merge this.

[scopatz]

Let's fix in repo

[jasongrout]

In #3 (comment), @CJ-Wright indicated that the CFEP would be updated to the naming convention .../conda-forge/label/matplotlib_rc, but it seems the text was never updated, so what was merged was the label convention rc_matplotlib. Was this intentional? (I don't know if there were other conversations off this thread that decided the rc_package convention instead of package_rc.)

[CJ-Wright]

Sorry my fault, I'll fix that in a follow up pr

[jasongrout]

Sorry my fault, I'll fix that in a follow up pr

I opened #21 to update the CFEP 5 text. I'm not sure exactly the right procedure for doing this, so please let me know if something is amiss.