ENH: Rework conda mirror to not use conda-index #14
Conversation
Current coverage is 92.77% (diff: 89.65%)@@ master #14 diff @@
==========================================
Files 2 2
Lines 169 180 +11
Methods 0 0
Messages 0 0
Branches 0 0
==========================================
+ Hits 154 167 +13
+ Misses 15 13 -2
Partials 0 0
|
|
what's left: figure out how to make validation work |
| try: | ||
| info = repodata[package] | ||
| except KeyError: | ||
| logging.info("%s is not in the upstream index. Removing..." |
There was a problem hiding this comment.
Nit: should this be a warn? Feels like an possible important indicator that something might be amiss if we have a package that upstream never had or not longer has.
| except tarfile.TarError: | ||
| logging.debug("tarfile error encountered. Original error below.") | ||
| logging.debug(pformat(traceback.format_exc())) | ||
| logging.info("Removing package: %s", filename) |
There was a problem hiding this comment.
Nit: Maybe use warn level? A corrupted tarball might indicate a bigger problem we need to investigate.
There was a problem hiding this comment.
corrupted tarball is more frequently a "connection interrupted" issue during download than anything else
| pformat(os.listdir(download_dir))) | ||
| for f in os.listdir(download_dir): | ||
|
|
||
| if f.endswith('json') or f.endswith('.bz2'): |
There was a problem hiding this comment.
Maybe I'm wrong, but I expect most changes to be addition of new packages rather than updates. In that case, it seems that moving all of the packages into place and then the JSON file would reduce the window of time when a user might do a conda install foo and get an error about a missing package because it hasn't been copied into place yet.
|
Thanks @parente |
No description provided.