Pkg cannot recover from corrupted METADATA

[mikewl]

The issue name is not exactly correct but difficult to condense into a single line.

This occurred with Julia 0.3.5 on Win x64

If you install Julia on windows without having git installed, Pkg.status() will run and silently fail.
When running Pkg.add() then git issues will pop up, which boils down to that git needed to be installed.
After installing git however the metadata has not been populated correctly and is not corrected by re-running Pkg.status().

At this point running Pkg.status() just returns No packages installed. Deleting the .Julia folder fixes all the issues and packages can be added again.
#7584 would fix this as git would not be required to be installed.

[tkelman]

Where did you install Julia from? A source build or a binary? The binaries have Git bundled and should never have this issue. A source build on Windows can't do anything Pkg-related because of this, there are hacky solutions but I was hoping #7584 would've been done by now or I would've improved the build system a little.

[mikewl]

I downloaded a binary.

It was on a fresh install and for once Julia was the 1st thing I installed so I just assumed it was git missing that was the cause!

Installing git and putting it on the path did fix this though.

[tkelman]

No, check the binary under /Git, it's all there. More likely you might need to run Pkg.init before Pkg.status can work?

You shouldn't need a separate git installation if you're downloading a binary.

[mikewl]

It is all there. Will try and reproduce issue without git installed and see what happens.

Will uninstall git now and delete the .Julia folder and see if it was not caused by running Pkg.status() before adding a package or running Pkg.init()

[mikewl]

Ha! Found the cause.

When the internet cuts out (which is most likely what happened here due to a crappy connection) or the julia REPL is closed midway through fetching METADATA then the metadata is corrupted and that causes the issue.

[tkelman]

Aha, that makes a lot more sense. What kind of problems was a corrupted partial-download of METADATA giving? I wonder whether there's any reasonable way of checking integrity here.

[mikewl]

I am considering (need to setup a build system 1st) to submit a pull request which just renames the METADATA folder METADATA_TEMP and then renames it after the operation is completed.

This should not cause any other issues (I hope) as only Dir.init() should need to be modified for this change.

[mikewl]

It depends on how far the download gets.

julia> Pkg.add("Jewel")
ERROR: could not spawn setenv(`git rev-parse --git-dir`; dir="C:\\Users\\Michael\\.julia\\v0.3\\METADATA"): no such file
 or directory (ENOENT)
 in _jl_spawn at process.jl:217 (repeats 2 times)

Is given when a very early termination occurs.
When more time is given:

julia> Pkg.add("Jewel")
fatal: ambiguous argument 'HEAD': unknown revision or path not in the working tree.
Use '--' to separate paths from revisions, like this:
'git <command> [<revision>...] -- [<file>...]'
ERROR: failed process: Process(`git '--work-tree=C:\Users\Michael\.julia\v0.3\METADATA' '--git-dir=C:\Users\Michael\.jul
ia\v0.3\METADATA\.git' rev-parse --symbolic-full-name --abbrev-ref HEAD`, ProcessExited(128)) [128]
 in readbytes at process.jl:462
 in readchomp at pkg/git.jl:24
 in branch at pkg/git.jl:56
 in add at pkg/entry.jl:51
 in add at pkg/entry.jl:71
 in anonymous at pkg/dir.jl:28
 in cd at file.jl:30
 in cd at pkg/dir.jl:28
 in add at pkg.jl:20

Is given instead, this is the error I had initially.

Interestingly you can't reinit or run update or run add once a corrupted METADATA exists

[emptist]

I encoutered into this too with version 0.3.7 binary on windows7 after a fresh intall. My wifi is not good anyway.

[emptist]

               _
   _       _ _(_)_     |  A fresh approach to technical computing
  (_)     | (_) (_)    |  Documentation: http://docs.julialang.org
   _ _   _| |_  __ _   |  Type "help()" for help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 0.3.7 (2015-03-23 21:36 UTC)
 _/ |\__'_|_|_|\__'_|  |  Official http://julialang.org/ release
|__/                   |  i686-w64-mingw32

julia> Pkg
Base.Pkg

julia> Pkg.add("Jewel")
fatal: ambiguous argument 'HEAD': unknown revision or path not in the working tr
ee.
Use '--' to separate paths from revisions, like this:
'git <command> [<revision>...] -- [<file>...]'
ERROR: failed process: Process(`git '--work-tree=C:\Users\Administrator\.julia\v
0.3\METADATA' '--git-dir=C:\Users\Administrator\.julia\v0.3\METADATA\.git' rev-p
arse --symbolic-full-name --abbrev-ref HEAD`, ProcessExited(128)) [128]
 in error at error.jl:22
 in branch at pkg/git.jl:56
 in add at pkg/entry.jl:51
 in anonymous at pkg/dir.jl:28
 in cd at file.jl:30
 in cd at pkg/dir.jl:28

[tkelman]

Until we figure a way to make git-related actions more robust, the best thing to do would be to manually delete C:\Users\Administrator\.julia\v0.3\METADATA (or, if things go really wrong, all of C:\Users\Administrator\.julia\v0.3) and try Pkg.init() again.