dh-make-golang no longer able to calculate dependencies of package with vendoring (as of golang-1.6.2?)

[anthonyfok]

I discovered yesterday that the command:

dh-make-golang github.com/kardianos/govendor

is no longer able to calculate the necessary dependent Go packages in debian/control. The govender repository contains vendoring in the vendor/ directory.

But I remember that it worked back on 2016-05-01 (a little more than a month ago) when I created the initial package for govendor, and dh-make-golang has not changed since. Could it have anything to do with recent changes in golang-go packaging, especially the new go1.6.2?

(I have yet to investigate in depth, but filing an issue so I don't forget.)

[stapelberg]

I think the relevant change might be:

Go 1.5 introduced experimental support for vendoring, enabled by setting the GO15VENDOREXPERIMENT environment variable to 1. Go 1.6 keeps the vendoring support, no longer considered experimental, and enables it by default. It can be disabled explicitly by setting the GO15VENDOREXPERIMENT environment variable to 0. Go 1.7 will remove support for the environment variable.

(from https://tip.golang.org/doc/go1.6#go_command)

[anthonyfok]

Thank you @stapelberg! Yes, setting GO15VENDOREXPERIMENT=0 when calling dh-make-golang would do the trick for the time being, until Go 1.7 I suppose.

I took a look at the dh-make-golang, and Ithink I will make a pull request to make.go in the near future.

[Baisang]

How does this work now in Go 1.7? It seems my debian/control file doesn't have the dependencies.

I'm new to building packages in general, so would appreciate some help.

[stapelberg]

@Baisang Can you file a separate issue please with the command you’re running, the version of dh-make-golang you’re using and the output you’re seeing? In general, the program works.

[stapelberg]

Had a chance to reproduce this issue using @Baisang’s example invocation dh-make-golang github.com/quilt/quilt.

The main problem is

dh-make-golang/make.go

Line 246 in be72733

rr, err := vcs.RepoRootForImportPath(importpath, false)

, where we discover the VCS root for the imported package. That line resolves e.g. golang.org/x/tools/go/vcs to golang.org/x/tools, which we then convert into a Debian package name.

When using vendoring, the VCS information needs to be determined differently. In the particular case of github.com/quilt/quilt, there is a vendor/vendor.json file. Unfortunately, it only contains paths; the repo root is lost.

I’m not sure how other vendoring tools deal with this and whether there’s a spec — any pointers welcome. I’ll file an issue with github.com/golang/dep to make sure our requirements are known to the hopefully-soon canonical vendoring tool.

For the moment, one either needs to locate all dependencies manually, or un-vendor the package (possibly in a private fork on GitHub) before packaging.

[olasd]

Hey, I've been bitten by this because I need to package https://github.com/mheese/journalbeat for work, and that has vendored all its dependencies.

What I've done is just nuking the vendor directory, and reinstalling the dependencies with go get -t -d ./.... It's a bit drastic but it seems to work ok. :)

https://github.com/olasd/dh-make-golang/tree/bugfix/vendoring has my work, feel free to pull from it if the approach makes sense.

[stapelberg]

@olasd Thanks, I like that solution. It should fix the issue independent of the vendoring tool used. Pulled (and cleaned up a little).

[Baisang]

One last thing I would like to point out for any future readers - you will have to correct for dependencies that have different names/that are sub-packages, e.g. I had imported github.com/coreos/etcd/client, but the package to install is golang-etcd-server-dev (only in unstable as of writing). Additionally, there may be some dependencies that don't have a package available online (e.g. github.com/docker/go-plugins-helpers).

What I ended up doing was just requiring golang-any and have my debian/rules just do go install, as it'll use the vendor/ subdirectory when compiling.

[stapelberg]

@Baisang Note that a package including the vendor/ subdirectory is not suitable for inclusion in Debian proper, i.e. will only be useful in private repositories.

[Baisang]

@stapelberg if there is a project that uses a dependency that isn't available in "apt"/doesn't have a corresponding "debian name" (for lack of a better term, sorry if I am mixing up my terminology here), like the github.com/docker/go-plugins-helpers dependency, how can the project be built so it would be suitable?

Sorry, I feel like there is a key point I am not understanding here. I haven't been able to find much literature on what I'm trying to do (building a debian package for golang project that uses vendoring), so I really appreciate your help thus far!

[stapelberg]

The dependencies need to be packaged first.