Failed to update golang.org/x/tools/cmd/goimports by go datasource

[suzuki-shunsuke]

https://app.renovatebot.com/dashboard#github/suzuki-shunsuke/test-renovate/762018971

DEBUG: Found no results from datasource that look like a version (golang.org/x/tools/cmd/goimports)(dependency="golang.org/x/tools/cmd/goimports")
{
  "result": {
    "sourceUrl": "https://github.com/golang/tools",
    "releases": []
  }
}

https://app.renovatebot.com/dashboard#github/suzuki-shunsuke/test-renovate/762018971

renovate.json

{
  "extends": [
    "config:base",
  ],
  "prHourlyLimit": 0,
  "prConcurrentLimit": 0,
  "branchConcurrentLimit": 0,
  "regexManagers": [
    {
      "fileMatch": ["\\.?aqua\\.ya?ml"],
      "matchStrings": [
        " +['\"]?(version|ref)['\"]? *: +['\"]?(?<currentValue>[^'\" \\n]+?)['\"]? +# renovate: depName=(?<depName>[^\\n]+)",
        " +['\"]?name['\"]? *: +['\"]?(?<depName>[^\\n]+\\.[^\\n]+)*@(?<currentValue>[^'\" \\n]+)['\"]?"
      ],
      "datasourceTemplate": "go"
    }
  ]
}

registries:
- type: standard
  ref: v3.21.0
packages:
- name: golang.org/x/tools/cmd/[email protected]

[suzuki-shunsuke]

https://github.com/renovatebot/renovate/blob/6b584c27938911a683df480cac8ffe91dbf6e46d/lib/workers/repository/process/lookup/index.ts#L112-L118

[frezbo]

Okay I seem to have found the issue, renovate uses this url to list for version updates:

https://proxy.golang.org/<name defined in the registry yaml>/@v/list

So in case of golang.org/x/tools/gopls it would be:

❯ curl -L "https://proxy.golang.org/golang.org/x/tools/gopls/@v/list"
v0.6.5-pre.1
v0.9.0-pre.3
v0.5.2-pre.2
v0.8.2-pre.1
v0.8.1-pre.2
v0.7.4-pre.1
v0.1.4

but it seems that for goimports, the proxy.golang.org doesn't return anything:

❯ curl -L "https://proxy.golang.org/golang.org/x/tools/goimports/@v/list"
not found: module golang.org/x/tools/goimports: no matching versions for query "latest"⏎

but if golang.org/x/tools/cmd/goimports is replaced to use the github repo for golang/tools which contain goimports cli and dropping the cmd/goimports , we get results:

 curl -L "https://proxy.golang.org/github.com/golang/tools/@v/list"
v0.1.4
v0.1.12
v0.1.3
v0.1.9
v0.1.7
v0.1.10
v0.1.6
v0.1.2
v0.1.11
v0.1.8
v0.1.0
v0.1.5
v0.1.1

i think this has to do with how a single repo hosts multiple cli and how proxy.golang.org finds package version. I'll try to figure out how it works internally and maybe propose some fix

[frezbo]

i think this has to do with how a single repo hosts multiple cli and how proxy.golang.org finds package version. I'll try to figure out how it works internally and maybe propose some fix

okay, i found the issue:

If we look at the gitrepo, it has two go modules:

• https://github.com/golang/tools/blob/master/go.mod#L1 for just golang.org/x/tools which hosts all the commands seen here: https://github.com/golang/tools/tree/master/cmd
• https://github.com/golang/tools/blob/master/gopls/go.mod#L1 which is specific for gopls

so in order to retrieve the gopls version we should use:

❯ curl -L "https://proxy.golang.org/golang.org/x/tools/gopls/@v/list"
v0.6.5-pre.1
v0.9.0-pre.3
v0.5.2-pre.2
v0.8.2-pre.1
v0.8.1-pre.2
v0.7.4-pre.1
v0.1.4

and in-order to retrieve the version of any command under cmd folder, in this case goimports we should use:

❯ curl -L "https://proxy.golang.org/golang.org/x/tools/@v/list"
v0.1.4
v0.1.12
v0.1.3
v0.1.9
v0.1.7
v0.1.10
v0.1.6
v0.1.2
v0.1.11
v0.1.8
v0.1.0
v0.1.5
v0.1.1

so basically the module name defined in go.mod

In aqua we can fix it by using something like this:

 - type: go_install
    name: golang.org/x/tools
    path: golang.org/x/tools/cmd/goimports
    repo_owner: golang
    repo_name: tools
    version_source: github_tag
    description: updates your Go import lines, adding missing ones and removing unreferenced ones
    link: https://pkg.go.dev/golang.org/x/tools/cmd/goimports
    files:
      - name: goimports

but this would mean we lose the ability to tell the package name is goimports, i think we need a way to define custom name for different cmd's provided by a single go module for go_install

[suzuki-shunsuke]

Thank you for your investigation.

I have two ideas to solve the issue.

1. Use github_tags datasource
2. Use go datasource

I think 1 is better.

1. Use github_tags datasource

registry.yaml

packages:
  - type: go_install
    name: golang/tools/goimports
    aliases: # Set aliases to keep the compatibility
      - name: golang.org/x/tools/cmd/goimports
    path: golang.org/x/tools/cmd/goimports
    repo_owner: golang
    repo_name: tools
    version_source: github_tag
    description: updates your Go import lines, adding missing ones and removing unreferenced ones
    link: https://pkg.go.dev/golang.org/x/tools/cmd/goimports
    files:
      - name: goimports

aqua.yaml

packages:
  - name: golang/tools/[email protected]

2. Use go datasource

aqua.yaml

packages:
  - name: golang.org/x/tools/cmd/goimports
    version: v0.1.4 # renovate: depName=golang.org/x/tools

[frezbo]

Yes, I also like the option 1 since the user doesn't need to set a custom renovate comment. I'll create a PR to address this

[frezbo]

hmm I see a problem, since https://github.com/aquaproj/aqua-renovate-config/blob/main/default.json#L9 is only for github releases, renovate would fail to find the github tags I assume (I could be wrong here), also golang/tools have different github releases and github tagging schema.

[suzuki-shunsuke]

How about this?

#207

[suzuki-shunsuke]

Solved. #207

[suzuki-shunsuke]

#211
#207 (comment)