Feat: Log enhancement for command generate and update #2461

dreamjz · 2023-11-13T20:20:06Z

Solve issue mentioned in #2460

Description

Log processing time for GitHub API
Log GitHub API Rate Limit info after retrieving the version.
Log errors when command encounters err.

Test cases

generate

# get latest version
$ aqua g cli/cli
INFO[0003] GitHub API rate limit info                    aqua_version= env=linux/amd64 gh_rate_limit=60 gh_rate_remaining=59 program=aqua repo=cli/cli
INFO[0003] Retrieve pkg version(s) in 3.734309466s       aqua_version= env=linux/amd64 program=aqua repo=cli/cli
- name: cli/[email protected]

# get multiple versions
$ aqua g -s cli/cli
INFO[0007] Retrieve pkg version(s) in 5.211403407s       aqua_version= env=linux/amd64 gh_rate_limit=60 gh_rate_remaining=58 program=aqua repo=cli/cli
- name: cli/[email protected]

$ aqua g -s -l -1 cli/cli
INFO[0011] Retrieve pkg version(s) in 5.069310084s       aqua_version= env=linux/amd64 gh_rate_limit=60 gh_rate_remaining=56 program=aqua repo=cli/cli
- name: cli/[email protected]

# Reach the Rate Limit
$ INFO[0000] GitHub API rate limit info                    aqua_version= env=linux/amd64 gh_rate_limit=60 gh_rate_remaining=0 program=aqua repo=cli/cli
INFO[0000] Retrieve pkg version(s) in 539.007192ms       aqua_version= env=linux/amd64 program=aqua repo=cli/cli
WARN[0000] Ver retrieving err                            aqua_version= env=linux/amd64 error="get the latest GitHub Release: GET https://api.github.com/repos/cli/cli/releases/latest: 403 API rate limit exceeded for xxx. (But here's the good news: Authenticated requests get a higher rate limit. Check out the documentation for more details.) [rate reset in 49m54s]" program=aqua repo=cli/cli
- name: cli/cli
  version: '[SET PACKAGE VERSION]'

update

$ aqua up pryrite 
INFO[0000] GitHub API rate limit info                    aqua_version= env=linux/amd64 gh_rate_limit=60 gh_rate_remaining=57 program=aqua repo=1xyz/pryrite
INFO[0000] Retrieve pkg version(s) in 908.514298ms       aqua_version= env=linux/amd64 program=aqua repo=1xyz/pryrite

$ aqua up -s pryrite
INFO[0002] Retrieve pkg version(s) in 803.870181ms       aqua_version= env=linux/amd64 gh_rate_limit=60 gh_rate_remaining=56 program=aqua repo=1xyz/pryrite

$ aqua up -s -l -1 pryrite
INFO[0002] Retrieve pkg version(s) in 899.726603ms       aqua_version= env=linux/amd64 gh_rate_limit=60 gh_rate_remaining=55 program=aqua repo=1xyz/pryrite


# Reach the Rate Limit
$ aqua up pryrite  
INFO[0000] GitHub API rate limit info                    aqua_version= env=linux/amd64 gh_rate_limit=60 gh_rate_remaining=0 program=aqua repo=1xyz/pryrite
INFO[0000] Retrieve pkg version(s) in 453.074916ms       aqua_version= env=linux/amd64 program=aqua repo=1xyz/pryrite
WARN[0000] Version retrieving err                        aqua_version= env=linux/amd64 error="get the latest GitHub Release: GET https://api.github.com/repos/1xyz/pryrite/releases/latest: 403 API rate limit exceeded for xxx. (But here's the good news: Authenticated requests get a higher rate limit. Check out the documentation for more details.) [rate reset in 9m47s]" program=aqua repo=1xyz/pryrit

suzuki-shunsuke · 2023-11-13T22:13:00Z

Thank you!

suzuki-shunsuke · 2023-11-13T22:16:44Z

pkg/versiongetter/cargo.go

@@ -23,11 +24,11 @@ func NewCargo(client CargoClient) *CargoVersionGetter {
 	}
 }

-func (c *CargoVersionGetter) Get(ctx context.Context, pkg *registry.PackageInfo, filters []*Filter) (string, error) {
+func (c *CargoVersionGetter) Get(ctx context.Context, _ *logrus.Entry, pkg *registry.PackageInfo, filters []*Filter) (string, error) {


This is unused, so we don't need to add this at the moment.

Suggested change

func (c *CargoVersionGetter) Get(ctx context.Context, _ *logrus.Entry, pkg *registry.PackageInfo, filters []*Filter) (string, error) {

func (c *CargoVersionGetter) Get(ctx context.Context, pkg *registry.PackageInfo, filters []*Filter) (string, error) {

There are five classes/structs that implement the interface VersionGetter:

GeneralVersionGetter

CargoVersionGetter

GitHubReleaseVersionGetter

GitHubTagVersionGetter

MockVersionGetter

Since I’ve modified the VersionGetter. Removing the _ *logrus.Entry, will result in a compile error in GeneralVersionGetter.get.

After considering the feature of log enhancement, I think there are two options:

Keep it simple. Do not modify the interface VersionGetter, log the ERROR in ‎FuzzyGetter.Get‎ only.

Modify the interface VersionGetter and log the GH API Rate Limit info as DEBUG level. The CargoVersionGetter has to be modified.

What do you think about it? If you have a better approach, please tell me.

Since I’ve modified the VersionGetter. Removing the _ *logrus.Entry, will result in a compile error in GeneralVersionGetter.get.

Oh, I see. Thank you for your explanation.

Keep it simple. Do not modify the interface VersionGetter, log the ERROR in ‎FuzzyGetter.Get‎ only.

I disagree with this option. We usually pass *logrus.Entry as an argument to take over the context (log's structured data).

Modify the interface VersionGetter and log the GH API Rate Limit info as DEBUG level. The CargoVersionGetter has to be modified.

The log about GitHub API rate limit should be outputted only when GitHub API rate limit error occurs.

https://pkg.go.dev/github.com/google/go-github/github#hdr-Rate_Limiting

The log about GitHub API rate limit should be outputted only when GitHub API rate limit error occurs.

https://pkg.go.dev/github.com/google/go-github/github#hdr-Rate_Limiting

Ah, sorry. Let me correct.
Debug log is fine.

pkg/versiongetter/cargo.go

pkg/versiongetter/github_release.go

suzuki-shunsuke · 2023-11-14T00:56:34Z

INFO[0011] Retrieve pkg version(s) in 5.069310084s

This log also should not be outputted always.

suzuki-shunsuke · 2023-11-14T12:42:22Z

pkg/versiongetter/general.go

+}
+
+// fuzzy-finder's output will overwrite the log, add fields to original logE
+func addRteLimitInfo(logE *logrus.Entry, resp *github.Response) {


Typo?

Suggested change

func addRteLimitInfo(logE *logrus.Entry, resp *github.Response) {

func addRateLimitInfo(logE *logrus.Entry, resp *github.Response) {

suzuki-shunsuke · 2023-11-14T12:45:22Z

pkg/versiongetter/general.go

+func addRteLimitInfo(logE *logrus.Entry, resp *github.Response) {
+	*logE = *logE.WithFields(logrus.Fields{
+		"gh_rate_limit":     resp.Rate.Limit,
+		"gh_rate_remaining": resp.Rate.Remaining,
+	})
+}


Suggested change

func addRteLimitInfo(logE *logrus.Entry, resp *github.Response) {

*logE = *logE.WithFields(logrus.Fields{

"gh_rate_limit": resp.Rate.Limit,

"gh_rate_remaining": resp.Rate.Remaining,

})

}

func addRateLimitInfo(logE *logrus.Entry, resp *github.Response) *logrus.Entry {

return logE.WithFields(logrus.Fields{

"gh_rate_limit": resp.Rate.Limit,

"gh_rate_remaining": resp.Rate.Remaining,

})

}

I think this is more natural.

logE = addRateLimitInfo(logE, resp)

logE = addRateLimitInfo(logE, resp)

This statement generate a derived logE and assign its pointer to a local variable.

This will not change its orginal value.

see this example in playground

*logE = *addRateLimitInfo(logE, resp)

What do you think of editing it like this?

You don't need to change the original value.
You just create a new *logrus.Entry and use it.
I think this is the design of logrus.

The reason why I want to change its original value is the fuzzy-finder's output will overwirte the log output.

See this code

List is invoked before openning fuzzy-finder, if log something in List , it will be overwritten by fuzzy-finder.

List -> print log -> open fuzzy-finder -> print content (overwrite the log)

I record this to help you to understand what I'm describing.

Thank you for your explanation. I understood the issue, but I don't want to change *logrus.Entry.
This is tricky.
I don't think the information about API rate limit is so important that we need to output it with the tricky way unless API rate limit error occurs.

…p_cmd-log-enhancement

dreamjz · 2023-11-16T14:32:21Z

Description

Fix typo, rename addRteLimitInfo to addRateLimitInfo
Check nil for resp to avoid panic. (When a network error occurs, resp will be nil)
For Get version, log the GH Rate Limit and processing time at DEBUG level
For List versions, log the GH Rate Limit at ERROR level only when the error occurs.

One more question

I don't think the information about API rate limit is so important that we need to output it with the tricky way unless API rate limit error occurs.

May I ask why this tricky approach is not recommended? Are there any disadvantages to it?

Test cases

generate

# DEFAULT LEVEL
# Releases
$ aqua g cli/cli
- name: cli/[email protected]
$ aqua g -s cli/cli
- name: cli/[email protected]
$ aqua g -s -l -1 cli/cli
- name: cli/[email protected]

# Rate Limit
$ aqua g cli/cli
WARN[0000] Version retrieving error                      aqua_version= env=linux/amd64 error="get the latest GitHub Release: GET https://api.github.com/repos/cli/cli/releases/latest: 403 API rate limit exceeded for xxx. (But here's the good news: Authenticated requests get a higher rate limit. Check out the documentation for more details.) [rate reset in 2m28s]" program=aqua repo=cli/cli
- name: cli/cli
  version: '[SET PACKAGE VERSION]'

# Network error
$ aqua g cli/cli
WARN[0000] Version retrieving error                      aqua_version= env=linux/amd64 error="get the latest GitHub Release: Get \"https://api.github.com/repos/cli/cli/releases/latest\": proxyconnect tcp: dial tcp xxx:8888: connect: connection refused" program=aqua repo=cli/cli
- name: cli/cli
  version: '[SET PACKAGE VERSION]'

$ aqua g -s cli/cli
WARN[0000] Version retrieving error                      aqua_version= env=linux/amd64 error="list tags: GET https://api.github.com/repos/cli/cli/releases?per_page=100: 403 API rate limit exceeded for xxx. (But here's the good news: Authenticated requests get a higher rate limit. Check out the documentation for more details.) [rate reset in 1m09s]" gh_rate_limit=60 gh_rate_remaining=0 program=aqua repo=cli/cli
- name: cli/cli
  version: '[SET PACKAGE VERSION]'

# Tags
$ aqua g golang/go
- name: golang/[email protected]
$ aqua g -s golang/go
- name: golang/[email protected]
$ aqua g -s -l -1 golang/go
- name: golang/[email protected]

# Rate Limit
$ aqua g golang/go
WARN[0000] Version retrieving error                      aqua_version= env=linux/amd64 error="list tags: GET https://api.github.com/repos/golang/go/tags?per_page=100: 403 API rate limit exceeded for xxx. (But here's the good news: Authenticated requests get a higher rate limit. Check out the documentation for more details.) [rate reset in 40s]" gh_rate_limit=60 gh_rate_remaining=0 program=aqua repo=golang/go
WARN[0000] same package already exists                   aqua_version= env=linux/amd64 package_name=golang/go package_registry=standard program=aqua
- name: golang/go
  version: '[SET PACKAGE VERSION]'

# Network error
$ aqua g golang/go
WARN[0000] Version retrieving error                      aqua_version= env=linux/amd64 error="list tags: Get \"https://api.github.com/repos/golang/go/tags?per_page=30\": proxyconnect tcp: dial tcp xxx:8888: connect: connection refused" program=aqua repo=golang/go
WARN[0000] same package already exists                   aqua_version= env=linux/amd64 package_name=golang/go package_registry=standard program=aqua
- name: golang/go
  version: '[SET PACKAGE VERSION]'

# DEBUG LEVEL
export AQUA_LOG_LEVEL=DEBUG
# Releases
$ aqua g cli/cli
DEBU[0000] GitHub API Rate Limit info                    aqua_version= env=windows/amd64 gh_rate_limit=60 gh_rate_remaining=40 program=aqua repo=cli/cli
DEBU[0000] Retrieve pkg version(s) in 863.4326ms         aqua_version= env=windows/amd64 program=aqua repo=cli/cli
- name: cli/[email protected]

$ aqua g -s cli/cli
DEBU[0004] Retrieve pkg version(s) in 3.7289863s         aqua_version= env=windows/amd64 program=aqua repo=cli/cli
- name: cli/[email protected]

$ aqua g -s -l -1 cli/cli
DEBU[0005] Retrieve pkg version(s) in 4.0286107s         aqua_version= env=windows/amd64 program=aqua repo=cli/cli
- name: cli/[email protected]

# Rate Limit
$ aqua g cli/cli
DEBU[0000] GitHub API Rate Limit info                    aqua_version= env=linux/amd64 gh_rate_limit=60 gh_rate_remaining=0 program=aqua repo=cli/cli
DEBU[0000] Retrieve pkg version(s) in 469.521004ms       aqua_version= env=linux/amd64 program=aqua repo=cli/cli
WARN[0000] Version retrieving error                      aqua_version= env=linux/amd64 error="get the latest GitHub Release: GET https://api.github.com/repos/cli/cli/releases/latest: 403 API rate limit exceeded for xxx. (But here's the good news: Authenticated requests get a higher rate limit. Check out the documentation for more details.) [rate reset in 23m57s]" program=aqua repo=cli/cli
- name: cli/cli
  version: '[SET PACKAGE VERSION]'
  
# Tags
$ aqua g golang/go
DEBU[0001] GitHub API Rate Limit info                    aqua_version= env=windows/amd64 gh_rate_limit=60 gh_rate_remaining=33 program=aqua repo=golang/go
DEBU[0001] Retrieve pkg version(s) in 1.8703541s         aqua_version= env=windows/amd64 program=aqua repo=golang/go
- name: golang/[email protected]

$ aqua g -s golang/go
DEBU[0002] Retrieve pkg version(s) in 1.1045973s         aqua_version= env=windows/amd64 program=aqua repo=golang/go
- name: golang/[email protected]

$ aqua g -s -l -1 golang/go
DEBU[0002] Retrieve pkg version(s) in 1.6950486s         aqua_version= env=windows/amd64 program=aqua repo=golang/go
- name: golang/[email protected]

# Rate limit
$ aqua g golang/go
DEBU[0000] GitHub API Rate Limit info                    aqua_version= env=linux/amd64 gh_rate_limit=60 gh_rate_remaining=0 program=aqua repo=golang/go
DEBU[0000] Retrieve pkg version(s) in 562.864474ms       aqua_version= env=linux/amd64 program=aqua repo=golang/go
WARN[0000] Version retrieving error                      aqua_version= env=linux/amd64 error="list tags: GET https://api.github.com/repos/golang/go/tags?per_page=30: 403 API rate limit exceeded for xxx. (But here's the good news: Authenticated requests get a higher rate limit. Check out the documentation for more details.) [rate reset in 22m32s]" program=aqua repo=golang/go
WARN[0000] same package already exists                   aqua_version= env=linux/amd64 package_name=golang/go package_registry=standard program=aqua
- name: golang/go
  version: '[SET PACKAGE VERSION]'

update

# DEFAULT LEVEL
# Releases
$ aqua up pryrite

$ aqua up -s pryrite

$ aqua up -s -l -1 pryrite

# Network error
$ aqua up pryrite
WARN[0000] Version retrieving error                      aqua_version= env=linux/amd64 error="get the latest GitHub Release: Get \"https://api.github.com/repos/1xyz/pryrite/releases/latest\": proxyconnect tcp: dial tcp xxx:8888: connect: connection refused" program=aqua repo=1xyz/pryrite

# Rate Limit
$ aqua up pryrite
WARN[0000] Version retrieving error                      aqua_version= env=linux/amd64 error="get the latest GitHub Release: GET https://api.github.com/repos/1xyz/pryrite/releases/latest: 403 API rate limit exceeded for xxx. (But here's the good news: Authenticated requests get a higher rate limit. Check out the documentation for more details.) [rate reset in 13m45s]" program=aqua repo=1xyz/pryrite

# Tags
$ aqua up go

$ aqua up -s go

$ aqua up -s -l -1 go

# Network error
$ aqua up go
WARN[0000] Version retrieving error                      aqua_version= env=linux/amd64 error="list tags: Get \"https://api.github.com/repos/golang/go/tags?per_page=30\": proxyconnect tcp: dial tcp xxx:8888: connect: connection refused" program=aqua repo=golang/go

# Rate Limit
$ aqua up go
WARN[0000] Version retrieving error                      aqua_version= env=linux/amd64 error="list tags: GET https://api.github.com/repos/golang/go/tags?per_page=30: 403 API rate limit exceeded for xxx. (But here's the good news: Authenticated requests get a higher rate limit. Check out the documentation for more details.) [rate reset in 15m16s]" program=aqua repo=golang/go


# DEBUG LEVEL
export AQUA_LOG_LEVEL=DEBUG
# Releases
$ aqua up pryrite
DEBU[0000] match the version_constraint                  aqua_version= env=linux/amd64 package_name=1xyz/pryrite package_semver=0.10.20 package_version=0.10.20 program=aqua registry_name=standard version_constraint=true
DEBU[0000] GitHub API Rate Limit info                    aqua_version= env=linux/amd64 gh_rate_limit=60 gh_rate_remaining=13 program=aqua repo=1xyz/pryrite
DEBU[0000] Retrieve pkg version(s) in 861.525691ms       aqua_version= env=linux/amd64 program=aqua repo=1xyz/pryrite
DEBU[0000] already latest                                aqua_version= env=linux/amd64 program=aqua
DEBU[0000] version isn't found                           aqua_version= env=linux/amd64 program=aqua

$ aqua up -s pryrite
DEBU[0000] match the version_constraint                  aqua_version= env=linux/amd64 package_name=1xyz/pryrite package_semver=0.10.20 package_version=0.10.20 program=aqua registry_name=standard version_constraint=true
DEBU[0002] Retrieve pkg version(s) in 945.278908ms       aqua_version= env=linux/amd64 program=aqua repo=1xyz/pryrite
DEBU[0002] already latest                                aqua_version= env=linux/amd64 program=aqua
DEBU[0002] version isn't found                           aqua_version= env=linux/amd64 program=aqua

$ aqua up -s -l -1 pryrite
DEBU[0000] match the version_constraint                  aqua_version= env=linux/amd64 package_name=1xyz/pryrite package_semver=0.10.20 package_version=0.10.20 program=aqua registry_name=standard version_constraint=true
DEBU[0001] Retrieve pkg version(s) in 780.912784ms       aqua_version= env=linux/amd64 program=aqua repo=1xyz/pryrite
DEBU[0001] already latest                                aqua_version= env=linux/amd64 program=aqua
DEBU[0001] version isn't found                           aqua_version= env=linux/amd64 program=aqua

# Rate Limit
$ aqua up pryrite
DEBU[0000] match the version_constraint                  aqua_version= env=linux/amd64 package_name=1xyz/pryrite package_semver=0.10.20 package_version=0.10.20 program=aqua registry_name=standard version_constraint=true
DEBU[0000] GitHub API Rate Limit info                    aqua_version= env=linux/amd64 gh_rate_limit=60 gh_rate_remaining=0 program=aqua repo=1xyz/pryrite
DEBU[0000] Retrieve pkg version(s) in 503.917687ms       aqua_version= env=linux/amd64 program=aqua repo=1xyz/pryrite
WARN[0000] Version retrieving error                      aqua_version= env=linux/amd64 error="get the latest GitHub Release: GET https://api.github.com/repos/1xyz/pryrite/releases/latest: 403 API rate limit exceeded for xxx. (But here's the good news: Authenticated requests get a higher rate limit. Check out the documentation for more details.) [rate reset in 16m19s]" program=aqua repo=1xyz/pryrite
DEBU[0000] version isn't found                           aqua_version= env=linux/amd64 program=aqua
DEBU[0000] version isn't found                           aqua_version= env=linux/amd64 program=aqua

# Network error
$ aqua up pryrite
DEBU[0000] match the version_constraint                  aqua_version= env=linux/amd64 package_name=1xyz/pryrite package_semver=0.10.20 package_version=0.10.20 program=aqua registry_name=standard version_constraint=true
DEBU[0000] Retrieve pkg version(s) in 799.3µs            aqua_version= env=linux/amd64 program=aqua repo=1xyz/pryrite
WARN[0000] Version retrieving error                      aqua_version= env=linux/amd64 error="get the latest GitHub Release: Get \"https://api.github.com/repos/1xyz/pryrite/releases/latest\": proxyconnect tcp: dial tcp xxx:8888: connect: connection refused" program=aqua repo=1xyz/pryrite
DEBU[0000] version isn't found                           aqua_version= env=linux/amd64 program=aqua
DEBU[0000] version isn't found                           aqua_version= env=linux/amd64 program=aqua

# Tags
$ aqua up go
DEBU[0000] match the version_constraint                  aqua_version= env=linux/amd64 package_name=1xyz/pryrite package_semver=0.10.20 package_version=0.10.20 program=aqua registry_name=standard version_constraint=true
DEBU[0000] no version_constraint                         aqua_version= env=linux/amd64 package_name=golang/go program=aqua registry_name=standard
DEBU[0002] GitHub API Rate Limit info                    aqua_version= env=linux/amd64 gh_rate_limit=60 gh_rate_remaining=7 program=aqua repo=golang/go
DEBU[0002] Retrieve pkg version(s) in 2.657070993s       aqua_version= env=linux/amd64 program=aqua repo=golang/go
DEBU[0002] version isn't found                           aqua_version= env=linux/amd64 program=aqua
DEBU[0002] already latest                                aqua_version= env=linux/amd64 program=aqua

$ aqua up -s go
DEBU[0000] match the version_constraint                  aqua_version= env=linux/amd64 package_name=1xyz/pryrite package_semver=0.10.20 package_version=0.10.20 program=aqua registry_name=standard version_constraint=true
DEBU[0000] no version_constraint                         aqua_version= env=linux/amd64 package_name=golang/go program=aqua registry_name=standard
DEBU[0002] Retrieve pkg version(s) in 1.109959998s       aqua_version= env=linux/amd64 program=aqua repo=golang/go
DEBU[0002] version isn't found                           aqua_version= env=linux/amd64 program=aqua
DEBU[0002] already latest                                aqua_version= env=linux/amd64 program=aqua

$ aqua up -s -l -1 go
DEBU[0000] match the version_constraint                  aqua_version= env=linux/amd64 package_name=1xyz/pryrite package_semver=0.10.20 package_version=0.10.20 program=aqua registry_name=standard version_constraint=true
DEBU[0000] no version_constraint                         aqua_version= env=linux/amd64 package_name=golang/go program=aqua registry_name=standard
DEBU[0002] Retrieve pkg version(s) in 1.828739s          aqua_version= env=linux/amd64 program=aqua repo=golang/go
DEBU[0002] version isn't found                           aqua_version= env=linux/amd64 program=aqua
DEBU[0002] already latest                                aqua_version= env=linux/amd64 program=aqua

# Rate Limit
$ aqua up go
DEBU[0000] match the version_constraint                  aqua_version= env=linux/amd64 package_name=1xyz/pryrite package_semver=0.10.20 package_version=0.10.20 program=aqua registry_name=standard version_constraint=true
DEBU[0000] no version_constraint                         aqua_version= env=linux/amd64 package_name=golang/go program=aqua registry_name=standard
DEBU[0000] GitHub API Rate Limit info                    aqua_version= env=linux/amd64 gh_rate_limit=60 gh_rate_remaining=0 program=aqua repo=golang/go
DEBU[0000] Retrieve pkg version(s) in 466.859563ms       aqua_version= env=linux/amd64 program=aqua repo=golang/go
WARN[0000] Version retrieving error                      aqua_version= env=linux/amd64 error="list tags: GET https://api.github.com/repos/golang/go/tags?per_page=30: 403 API rate limit exceeded for xxx. (But here's the good news: Authenticated requests get a higher rate limit. Check out the documentation for more details.) [rate reset in 18m53s]" program=aqua repo=golang/go
DEBU[0000] version isn't found                           aqua_version= env=linux/amd64 program=aqua
DEBU[0000] version isn't found                           aqua_version= env=linux/amd64 program=aqua

# Network error
$ aqua up go
DEBU[0000] match the version_constraint                  aqua_version= env=linux/amd64 package_name=1xyz/pryrite package_semver=0.10.20 package_version=0.10.20 program=aqua registry_name=standard version_constraint=true
DEBU[0000] no version_constraint                         aqua_version= env=linux/amd64 package_name=golang/go program=aqua registry_name=standard
DEBU[0000] Retrieve pkg version(s) in 455.2µs            aqua_version= env=linux/amd64 program=aqua repo=golang/go
WARN[0000] Version retrieving error                      aqua_version= env=linux/amd64 error="list tags: Get \"https://api.github.com/repos/golang/go/tags?per_page=30\": proxyconnect tcp: dial tcp xxx:8888: connect: connection refused" program=aqua repo=golang/go
DEBU[0000] version isn't found                           aqua_version= env=linux/amd64 program=aqua
DEBU[0000] version isn't found                           aqua_version= env=linux/amd64 program=aqua

dreamjz · 2023-11-16T14:43:40Z

Rate info in error log:

$ aqua g -s cli/cli                                                                
WARN[0000] Version retrieving error                      aqua_version= env=linux/amd64 error="...403 API rate limit exceeded for xxx. ..." gh_rate_limit=60 gh_rate_remaining=0
 program=aqua repo=cli/cli
- name: cli/cli
  version: '[SET PACKAGE VERSION]'

suzuki-shunsuke · 2023-11-18T05:39:47Z

May I ask why this tricky approach is not recommended? Are there any disadvantages to it?

aqua passes *logrus.Entry in many places but we don't expect the side effect that *logrus.Entry is changed in functions.
The information about API rate limiting is tied to the specific API call and it isn't desirable that the information is outputted with unrelated log.
If we call GitHub API at the caller and output log without overriding fields related to API rate limiting,
the fields are outputted with the log of different API call, which is confusing.

e.g.

g.getter.List(ctx, logE, pkg, filters, limit) // information about api rate limiting are set to logE
// ...
version, err := g.getter.Get(ctx, logE, pkg, filters)
if err != nil {
	logE.WithError(err).Warn(getVerErrWarn) // information set by `g.getter.List` are outputted
}

Anyway, thank you for your contribution!

- #2461 (comment)

dreamjz · 2023-11-18T10:33:16Z

Thank you too, I learned a lot from you and aqua. I like this tool.
BTW, I made this meme and put it to my fork repo for fun. 😊

suzuki-shunsuke · 2023-11-18T10:58:43Z

Oh! Nice meme 🤣

dreamjz added 8 commits November 14, 2023 02:28

feat: log processing time

a9c0b33

feat: log gh rate limit for listing versions

fc87715

feat: log gh rate limit for getting latest version

3e6207e

Reduce duplicate codes

89d1a08

Using defer to avoid duplicate invocation

4d12d89

Edit test

43dc80e

Edit test

7f5557e

Merge branch 'aquaproj:main' into feat-g_up_cmd-log-enhancement

9bdd4af

suzuki-shunsuke added this to the v2.17.2 milestone Nov 13, 2023

suzuki-shunsuke requested changes Nov 14, 2023

View reviewed changes

dreamjz added 2 commits November 14, 2023 12:56

Merge branch 'aquaproj:main' into feat-g_up_cmd-log-enhancement

cd5135f

Merge branch 'aquaproj:main' into feat-g_up_cmd-log-enhancement

c1d8eef

suzuki-shunsuke reviewed Nov 14, 2023

View reviewed changes

suzuki-shunsuke modified the milestones: v2.17.2, v2.17.3 Nov 15, 2023

dreamjz added 3 commits November 16, 2023 20:14

feat: rate limit info for debug

1ba35c0

Merge branch 'main' of https://github.com/aquaproj/aqua into feat-g_u…

245570d

…p_cmd-log-enhancement

feat: log when resp isn't nil

5ea3805

suzuki-shunsuke modified the milestones: v2.17.3, v2.17.4, v2.17.5 Nov 17, 2023

suzuki-shunsuke self-requested a review November 18, 2023 05:28

suzuki-shunsuke approved these changes Nov 18, 2023

View reviewed changes

suzuki-shunsuke merged commit ad89a92 into aquaproj:main Nov 18, 2023
10 checks passed

suzuki-shunsuke added a commit that referenced this pull request Nov 18, 2023

fix: copy logE to restrict the scope of the side effect

5c2f511

- #2461 (comment)

suzuki-shunsuke mentioned this pull request Nov 18, 2023

fix: copy logE to limit the scope of the side effect #2487

Merged

suzuki-shunsuke added a commit that referenced this pull request Nov 18, 2023

fix: copy logE to limit the scope of the side effect

1c99850

- #2461 (comment)

suzuki-shunsuke added a commit that referenced this pull request Nov 18, 2023

fix: copy logE to limit the scope of the side effect (#2487)

c13b2c5

- #2461 (comment)

dreamjz deleted the feat-g_up_cmd-log-enhancement branch September 18, 2024 12:38

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Feat: Log enhancement for command generate and update #2461

Feat: Log enhancement for command generate and update #2461

dreamjz commented Nov 13, 2023 •

edited

Loading

suzuki-shunsuke commented Nov 13, 2023

suzuki-shunsuke Nov 13, 2023

dreamjz Nov 14, 2023

suzuki-shunsuke Nov 14, 2023 •

edited

Loading

suzuki-shunsuke Nov 14, 2023

suzuki-shunsuke commented Nov 14, 2023

suzuki-shunsuke Nov 14, 2023

suzuki-shunsuke Nov 14, 2023

dreamjz Nov 14, 2023

suzuki-shunsuke Nov 14, 2023

dreamjz Nov 14, 2023

dreamjz Nov 14, 2023

suzuki-shunsuke Nov 15, 2023

dreamjz commented Nov 16, 2023

dreamjz commented Nov 16, 2023

suzuki-shunsuke commented Nov 18, 2023 •

edited

Loading

dreamjz commented Nov 18, 2023

suzuki-shunsuke commented Nov 18, 2023

	func (c CargoVersionGetter) Get(ctx context.Context, _ logrus.Entry, pkg registry.PackageInfo, filters []Filter) (string, error) {
	func (c CargoVersionGetter) Get(ctx context.Context, pkg registry.PackageInfo, filters []*Filter) (string, error) {

	func addRteLimitInfo(logE logrus.Entry, resp github.Response) {
	func addRateLimitInfo(logE logrus.Entry, resp github.Response) {

Feat: Log enhancement for command generate and update #2461

Feat: Log enhancement for command generate and update #2461

Conversation

dreamjz commented Nov 13, 2023 • edited Loading

Description

Test cases

generate

update

suzuki-shunsuke commented Nov 13, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

suzuki-shunsuke Nov 14, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

suzuki-shunsuke commented Nov 14, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dreamjz commented Nov 16, 2023

Description

One more question

Test cases

generate

update

dreamjz commented Nov 16, 2023

suzuki-shunsuke commented Nov 18, 2023 • edited Loading

dreamjz commented Nov 18, 2023

suzuki-shunsuke commented Nov 18, 2023

dreamjz commented Nov 13, 2023 •

edited

Loading

suzuki-shunsuke Nov 14, 2023 •

edited

Loading

suzuki-shunsuke commented Nov 18, 2023 •

edited

Loading