Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

osvVulnerabilityAlerts - Connect Timeout Error behind HTTP proxy #252

Closed
philippe-granet opened this issue Mar 13, 2023 · 2 comments · Fixed by #271
Closed

osvVulnerabilityAlerts - Connect Timeout Error behind HTTP proxy #252

philippe-granet opened this issue Mar 13, 2023 · 2 comments · Fixed by #271
Assignees
@JamieMagee
Labels
bug Something isn't working

Comments

@philippe-granet
Copy link

philippe-granet commented Mar 13, 2023 •
edited
Loading

Issue created following the discussion renovatebot/renovate#20876

How are you running Renovate?

Self-hosted

If you're self-hosting Renovate, tell us what version of Renovate you run.

35.1.1

If you're self-hosting Renovate, select which platform you are using.

GitLab self-hosted

If you're self-hosting Renovate, tell us what version of the platform you run.

15.8.2

Was this something which used to work for you, and then stopped?

It used to work, and then stopped

Wanted end result.

After upgrading to renovate >= v35.x , osvVulnerabilityAlerts feature doesn't work anymore and there is only a warn log (even with trace log level): Unable to read vulnerability information.
It work correctly on version <= 34.160.0.
I'm behinh an HTTP proxy.

What you tried so far.

With adding debug logs, I see there is an error when calling fetch nodejs api: throw ConnectTimeoutError: Connect Timeout Error.
I'm behind an HTTP proxy with these env vars:

export HTTP_PROXY=http://[REDACTED]:3131
export HTTPS_PROXY=http://[REDACTED]:3131
export http_proxy=http://[REDACTED]:3131
export https_proxy=http://[REDACTED]:3131

All works with my HTTP proxy with renovate < v35.x.

I see that renovatebot/osv-offline use Octokit api to download osv database.
There is a new fetch api in Node18 and Octokit use this native api when available:
octokit/request.js@d000a0a

And Node have been upgrade from v16 to Node v18 in Renovate Docker image since v35:
renovatebot/docker-renovate@f367ca3

I think my problem is because fetch native api is now used in Renovate Docker image >= v35 and don't work by default with HTTP proxy env vars (nodejs/undici#1650).
It seems to be possible to configure Octokit used by renovatebot/osv-offline (https://github.com/renovatebot/osv-offline/blob/main/packages/osv-offline/src/lib/download.ts#L44) with an HttpProxyAgent :
https://github.com/octokit/core.js/blob/main/test/agent-proxy/agent-proxy-test.test.ts#L62

Relevant debug logs

Logs 
TypeError: fetch failed
    at Object.fetch (node:internal/deps/undici/undici:11413:11)
    at processTicksAndRejections (node:internal/process/task_queues:95:5)
    at tryDownloadDb (/opt/buildpack/tools/renovate/35.0.1/node_modules/@renovatebot/osv-offline/dist/lib/download.js:40:26)
    at OsvOffline.initialize (/opt/buildpack/tools/renovate/35.0.1/node_modules/@renovatebot/osv-offline/dist/lib/osv-offline.js:13:25)
    at Function.create (/opt/buildpack/tools/renovate/35.0.1/node_modules/@renovatebot/osv-offline/dist/lib/osv-offline.js:25:9)
    at Vulnerabilities.initialize (/opt/buildpack/tools/renovate/35.0.1/node_modules/renovate/lib/workers/repository/process/vulnerabilities.ts:44:23)
    at Function.create (/opt/buildpack/tools/renovate/35.0.1/node_modules/renovate/lib/workers/repository/process/vulnerabilities.ts:49:5)
    at fetchVulnerabilities (/opt/buildpack/tools/renovate/35.0.1/node_modules/renovate/lib/workers/repository/process/extract-update.ts:177:31)
    at lookup (/opt/buildpack/tools/renovate/35.0.1/node_modules/renovate/lib/workers/repository/process/extract-update.ts:189:3)
    at extractDependencies (/opt/buildpack/tools/renovate/35.0.1/node_modules/renovate/lib/workers/repository/process/index.ts:131:31)
    at Object.renovateRepository (/opt/buildpack/tools/renovate/35.0.1/node_modules/renovate/lib/workers/repository/index.ts:56:9)
    at attributes.repository (/opt/buildpack/tools/renovate/35.0.1/node_modules/renovate/lib/workers/global/index.ts:181:11)
    at start (/opt/buildpack/tools/renovate/35.0.1/node_modules/renovate/lib/workers/global/index.ts:166:7)
    at /opt/buildpack/tools/renovate/35.0.1/node_modules/renovate/lib/renovate.ts:18:22 {
  cause: ConnectTimeoutError: Connect Timeout Error
      at onConnectTimeout (node:internal/deps/undici/undici:8380:28)
      at node:internal/deps/undici/undici:8338:50
      at Immediate._onImmediate (node:internal/deps/undici/undici:8369:13)
      at processImmediate (node:internal/timers:476:21) {
    code: 'UND_ERR_CONNECT_TIMEOUT'
  }
}

RequestError [HttpError]: fetch failed
    at /opt/buildpack/tools/renovate/35.0.1/node_modules/@octokit/request/dist-src/fetch-wrapper.js:96:15
    at processTicksAndRejections (node:internal/process/task_queues:95:5)
    at tryDownloadDb (/opt/buildpack/tools/renovate/35.0.1/node_modules/@renovatebot/osv-offline/dist/lib/download.js:40:26)
    at OsvOffline.initialize (/opt/buildpack/tools/renovate/35.0.1/node_modules/@renovatebot/osv-offline/dist/lib/osv-offline.js:13:25)
    at Function.create (/opt/buildpack/tools/renovate/35.0.1/node_modules/@renovatebot/osv-offline/dist/lib/osv-offline.js:25:9)
    at Vulnerabilities.initialize (/opt/buildpack/tools/renovate/35.0.1/node_modules/renovate/lib/workers/repository/process/vulnerabilities.ts:44:23)
    at Function.create (/opt/buildpack/tools/renovate/35.0.1/node_modules/renovate/lib/workers/repository/process/vulnerabilities.ts:49:5)
    at fetchVulnerabilities (/opt/buildpack/tools/renovate/35.0.1/node_modules/renovate/lib/workers/repository/process/extract-update.ts:177:31)
    at lookup (/opt/buildpack/tools/renovate/35.0.1/node_modules/renovate/lib/workers/repository/process/extract-update.ts:189:3)
    at extractDependencies (/opt/buildpack/tools/renovate/35.0.1/node_modules/renovate/lib/workers/repository/process/index.ts:131:31)
    at Object.renovateRepository (/opt/buildpack/tools/renovate/35.0.1/node_modules/renovate/lib/workers/repository/index.ts:56:9)
    at attributes.repository (/opt/buildpack/tools/renovate/35.0.1/node_modules/renovate/lib/workers/global/index.ts:181:11)
    at start (/opt/buildpack/tools/renovate/35.0.1/node_modules/renovate/lib/workers/global/index.ts:166:7)
    at /opt/buildpack/tools/renovate/35.0.1/node_modules/renovate/lib/renovate.ts:18:22 {
  status: 500,
  request: {
    method: 'GET',
    url: 'https://api.github.com/repos/renovatebot/osv-offline/releases',
    headers: {
      accept: 'application/vnd.github.v3+json',
      'user-agent': 'octokit-rest.js/19.0.7 octokit-core.js/4.2.0 Node.js/18.15.0 (linux; x64)',
      authorization: 'token [REDACTED]'
    },
    request: { hook: [Function: bound bound register] }
  }
}
@viceice viceice added the bug Something isn't working label Mar 13, 2023
@JamieMagee
Copy link
Contributor

Thanks for the detailed bug report. The bug fix is going to be either adding an HttpProxyAgent, or overriding the fetch implementation used:

https://github.com/octokit/types.ts/blob/93dce436de8ad7ef1a227f06da4ded7fefe5722d/src/RequestRequestOptions.ts#L14-L17

@philippe-granet
Copy link
Author

philippe-granet commented Mar 25, 2023 •
edited
Loading

If someone have the same problem, you can use this temporary fix:
sed -i "s/const fetch.*/const fetch = nodeFetch;/" /usr/src/app/node_modules/@octokit/request/dist-node/index.js

JamieMagee added a commit that referenced this issue Mar 27, 2023
@JamieMagee
fix: use 'node-fetch' as Octokit's fetch
a764d68 
In v6.1.0 Octokit started using Node.js's built-in `fetch`[^1] (provided by [undici][1]. Unfortunately, it is not 100% compatible with `node-fetch`, and notably it doesn't support `HTTP_PROXY` environment variables[^2].

This change switches `osv-offline` to explicitly use `node-fetch`.

Closes #252

[1]: https://github.com/nodejs/undici

[^1]: octokit/request.js@d000a0a
[^2]: nodejs/undici#1650
@JamieMagee JamieMagee mentioned this issue Mar 27, 2023
Merged
JamieMagee added a commit that referenced this issue Mar 27, 2023
@JamieMagee
fix: use 'node-fetch' as Octokit's fetch
277ee49 
In v6.1.0 Octokit started using Node.js's built-in `fetch`[^1] (provided by [undici][1]. Unfortunately, it is not 100% compatible with `node-fetch`, and notably it doesn't support `HTTP_PROXY` environment variables[^2].

This change switches `osv-offline` to explicitly use `node-fetch`.

Closes #252

[1]: https://github.com/nodejs/undici

[^1]: octokit/request.js@d000a0a
[^2]: nodejs/undici#1650
JamieMagee added a commit that referenced this issue Mar 28, 2023
@JamieMagee
fix: use 'node-fetch' as Octokit's fetch
942801b 
In v6.1.0 Octokit started using Node.js's built-in `fetch`[^1] (provided by [undici][1]. Unfortunately, it is not 100% compatible with `node-fetch`, and notably it doesn't support `HTTP_PROXY` environment variables[^2].

This change switches `osv-offline` to explicitly use `node-fetch`.

Closes #252

[1]: https://github.com/nodejs/undici

[^1]: octokit/request.js@d000a0a
[^2]: nodejs/undici#1650
JamieMagee added a commit that referenced this issue Apr 1, 2023
@JamieMagee
fix: use 'node-fetch' as Octokit's fetch
4f7bb0b 
In v6.1.0 Octokit started using Node.js's built-in `fetch`[^1] (provided by [undici][1]. Unfortunately, it is not 100% compatible with `node-fetch`, and notably it doesn't support `HTTP_PROXY` environment variables[^2].

This change switches `osv-offline` to explicitly use `node-fetch`.

Closes #252

[1]: https://github.com/nodejs/undici

[^1]: octokit/request.js@d000a0a
[^2]: nodejs/undici#1650
JamieMagee added a commit that referenced this issue Apr 1, 2023
@JamieMagee
fix: use 'node-fetch' as Octokit's fetch
63ba618 
In v6.1.0 Octokit started using Node.js's built-in `fetch`[^1] (provided by [undici][1]. Unfortunately, it is not 100% compatible with `node-fetch`, and notably it doesn't support `HTTP_PROXY` environment variables[^2].

This change switches `osv-offline` to explicitly use `node-fetch`.

Closes #252

[1]: https://github.com/nodejs/undici

[^1]: octokit/request.js@d000a0a
[^2]: nodejs/undici#1650
JamieMagee added a commit that referenced this issue Apr 1, 2023
@JamieMagee
fix: use 'node-fetch' as Octokit's fetch
8caa7e7 
In v6.1.0 Octokit started using Node.js's built-in `fetch`[^1] (provided by [undici][1]. Unfortunately, it is not 100% compatible with `node-fetch`, and notably it doesn't support `HTTP_PROXY` environment variables[^2].

This change switches `osv-offline` to explicitly use `node-fetch`.

Closes #252

[1]: https://github.com/nodejs/undici

[^1]: octokit/request.js@d000a0a
[^2]: nodejs/undici#1650
JamieMagee added a commit that referenced this issue Apr 1, 2023
@JamieMagee
fix: use 'node-fetch' as Octokit's fetch
fa62146 
In v6.1.0 Octokit started using Node.js's built-in `fetch`[^1] (provided by [undici][1]. Unfortunately, it is not 100% compatible with `node-fetch`, and notably it doesn't support `HTTP_PROXY` environment variables[^2].

This change switches `osv-offline` to explicitly use `node-fetch`.

Closes #252

[1]: https://github.com/nodejs/undici

[^1]: octokit/request.js@d000a0a
[^2]: nodejs/undici#1650
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@JamieMagee JamieMagee

Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

fix: use node-fetch as Octokit's fetch renovatebot/osv-offline
3 participants
@JamieMagee @viceice @philippe-granet