Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use AcceptResponseHandler in goproxy https CONNECT hook #199

Merged
merged 6 commits into from
Aug 3, 2023

Conversation

cmoresco-stripe
Copy link
Collaborator

Wire up the AcceptResponseHandler to the new goproxy hook that was added in stripe/goproxy#21

This does require changing the type of the AcceptResponseHandler to include a context field, but since this handler was only added to the config ~days ago this shouldn't be disruptive.

Smokescreen takes a function that uses a smokescreenContext and wraps it in a function that takes a goproxy.ProxyCtx before passing it to the underlying goproxy, so that we don't leak any goproxy interfaces in the smokescreen API (remain eternally hopeful that we'll strip out goproxy entirely someday...)

Writing a unit test would require the new go 1.20 http.OnProxyConnectResponse to hook into the proxy connect response, and I don't think we're ready to upgrade smokescreen to go 1.20 yet, so the https logic here is not covered by unit tests.

@github-actions
Copy link

github-actions bot commented Aug 2, 2023

Pull Request Test Coverage Report for Build 5742065861

  • 3 of 8 (37.5%) changed or added relevant lines in 1 file are covered.
  • No unchanged relevant lines lost coverage.
  • Overall coverage decreased (-0.1%) to 62.524%

Changes Missing Coverage Covered Lines Changed/Added Lines %
pkg/smokescreen/smokescreen.go 3 8 37.5%
Totals Coverage Status
Change from base Build 5681729420: -0.1%
Covered Lines: 1303
Relevant Lines: 2084

💛 - Coveralls

Copy link
Collaborator

@jjiang-stripe jjiang-stripe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah sorry I thought I hit approve already :')

this seems reasonable to me! we're just hooking into the ConnectRespHandler functionality we added.

@cmoresco-stripe cmoresco-stripe merged commit 48069eb into master Aug 3, 2023
@cmoresco-stripe cmoresco-stripe deleted the cmoresco/hooks branch August 3, 2023 18:16
matt-intercom added a commit to intercom/smokescreen that referenced this pull request Jan 4, 2024
* add a custom interface for the resolver instead of forcing *net.Resolver (stripe#187)

* feature/add prometheus metrics (stripe#179)

* STORY-25143 - Add prometheus metrics to smokescreen

* STORY-25143 - Cleanup

* STORY-25143 - Fix tests to compare new metric labels

* STORY-25143 - Host prometheus endpoint on separate port

* STORY-25143 - Use value provided via command line flag

* STORY-25143 - Add prometheus timing metrics

* STORY-25143 - Fix nil map assignment and prometheus metric name sanitisation

* STORY-25143 - Cleanup comments

* STORY-25143 - Remove some repetition + add further unit testing

* STORY-25143 - Document new prometheus features in README + add port flag to prometheus config

* STORY-25143 - Make PR requested changes:
* Don't export metrics list
* Follow project sytlistic choices

* STORY-25143 - Rename only one receiver

* STORY-25143 - Add new `--expose-prometheus-metrics` flag to CLI to toggle exposing prometheus metrics

* Small cleanup of timer metrics

* Fix go module vendoring

* Use ElementsMatch to ignore order

* Just use require

* Move the custom request handler call after the main acl check

* Use local server instead of httpbin (stripe#192)

* Do not return a denyError for DNS resolution failures (stripe#194)

* dont return denial errors for dns resolution failures

* fix test

* move DNSError check into net.Error assertion, extend test

* fix integration test

* add AcceptResponseHandler to modify accepted responses (stripe#196)

* add AcceptResponseHandler to modify accepted responses

* customer->custom

* Update docs to clarify global_deny_list (stripe#197)

* update docs to clarify global_deny_list behavior

* consistent example domain

* be more concise

* Use AcceptResponseHandler in goproxy https CONNECT hook (stripe#199)

* pipe AcceptResponseHandler into new goproxy hook

* update comment

* go mod vendor

* unit test

* use smokescreenctx in acceptresponsehandler

* fix unit tests

* Export SmokescreenContext type (stripe#200)

* export SmokescreenContext type

* also export AclDecision

* ResolvedAddr too

* consistent caps

* Update pkg/smokescreen/smokescreen.go

Co-authored-by: jjiang-stripe <[email protected]>

* export Decision

---------

Co-authored-by: jjiang-stripe <[email protected]>

* generate new test pki (stripe#206)

* allow listen address specification for prom (stripe#203)

* Bump golang.org/x/net from 0.7.0 to 0.17.0 (stripe#204)

Bumps [golang.org/x/net](https://github.com/golang/net) from 0.7.0 to 0.17.0.
- [Commits](golang/net@v0.7.0...v0.17.0)

---
updated-dependencies:
- dependency-name: golang.org/x/net
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* bump go versions (stripe#207)

* update dependency

* configure addr in smokescreen and add unit test

* use fmt

* try this workaround

* variable name change

* Update docs to disambiguate ACL vs --deny-address behavior (stripe#210)

* update docs to clarify how IP filtering works

---------

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: JulesD <[email protected]>
Co-authored-by: Josh McConnell <[email protected]>
Co-authored-by: Kevin Vincent <[email protected]>
Co-authored-by: kevinv-stripe <[email protected]>
Co-authored-by: Sergey Rud <[email protected]>
Co-authored-by: cmoresco-stripe <[email protected]>
Co-authored-by: Craig Shannon <[email protected]>
Co-authored-by: jjiang-stripe <[email protected]>
Co-authored-by: Timofey Bakunin <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Yuxi Xie <[email protected]>
Co-authored-by: xieyuxi-stripe <[email protected]>
matt-intercom pushed a commit to intercom/smokescreen that referenced this pull request Jan 4, 2024
* pipe AcceptResponseHandler into new goproxy hook

* update comment

* go mod vendor

* unit test

* use smokescreenctx in acceptresponsehandler

* fix unit tests
matt-intercom added a commit to intercom/smokescreen that referenced this pull request Jan 4, 2024
amber-higgins added a commit to intercom/smokescreen that referenced this pull request Jan 27, 2025
* add a custom interface for the resolver instead of forcing *net.Resolver (stripe#187)

* feature/add prometheus metrics (stripe#179)

* STORY-25143 - Add prometheus metrics to smokescreen

* STORY-25143 - Cleanup

* STORY-25143 - Fix tests to compare new metric labels

* STORY-25143 - Host prometheus endpoint on separate port

* STORY-25143 - Use value provided via command line flag

* STORY-25143 - Add prometheus timing metrics

* STORY-25143 - Fix nil map assignment and prometheus metric name sanitisation

* STORY-25143 - Cleanup comments

* STORY-25143 - Remove some repetition + add further unit testing

* STORY-25143 - Document new prometheus features in README + add port flag to prometheus config

* STORY-25143 - Make PR requested changes:
* Don't export metrics list
* Follow project sytlistic choices

* STORY-25143 - Rename only one receiver

* STORY-25143 - Add new `--expose-prometheus-metrics` flag to CLI to toggle exposing prometheus metrics

* Small cleanup of timer metrics

* Fix go module vendoring

* Use ElementsMatch to ignore order

* Just use require

* Move the custom request handler call after the main acl check

* Use local server instead of httpbin (stripe#192)

* Do not return a denyError for DNS resolution failures (stripe#194)

* dont return denial errors for dns resolution failures

* fix test

* move DNSError check into net.Error assertion, extend test

* fix integration test

* add AcceptResponseHandler to modify accepted responses (stripe#196)

* add AcceptResponseHandler to modify accepted responses

* customer->custom

* Update docs to clarify global_deny_list (stripe#197)

* update docs to clarify global_deny_list behavior

* consistent example domain

* be more concise

* Use AcceptResponseHandler in goproxy https CONNECT hook (stripe#199)

* pipe AcceptResponseHandler into new goproxy hook

* update comment

* go mod vendor

* unit test

* use smokescreenctx in acceptresponsehandler

* fix unit tests

* Export SmokescreenContext type (stripe#200)

* export SmokescreenContext type

* also export AclDecision

* ResolvedAddr too

* consistent caps

* Update pkg/smokescreen/smokescreen.go

Co-authored-by: jjiang-stripe <[email protected]>

* export Decision

---------

Co-authored-by: jjiang-stripe <[email protected]>

* generate new test pki (stripe#206)

* allow listen address specification for prom (stripe#203)

* Bump golang.org/x/net from 0.7.0 to 0.17.0 (stripe#204)

Bumps [golang.org/x/net](https://github.com/golang/net) from 0.7.0 to 0.17.0.
- [Commits](golang/net@v0.7.0...v0.17.0)

---
updated-dependencies:
- dependency-name: golang.org/x/net
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* bump go versions (stripe#207)

* update dependency

* configure addr in smokescreen and add unit test

* use fmt

* try this workaround

* variable name change

* Update docs to disambiguate ACL vs --deny-address behavior (stripe#210)

* update docs to clarify how IP filtering works

* fix fields bug

* remove extra field setting

* trigger build

* Add support for Smokescreen -> HTTPS CONNECT Proxy ACLs (stripe#213)

* Introduce CONNECT Proxy URL ACL Support

Add gitignore debug changes

WIP

Basic concept working

WIP

Cleaned up some things prereview

fixed tests

Removed extraneous yaml file

Add correctly failing test

tmp

WIP

WIP

WIP

WIP

WIP

WIP

* WIP

* WIP

* PR feedback 1

* Fixed tests

* testing again

* WIP

* Added extra test

* Bump goproxy version to incorporate CONNECT proxy header changes

* WIP

* Bump google.golang.org/protobuf from 1.28.1 to 1.33.0 (stripe#216)

Bumps google.golang.org/protobuf from 1.28.1 to 1.33.0.

---
updated-dependencies:
- dependency-name: google.golang.org/protobuf
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Add support for username / password auth in URLs to external CONNECT proxies (stripe#222)

* Add support for UN / PW Auth for External CONNECT Proxies

* Fixed naming of log line

* PR feedback

* Debug commit

* Removing modifications of vendor-ed code

* Removed debug

* Removed missed cruft

* Fixed bug with env var proxy arg

* Add failure kind

* update goproxy version to master commit

* Ensure proxy passed in X-Upstream-Https-Proxy is parsable

* Update Github build workflows (stripe#228)

Co-authored-by: Harold Simpson <[email protected]>

* Use goveralls parallel build

* go get -d github.com/stripe/goproxy@latest && go mod vendor

* Add MITM support to Smokescreen

* Use MitmTLSConfig in the config instead of MitmCa

* PR feedback + remove CloseIdleConnections

* Refactor allowed_domains_mitm to mitm_domains

* Rename ValidateRule

* Add Support for Reject Handler with Context

* Update comment

* Block smokescreen init incase of invalid config

* fix: fix slice init length

* Remove duplicate validation

* Make SmokeScreen Fields Public

* Revert Role fixes

* Revert Role fixes

* Update goproxy version to v0.0.0-20241017101008-e12ef0653f22 (stripe#235)

* Adding [allow|deny]_addresses settings to yaml config file

* Update goproxy version to v0.0.0-20241022131412-58117846327a (stripe#238)

* Ignore goveralls

---------

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: JulesD <[email protected]>
Co-authored-by: Josh McConnell <[email protected]>
Co-authored-by: Kevin Vincent <[email protected]>
Co-authored-by: kevinv-stripe <[email protected]>
Co-authored-by: Sergey Rud <[email protected]>
Co-authored-by: cmoresco-stripe <[email protected]>
Co-authored-by: Craig Shannon <[email protected]>
Co-authored-by: jjiang-stripe <[email protected]>
Co-authored-by: Timofey Bakunin <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Yuxi Xie <[email protected]>
Co-authored-by: xieyuxi-stripe <[email protected]>
Co-authored-by: Jessica Jiang <[email protected]>
Co-authored-by: pspieker-stripe <[email protected]>
Co-authored-by: Patrick Spieker <[email protected]>
Co-authored-by: Gautham Warrier <[email protected]>
Co-authored-by: gauthamw-stripe <[email protected]>
Co-authored-by: harold-stripe <[email protected]>
Co-authored-by: Harold Simpson <[email protected]>
Co-authored-by: Saurabh Bhatia <[email protected]>
Co-authored-by: cui fliter <[email protected]>
Co-authored-by: Bryan Eastes <[email protected]>
amber-higgins added a commit to intercom/smokescreen that referenced this pull request Jan 27, 2025
* add a custom interface for the resolver instead of forcing *net.Resolver (stripe#187)

* feature/add prometheus metrics (stripe#179)

* STORY-25143 - Add prometheus metrics to smokescreen

* STORY-25143 - Cleanup

* STORY-25143 - Fix tests to compare new metric labels

* STORY-25143 - Host prometheus endpoint on separate port

* STORY-25143 - Use value provided via command line flag

* STORY-25143 - Add prometheus timing metrics

* STORY-25143 - Fix nil map assignment and prometheus metric name sanitisation

* STORY-25143 - Cleanup comments

* STORY-25143 - Remove some repetition + add further unit testing

* STORY-25143 - Document new prometheus features in README + add port flag to prometheus config

* STORY-25143 - Make PR requested changes:
* Don't export metrics list
* Follow project sytlistic choices

* STORY-25143 - Rename only one receiver

* STORY-25143 - Add new `--expose-prometheus-metrics` flag to CLI to toggle exposing prometheus metrics

* Small cleanup of timer metrics

* Fix go module vendoring

* Use ElementsMatch to ignore order

* Just use require

* Move the custom request handler call after the main acl check

* Use local server instead of httpbin (stripe#192)

* Do not return a denyError for DNS resolution failures (stripe#194)

* dont return denial errors for dns resolution failures

* fix test

* move DNSError check into net.Error assertion, extend test

* fix integration test

* add AcceptResponseHandler to modify accepted responses (stripe#196)

* add AcceptResponseHandler to modify accepted responses

* customer->custom

* Update docs to clarify global_deny_list (stripe#197)

* update docs to clarify global_deny_list behavior

* consistent example domain

* be more concise

* Use AcceptResponseHandler in goproxy https CONNECT hook (stripe#199)

* pipe AcceptResponseHandler into new goproxy hook

* update comment

* go mod vendor

* unit test

* use smokescreenctx in acceptresponsehandler

* fix unit tests

* Export SmokescreenContext type (stripe#200)

* export SmokescreenContext type

* also export AclDecision

* ResolvedAddr too

* consistent caps

* Update pkg/smokescreen/smokescreen.go

Co-authored-by: jjiang-stripe <[email protected]>

* export Decision

---------

Co-authored-by: jjiang-stripe <[email protected]>

* generate new test pki (stripe#206)

* allow listen address specification for prom (stripe#203)

* Bump golang.org/x/net from 0.7.0 to 0.17.0 (stripe#204)

Bumps [golang.org/x/net](https://github.com/golang/net) from 0.7.0 to 0.17.0.
- [Commits](golang/net@v0.7.0...v0.17.0)

---
updated-dependencies:
- dependency-name: golang.org/x/net
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* bump go versions (stripe#207)

* update dependency

* configure addr in smokescreen and add unit test

* use fmt

* try this workaround

* variable name change

* Update docs to disambiguate ACL vs --deny-address behavior (stripe#210)

* update docs to clarify how IP filtering works

* fix fields bug

* remove extra field setting

* trigger build

* Add support for Smokescreen -> HTTPS CONNECT Proxy ACLs (stripe#213)

* Introduce CONNECT Proxy URL ACL Support

Add gitignore debug changes

WIP

Basic concept working

WIP

Cleaned up some things prereview

fixed tests

Removed extraneous yaml file

Add correctly failing test

tmp

WIP

WIP

WIP

WIP

WIP

WIP

* WIP

* WIP

* PR feedback 1

* Fixed tests

* testing again

* WIP

* Added extra test

* Bump goproxy version to incorporate CONNECT proxy header changes

* WIP

* Bump google.golang.org/protobuf from 1.28.1 to 1.33.0 (stripe#216)

Bumps google.golang.org/protobuf from 1.28.1 to 1.33.0.

---
updated-dependencies:
- dependency-name: google.golang.org/protobuf
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Add support for username / password auth in URLs to external CONNECT proxies (stripe#222)

* Add support for UN / PW Auth for External CONNECT Proxies

* Fixed naming of log line

* PR feedback

* Debug commit

* Removing modifications of vendor-ed code

* Removed debug

* Removed missed cruft

* Fixed bug with env var proxy arg

* Add failure kind

* update goproxy version to master commit

* Ensure proxy passed in X-Upstream-Https-Proxy is parsable

* Update Github build workflows (stripe#228)

Co-authored-by: Harold Simpson <[email protected]>

* Use goveralls parallel build

* go get -d github.com/stripe/goproxy@latest && go mod vendor

* Add MITM support to Smokescreen

* Use MitmTLSConfig in the config instead of MitmCa

* PR feedback + remove CloseIdleConnections

* Refactor allowed_domains_mitm to mitm_domains

* Rename ValidateRule

* Add Support for Reject Handler with Context

* Update comment

* Block smokescreen init incase of invalid config

* fix: fix slice init length

* Remove duplicate validation

* Make SmokeScreen Fields Public

* Revert Role fixes

* Revert Role fixes

* Update goproxy version to v0.0.0-20241017101008-e12ef0653f22 (stripe#235)

* Adding [allow|deny]_addresses settings to yaml config file

* Update goproxy version to v0.0.0-20241022131412-58117846327a (stripe#238)

---------

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: JulesD <[email protected]>
Co-authored-by: Josh McConnell <[email protected]>
Co-authored-by: Kevin Vincent <[email protected]>
Co-authored-by: kevinv-stripe <[email protected]>
Co-authored-by: Sergey Rud <[email protected]>
Co-authored-by: cmoresco-stripe <[email protected]>
Co-authored-by: Craig Shannon <[email protected]>
Co-authored-by: jjiang-stripe <[email protected]>
Co-authored-by: Timofey Bakunin <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Yuxi Xie <[email protected]>
Co-authored-by: xieyuxi-stripe <[email protected]>
Co-authored-by: Jessica Jiang <[email protected]>
Co-authored-by: pspieker-stripe <[email protected]>
Co-authored-by: Patrick Spieker <[email protected]>
Co-authored-by: Gautham Warrier <[email protected]>
Co-authored-by: gauthamw-stripe <[email protected]>
Co-authored-by: harold-stripe <[email protected]>
Co-authored-by: Harold Simpson <[email protected]>
Co-authored-by: Saurabh Bhatia <[email protected]>
Co-authored-by: cui fliter <[email protected]>
Co-authored-by: Bryan Eastes <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants