Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BACK-3295/BACK-3332] Implement private plugin functionality in platform, update dependencies #803

Open
wants to merge 12 commits into
base: master
Choose a base branch
from

Conversation

darinkrauss
Copy link
Contributor

@darinkrauss darinkrauss commented Jan 11, 2025

[BACK-3295/BACK-3332] Implement private plugin functionality in platform, update dependencies

  • Implement private plugin functionality in platform
  • Update dependencies
  • Add Go workspace to build with private plugins
  • Update Makefile with targets to switch between public and private configuration
  • Update Makefile to utilize new Go workspace
  • Update Makefile with minor target additions and changes
  • Update Makefile with changes to Docker build process
  • Update Makefile to allow generation of phony targets
  • Update Dockerfiles to fix image cache miss issues
  • Update .travis.yml to latest dependencies and use new Go workspace
  • Update to latest Go version
  • Update tools production image to use Mongo version matching production
  • Add private plugin redwood submodule
  • Add minimal redwood plugin module and raw data and work packages
  • Integrate above into auth and data service
  • Remove unnecessary data.Context
  • Incorporate retry into OAuth client SendOAuthRequest for access token errors
  • Merge HTTPClientSource into TokenSource interface as unnecessary
  • Add generalized PrometheusCodePathResponseInspector
  • Update Dexcom Client with above changes
  • Remove unnecessary logger from request inspector
  • Remove vendor directory and various unused dependencies
  • Remove Makefile deploy target
  • Remove golint dependency
  • Remove Code Climate dependency
  • Update Go code generation
  • Remove go generate --build_flags due to conflict with Go workspaces
  • Standardize mockgen usage in source files
  • Update to latest go.uber.org/mock/gomock
  • Remove unnecessary mocks
  • Remove unnecessary esc module
  • https://tidepool.atlassian.net/browse/BACK-3295
  • https://tidepool.atlassian.net/browse/BACK-3332

➡️➡️➡️ NOTE: I suggest you review each separate commit independently. Each one is fairly isolated to a single intended change.

- Remove go generate --build_flags due to conflict with Go workspaces
- Standardize mockgen usage in source files
- Update to latest go.uber.org/mock/gomock
- Remove unnecessary mocks
- Remove unnecessary esc module
- Remove vendor directory and various unused dependencies
- Remove Makefile deploy target
- Remove golint dependency
- Remove Code Climate dependency
… errors

- Incorporate retry into OAuth client SendOAuthRequest for access token errors
- Merge HTTPClientSource into TokenSource interface as unnecessary
- Add generalized PrometheusCodePathResponseInspector
- Update Dexcom Client with above changes
- Add minimal redwood plugin module and raw data and work packages
- Integrate above into auth and data service
- Remove unnecessary data.Context
- Add Go workspace to build with private plugins
- Update Makefile with targets to switch between public and private configuration
- Update Makefile to utilize new Go workspace
- Update Makefile with minor target additions and changes
- Update Makefile with changes to Docker build process
- Update Makefile to allow generation of phony targets
- Update Dockerfiles to fix image cache miss issues
- Update .travis.yml to latest dependencies and use new Go workspace
- Update to latest Go version
- Update tools production image to use Mongo version matching production
- https://tidepool.atlassian.net/browse/BACK-3295
@darinkrauss darinkrauss changed the title Back 3295 private plugins [BACK-3295/BACK-3332] Implement private plugin functionality in platform, update dependencies Jan 11, 2025
toddkazakov
toddkazakov previously approved these changes Jan 17, 2025
@toddkazakov
Copy link
Contributor

Couldn't leave inline comments so adding few things that stood out:

  • The "build" stage of the docker file should probably be extracted in a separate file instead of duplicated for all services. In my experience, it's very easy for those fragments to get out of sync between services. The build can be orchestrated with the makefile instead (e.g. make build runs docker build . -t build and make release builds the service image similar to how it's done in the multistage build we have for each service
  • The use of PrometheusCodePathResponseInspector can get out of hand if the path variable is with high cardinality, leading to OOMs in prometheus. Paths ought to have wildcard segments. I discussed this with Eric when he added this for instrumenting Dexcom code and I believe this is the reason why the response inspector stayed close to the caller. Simply plugging it to an arbitrary http client might have unexpected consequences, because Prometheus creates a histogram for each combination of metric/path/status.

Those issues don't need to be fixed immediately.

The private build setup looks good! Thank you!

- Add path pattern matching to PrometheusCodePathResponseInspector
- Move common Dockerfile steps into separate common Dockerfile
- Update Makefile to build common Dockerfile
@darinkrauss
Copy link
Contributor Author

Addressed the PR comments:

  • Moved common Docker steps to common Dockerfile.build and updated Makefile to accommodate.
  • Added functionality to PrometheusCodePathResponseInspector to allow path pattern recognition and (assuming one uses the new functionality) avoid the high cardinality issue.

toddkazakov
toddkazakov previously approved these changes Jan 21, 2025
Copy link
Contributor

@toddkazakov toddkazakov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 👍

Copy link
Contributor

@jh-bate jh-bate left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed the latest commits after Todd's ok and LGTM

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants