Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

🪟 🔧 Refactor webapp configuration provider #21456

Merged
merged 31 commits into from
Jan 27, 2023

Conversation

josephkmh
Copy link
Contributor

@josephkmh josephkmh commented Jan 16, 2023

What

Simplifies our app config to use a single, central configuration object.

Closes #10955

How

Consolidates many config files into one

Previously we had configuration split across many files, organized by where the value was coming from (.env file, window object, config.json file, etc). These many configuration files have been consolidated into a single object where we can see all config values: src/config/config.ts.

For the moment, we are still setting some configuration properties via the window object. This allows us to "inject" runtime overrides in the nginx container. These will be removed in a future PR, in favor of setting all required configuration values via environment variables. This will allow us to configure each environment at build time (without any runtime overrides) and ultimately deploy the application as simple static files on a CDN.

Consolidates two useConfig() hooks into one

Previously we had two useConfig() hooks - one in src/packages/cloud/services/config and one in src/config. Since the config object has been consolidated, we don't need these two hooks anymore. We will only use the one located in src/config.

This has some implications: some services depend on configuration values that are only available in cloud environments (such as Firebase or Intercom configuration, a cloudApiUrl, etc). These services will now throw a MissingConfigError if their required config is not available:

  if (!intercom.appId) {
    throw new MissingConfigError("Missing required configuration intercom.appId");
  }

Note: I tried implementing a Proxy here to throw this error whenever config is accessed with a property that is undefined. Unfortunately TypeScript does not properly infer types from proxies (issue) which means the "optional" cloud-only types remain string | undefined. While we could force TS to cooperate here, I believe this could also give consumers the wrong impression that all config properties are always available, while in fact attempting to access an undefined property could throw an error at runtime. I think it's best to keep the string | undefined type for those config properties that are not guaranteed to be set.

What's next?

This is Phase 1, Step 1 of the plan outlined here: https://docs.google.com/document/d/1QIDhYP90VSsmL-ddn-2iL5QD-y0fDYgkDd0IErBXYBY/edit#heading=h.wc4fbc3h3qyl

The next step will be Phase 1, Step 2: Removing nginx runtime environment variables

🚨 User Impact 🚨

These changes could potentially break a variety of integrations and functionality, so they should be tested carefully.

Configuration is verified to be working in:

  • Local OSS/docker deployment
  • Local kubernetes deployment
  • frontend-dev
  • dev
  • dev-1
  • dev-2
  • dev-3
  • staging
  • production

@octavia-squidington-iv octavia-squidington-iv added the area/frontend Related to the Airbyte webapp label Jan 16, 2023
LAUNCHDARKLY_KEY?: string;
analytics: SegmentAnalytics.AnalyticsJS;
}
}

export interface Config {
export interface AirbyteWebappConfig {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Renamed to make it more clear what this is a config for

@josephkmh josephkmh changed the title 🪟 🔧 [DRAFT] Refactor webapp config providers 🪟 🔧 [DRAFT] Refactor webapp configuration provider Jan 17, 2023
@josephkmh josephkmh temporarily deployed to more-secrets January 18, 2023 15:51 — with GitHub Actions Inactive
Comment on lines +46 to +48
if (!cloudApiUrl) {
throw new MissingConfigError("Missing required configuration cloudApiUrl");
}
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Another option would be to log the error with our AppMonitoringService() instead of throwing it here. The downside is that this would not be surfaced to the user in environments without datadog.

@josephkmh josephkmh marked this pull request as ready for review January 19, 2023 10:13
@josephkmh josephkmh changed the title 🪟 🔧 [DRAFT] Refactor webapp configuration provider 🪟 🔧 Refactor webapp configuration provider Jan 19, 2023
@josephkmh josephkmh requested a review from timroes January 24, 2023 09:36
@@ -38,4 +62,4 @@ export type Provider<T> = () => T;

export type ValueProvider<T> = Array<ProviderAsync<DeepPartial<T>>>;

export type ConfigProvider<T extends Config = Config> = ProviderAsync<DeepPartial<T>>;
export type ConfigProvider<T extends AirbyteWebappConfig = AirbyteWebappConfig> = ProviderAsync<DeepPartial<T>>;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think most of those types could still be removed or the Provider which seems to be used in the auth service, just inlined there instead (and most likely doesn't need a named type).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, forgot that I wanted to clean up (remove) these unnecessary types!

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

addressed here 1550b3a

Copy link
Contributor

@timroes timroes left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code LGTM. Have tested locally in cloud mode. Everything seems to be applied as expected.

@josephkmh josephkmh enabled auto-merge (squash) January 25, 2023 10:36
@josephkmh
Copy link
Contributor Author

The e2e tests are legitimately failing here, will need to refactor useFreeConnectorProgram() a bit before merging this PR

@ambirdsall
Copy link
Contributor

I was too occupied to get in a timely review, but I'm glad I kept the tab open: this is great stuff, much cleaner.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/frontend Related to the Airbyte webapp
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Generalize configProvider
4 participants