Ability to use already retrieved Config in configuring things further down the chain

[kenfinnigan]

See microprofile/microprofile-config#470 for some details.

Basically explore loading ConfigSourceProvider and ConfigSource in low to high ordinal number, such that higher ordinal instances can use the "as is" config from lower ordinal instances

[phillip-kruger]

I'll look at this a.s.a.p.
:)

[geoand]

Hey folks,

Would you mind providing me some details on what the status of this is? I'd love to try it out if there is something in the works and / or help out since I'm really interested in it from the Quarkus side of things.

[dmlloyd]

It might not be a workable idea, at least not as presented. I think it might be a good idea to break out the use cases though, and maybe have issues for each of those, since this issue may or may not be solvable.

[geoand]

The use case I have in mind is to be able to use configuration from some source in order to then be able to instantiate a class that will create another source based on some data it will fetch (most likely over the wire).

[dmlloyd]

Let's start up a different issue for that. If we come up with a solution for this one, we can solve that with this, but if not, we can close this one out but still keep pursuing the use case.

[geoand]

OK, I created #215

[radcortez]

Our ZookeeperConfigSource is currently trying to configure itself with other properties:

smallrye-config/sources/zookeeper/src/main/java/io/smallrye/config/source/zookeeper/ZooKeeperConfigSource.java

Lines 88 to 95 in 7f48ead

	/*
	* Explicitly ignore all keys that are prefixed with the prefix used to configure the Zookeeper connection.
	* Other wise a stack overflow obviously happens.
	*/
	// TODO - radcortez - We need to add a feature that allows ConfigSource to config itself with other ConfigSource
	if (key.startsWith(IGNORED_PREFIX) || key.startsWith("smallrye.config")) {
	return null;
	}

This may cause issues with a possible Stackoverflow when looking for configs, so we really need to provide a way to configure ConfigSources.

Maybe we can use a similar solution to the one we use to instantiate interceptors in ConfigSourceInterceptorFactory.

The real question is, what should be exposed to the ConfigSource being built? Exposing the Config directly might be a bad idea, since the Config is also initializing. Expose the current initialized ConfigSources might be an alternative, but this requires the implementor to iterate and find the config manually, plus there is no Conversion available.

Maybe just expose a ConfigContext with a single getValue method?

[dmlloyd]

The interception model is really perfect for this use case: each config source could conceptually access all the ones configured before it, no state is mutated late, etc.

I think it would be great if there were a way to unify the config source and interceptor models. In particular I think that interceptors need a way to intercept config source iteration (that's the missing piece).

[radcortez]

Yes, I was first thinking that interceptors could easily solve this. On the other hand, I still have some doubts if both concepts should be mixed, at least to the implementor. A ConfigContext passed to the ConfigSource initialisation is just a wrapper on our interceptor chain.

Maybe they don't need to intercept the config source iteration. Maybe just intercepting the current "view" of already built ConfigSources is enough.

[dmlloyd]

There is an existing reason (and corresponding bug, which I should open) for interceptors to need to intercept iteration.

In Quarkus, we iterate property names to find properties defined in maps that we wouldn't ordinarily be able to find. If we switch to using an interceptor for configuration profile, then iteration will no longer return properties that are defined in profiles which don't have a corresponding non-profile configuration key defined somewhere.

On a more general note, if one were to include iteration in to interceptors, then an interceptor would be able to do everything that a config source can do (but better).

[radcortez]

Ok! Please open up the issue :)

[dmlloyd]

Opened in #285.

[radcortez]

I believe most of the work to support this is in place. We now just need to define the right API for each ConfigSource to access their predecessors.

An issue here is that we don't control the ConfigSource creation, since this is done by the SPI and the user might give us their own instances via ConfigSourceProvider, so a convention constructor that exposes the context like we have for the interceptor might be out of the picture.

Maybe we could support a @PostConstruct method call. We know that the ConfigSource is not usable while the Config is being initialized, so calling a @PostConstruct should be safe. The method may take a ConfigSourceInterceptorContext or even a list of higher priority ConfigSource to provide access to the current configs and do initialization.

@dmlloyd Any other brilliant idea? :)

[dmlloyd]

I believe most of the work to support this is in place. We now just need to define the right API for each ConfigSource to access their predecessors.

An issue here is that we don't control the ConfigSource creation, since this is done by the SPI and the user might give us their own instances via ConfigSourceProvider, so a convention constructor that exposes the context like we have for the interceptor might be out of the picture.

Maybe we could support a @PostConstruct method call. We know that the ConfigSource is not usable while the Config is being initialized, so calling a @PostConstruct should be safe. The method may take a ConfigSourceInterceptorContext or even a list of higher priority ConfigSource to provide access to the current configs and do initialization.

@dmlloyd Any other brilliant idea? :)

Remember that we aren't really constrained by the spec: if it doesn't do what we want, we don't have to comply exactly to it.

But in any case there's nothing preventing our users for implementing sources using the interceptor API rather than the CS API.

[radcortez]

There is an issue reported in the spec, but apart from a couple of ideas here and there, there is no actual proposal, so we can figure out something and make a proposal.

The idea was mostly to not introduce breaking changes, but maybe we can introduce a different Provider that works side by side with the one in the spec and we can make it as we want.

I'll try to come up with a few prototypes.

[kenfinnigan]

If we need to break spec/TCK to move things forward, then do so

[radcortez]

Hey @kenfinnigan welcome back :)

We have a proposal here: #313

[radcortez]

Done in #313.