Bump smallrye config to 1.9.1 and provide support for ConfigSourceFactory

[vsevel]

Description
smallrye config has been released as 1.9.1 with the ability to iterate over properties from previous config sources smallrye/smallrye-config#402

this requires ConfigSourceFactory to be supported by quarkus, instead of ConfigSource only.

this is required to make progress on #10390 and #12243

Implementation ideas
Ability to recognize an additional ConfigSource seems to be related to RunTimeConfigurationSourceBuildItem, later used by ConfigBuildSteps to create a ConfigSourceProvider. So it sounds like there should be RunTimeConfigurationSourceFactoryBuildItem?

It seems however that only the VaultProcessor uses the RunTimeConfigurationSourceBuildItem approach. I looked again at the consul, spring and k8s config sources and the approach taken is with a RunTimeConfigurationSourceValueBuildItem to work at the bootstrap level.

even with the work provided by smallrye config 1.9 (smallrye/smallrye-config#196 and smallrye/smallrye-config#402), I am not convinced we will get a completely clean solution (I believe I will still have to do some property parsing and databinding). in that sense I am wondering if we could revisit using a BOOTSTRAP config phase like some of the other config sources #9989, or reopen #4848.

/cc @geoand @dmlloyd @radcortez

[geoand]

This one is really up to Roberto and David.

If they want to use this approach I can certainly update the Consul, Spring Cloud Config and Kubernetes Config extensions to use the new method.

[quarkusbot]

/cc @kenfinnigan, @phillip-kruger, @jmartisk, @radcortez, @Ladicek

[radcortez]

The bump is here: #12501.

We can certainly provide a build step for the factory approach.

Right now, what do you think we are missing?

[vsevel]

hello @radcortez

Right now, what do you think we are missing?

in an ideal ideal world the VaultConfigSource would be able to use an @ApplicationScoped VaultKVSecretEngine CDI bean, which would be initialized using the VaultBuildTimeConfig and VaultRuntimeConfig config objects assembled from previous config sources.

but short term we can continue with what we had discussed previously (and I already started implementing it in master...vsevel:vault_config). and in that case I am only missing the additional RunTimeConfigurationSourceFactoryBuildItem build step.

I am just not sure how far we are going to be able to go in terms of simplification. a review from yourself would be certainly very helpful after I finish the first draft of rework.

[vsevel]

If they want to use this approach I can certainly update the Consul, Spring Cloud Config and Kubernetes Config extensions to use the new method.

@geoand I was thinking the other way around for a moment ;-) I like the simplicity of what you did for the consul config.

[geoand]

Even better for me then, I don't have to do anything 😆

[radcortez]

in an ideal ideal world the VaultConfigSource would be able to use an @ApplicationScoped VaultKVSecretEngine CDI bean, which would be initialized using the VaultBuildTimeConfig and VaultRuntimeConfig config objects assembled from previous config sources.

At this time, this is not possible because the ConfigSource lifecycle is managed by the Config and not CDI, so we cannot use injections in a ConfigSource. In certain cases, you may call the programmatic CDI API to retrieve a bean, but not for initialisation blocks, since Config objects are built before CDI Beans.

I already thought about this in the past. It may be possible to let CDI manage ConfigSource and then late bind them to the Config. It will probably work in most situations, but if you try to access the Configuration before CDI init is done, you may miss configuration from these CDI ConfigSource managed beans.

Going forward, I'm checking on how we can simplify the VaultSource.

[vsevel]

thanks for this analysis @radcortez
a solution would be to use new dedicated phase (e.g. CONFIG) for extensions initializing config sources. it would be assumed that those CONFIG extensions would only have access to lower level config sources (file, system property and environment).
but that is what the BOOTSTRAP phase is for (as described by the javadoc These values are used to configure ConfigSourceProvider implementations)?
is there a reason why we do not want to use BOOTSTRAP for the Vault Config Source?

[radcortez]

The only possible phase where this could work is BOOTSTRAP since it is done before the Config object is initialised.

In reality, with the ConfigSourceFactory approach, you don't really need a Config model class, except for documentation purposes and validation, because you can read the configuration properties directly from the ConfigContext in the factory. It is an option, but maybe there are more.

I'm writing a small prototype to see how this should be wired and figure out any missing pieces in the API if required. I'll post here when I'm done.

[vsevel]

because you can read the configuration properties directly from the ConfigContext in the factory.

the only drawback is that you are losing the databinding I suppose? we would still have to implements methods such as VaultConfigSource.loadRuntimeConfig() for instance.

[radcortez]

If we are using the ConfigSourceFactory approach, yes. This is because the factory init is part of the Config init, meaning that you don't have access to the Config object and use it to bind the config items.

An alternative, it to use the RunTimeConfigurationSourceValueBuildItem. You are able to databind and pass the the config item to the recorder. Internally, this uses a separate Config instance with minimal configuration (only default sources). The ConfigSourceFactory is actually able to look into any registered source.

[vsevel]

An alternative, it to use the RunTimeConfigurationSourceValueBuildItem

would that require using the bootstrap phase? that is what is described in the documentation of this build item:

quarkus/core/deployment/src/main/java/io/quarkus/deployment/builditem/RunTimeConfigurationSourceValueBuildItem.java

Line 14 in cf539e6

* The build step can optionally use a configuration object that uses the {@code BOOTSTRAP} config phase and pass this

or does it mean opening up RunTimeConfigurationSourceValueBuildItem to other usages such as the RUN_TIME config phase?

[radcortez]

Yes, you need to use the bootstrap phase.

[vsevel]

I ended up using a config source provider #13498 (comment)
so this evolution is not needed for vault anymore

[radcortez]

Ok. Thanks :)