Collection of Object support in @ConfigProperties

[hrstoyanov]

Describe the bug
Quarkus YAML configuration should be able to parse something like this:

@ConfigProperties(prefix = "prefix")
public interface MyConfig {

   @ConfigProperty(name = "inputs")
   List<Input> inputs();

   @ConfigProperty(name = "outputs")
   Output[] outputs();
}

Expected behavior
Quarkus config should be able to parse YAML with variable length collection of sub-structures.

Actual behavior
It appears Quarkus config fails with collection fields liek above, complaining it needs converter!

To Reproduce
A realistic YAML config example with no known workaround, is presented in this thread where the issue is discussed with @geoand and @gastaldi

[geoand]

Thanks for reporting

[gastaldi]

I created a project with the data you provided and I couldn't reproduce the error:

hello-test.zip

UPDATE: The error occurs when you try to inject the configuration property:
Eg. When adding the following snippet to my GreetingResource class, the error below occurs:

    @Inject
    @ConfigProperty(name = "sql_proxy")
    ProxyConfiguration configuration;

java.lang.IllegalArgumentException: SRCFG00013: No Converter registered for interface org.acme.ProxyConfiguration
	at io.smallrye.config.SmallRyeConfig.lambda$getConverter$2(SmallRyeConfig.java:289)
	at java.base/java.util.concurrent.ConcurrentHashMap.computeIfAbsent(ConcurrentHashMap.java:1737)
	at io.smallrye.config.SmallRyeConfig.getConverter(SmallRyeConfig.java:286)
	at io.smallrye.config.SmallRyeConfig.lambda$getOptionalConverter$1(SmallRyeConfig.java:271)
	at java.base/java.util.concurrent.ConcurrentHashMap.computeIfAbsent(ConcurrentHashMap.java:1705)
	at io.smallrye.config.SmallRyeConfig.getOptionalConverter(SmallRyeConfig.java:270)
	at io.smallrye.config.SmallRyeConfig.getOptionalValue(SmallRyeConfig.java:206)
	at io.quarkus.arc.runtime.ConfigRecorder.validateConfigProperties(ConfigRecorder.java:37)
	at io.quarkus.deployment.steps.ConfigBuildStep$validateConfigProperties1249763973.deploy_0(ConfigBuildStep$validateConfigProperties1249763973.zig:328)
	at io.quarkus.deployment.steps.ConfigBuildStep$validateConfigProperties1249763973.deploy(ConfigBuildStep$validateConfigProperties1249763973.zig:40)
	at io.quarkus.runner.ApplicationImpl.doStart(ApplicationImpl.zig:543)
	at io.quarkus.runtime.Application.start(Application.java:90)
	at io.quarkus.runtime.ApplicationLifecycleManager.run(ApplicationLifecycleManager.java:97)
	at io.quarkus.runtime.Quarkus.run(Quarkus.java:62)
	at io.quarkus.runtime.Quarkus.run(Quarkus.java:38)
	at io.quarkus.runtime.Quarkus.run(Quarkus.java:104)
	at io.quarkus.runner.GeneratedMain.main(GeneratedMain.zig:29)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at io.quarkus.runner.bootstrap.StartupActionImpl$3.run(StartupActionImpl.java:134)
	at java.base/java.lang.Thread.run(Thread.java:834)

[gastaldi]

Make sure you're using these imports in your MyConfig class:

import io.quarkus.arc.config.ConfigProperties;
import org.eclipse.microprofile.config.inject.ConfigProperty;

[gastaldi]

When I do:

    @Inject
    ProxyConfiguration configuration;

The config is injected, but the converter error is thrown when I call configuration.inputs() or outputs()

[hrstoyanov]

@gastaldi Exactly,
Calling inputs() or outputs() causes the "need a convertor" exception.
so this is still a gap - Quarkus should be able to handle variable list of sub-structures in a YAML config .

[hrstoyanov]

so what are the plans here? If this is going to be a major effort and take some time, do you have alternatives to suggest?

[geoand]

I haven't had time to look into yet unfortunately...

It should be on the top of my list when I'm back from PTO

[geoand]

@radcortez does the new MP Config handle this case somehow?

I see that in the case of YAML like in the example @gastaldi uploaded, the value is some kind of json object that I assume can be converted into the list of objects we want, so I would like to know if there is anything that I could reuse

[radcortez]

We have similar issues reported in SR Config:
smallrye/smallrye-config#447
smallrye/smallrye-config#436

The main issue here is that MP Config is backed by String key value pairs. The Converter API does not have any context information of additional keys. The conversion happens in a key/value fashion, making it very unfriendly to retrieve related configuration in the case of multi nested lists, maps or composed objects.

What we do is to expose a parent key that contains the entire YAML context that you can use to convert to what you need. Please, check here: https://github.com/smallrye/smallrye-config/blob/master/sources/yaml/src/test/java/io/smallrye/config/source/yaml/YamlConfigSourceTest.java

[geoand]

I'll check it out, thanks!

[geoand]

I see that in that example, under the covers the new ConfigMapping API is being used.
My plan is to rely on that for this use case - is that OK with you @radcortez ?
I understand it's experimental, but I since this will be far away from users, we can easily adapt on the Quarkus side of that API breaks.
WDYT?

[geoand]

Hm... Actually I don't think I can use the ConfigMapping API because SmallRyeConfig will not have not have it configured

[radcortez]

In Quarkus any ConfigMappings are automatically added now, so it should be fine.

Anyway, the trick here is the Converter piece: https://github.com/smallrye/smallrye-config/blob/a8c0fab9b056b8205f3b4282b433f63c26f83048/sources/yaml/src/test/java/io/smallrye/config/source/yaml/YamlConfigSourceTest.java#L203-L208

A parent property admin.users contains the entire YAML section as a String, so you can deserialize it in the Converter. For Quarkus, to make this work transparently, we need to generate converters based on the type (if using ConfigProperties / ConfigMapping) that handle the YAML deserialization.

[geoand]

Understood, thanks.

I'll have a closer look sometime this week.

[geoand]

I think I have way to make something like this work.

It involves generating some intermediate classes and some configuration that will be fed to SnakeYaml in order to use it to generate a proper converter.
It will definitely have issues, but should work fine for basic cases.

I'll start working on it tomorrow unless something of higher priority comes up in the meantime.

[geoand]

@hrstoyanov with #14123 (which will hopefullt be part of 1.11.CR1) you will get almost exactly what you described.
See

quarkus/extensions/config-yaml/deployment/src/test/java/io/quarkus/config/yaml/deployment/InterfaceConfigPropertiesTest.java

Line 82 in 0bc9f81

public interface SqlConfiguration {

for an example of what the supported configuration will look like.

[hrstoyanov]

@geoand thanks a lot! ... waiting for 1.11.CR1 to test it out!
Did you also expand on the Quarkus Configuration tutorial with the new capabilities, so other developers can benefit too?

[geoand]

You are welcome!

That is one thing I didn't do... I'll add some documentation today.
Good idea!

[geoand]

#14158 adds some documentation