Support for Map<String, T> and List<T>

[vsevel]

It would useful to support custom converters for Map<String, T> and List<T>, specifically when using the yaml format.
For instance:

    deploy:
      entities:
        - name: one
        - name: two
          option: foo

with:

public class DeployConfig {
    public List<DeployEntityConfig> entities;
}
public class DeployEntityConfig {

    public String name;
    public Optional<String> option;
}

if I try to run this as-is, I get: SRCFG00013: No Converter registered for class ....deploy.config.DeployEntityConfig

If I create a custom converter, the convert method gets called on {"entities": [{"name": "one"}, then on {"name": "two", then on "option": "foo"}]}, which is not appropriate to create DeployEntityConfig.

If I change the design to be:

    deploy:
      entities:
        one: {}
        two:
          option: foo

and

public class DeployConfig {
    public Map<String, DeployEntityConfig> entities;
}
public class DeployEntityConfig {

    public Optional<String> option;
}

I get the error SRCFG00013: No Converter registered for interface java.util.Map.

The closest I can get is encoding the structure in a property string. For instance a json document:

    deploy:
      entities:
        - {"name": "one"}
        - {"name": "two", "option": "foo"}

But I think we can do better, especially when using the yaml configuration flavor.

[radcortez]

I believe this is the same issue reported in #436.

One of the issues with have with this is that the Converter API does not have any context information of additional keys. The conversion happens in a key/value fashion, making it very unfriendly to retrieve related configuration in the case of multi nested lists or maps.

That is why you get the entire yaml section for the top level key, because it will allow you to get the entire context of your configuration and provide the conversion required. Check the following test:

smallrye-config/sources/yaml/src/test/java/io/smallrye/config/source/yaml/YamlConfigSourceTest.java

Lines 66 to 79 in 6665fbd

	@Test
	void config() throws Exception {
	SmallRyeConfig config = new SmallRyeConfigBuilder()
	.withSources(new YamlConfigSource("yaml", YamlConfigSourceTest.class.getResourceAsStream("/example-216.yml")))
	.withConverter(Users.class, 100, new UserConverter())
	.build();
	final Users users = config.getValue("admin.users", Users.class);
	assertEquals(2, users.getUsers().size());
	assertEquals(users.users.get(0).getEmail(), "[email protected]");
	assertEquals(users.users.get(0).getRoles(), Stream.of("Moderator", "Admin").collect(toList()));
	assertEquals("[email protected]", config.getRawValue("admin.users.[0].email"));
	}

Does that help?

[vsevel]

hello @radcortez
your example works, but only because you access the value through config.getValue(String, Class);
it would probably work also if you did:

    @ConfigProperty(name = "admin.users")
    Users users;

but I was not able to get it to work within a @ConfigProperties object, such as:

@ConfigProperties(prefix = "admin")
public class Admin {

    public List<User> users;

or

@ConfigProperties(prefix = "admin")
public class Admin {

    public Users users;

did I miss something? is there a way to do this?

[vsevel]

I have changed my Admin object to have: public Optional<String> users;

If I have:

      users:
        - name: bob
        - name: joe
          age: 45

I get the string as: Optional[{"users": [{"name": "bob"}, {"name": "joe", "age": !!int "45"}]}]

but if I change it to a map structure:

      users:
        bob: {}
        joe:
          age: 45

then I get an Optional.empty

why would not this work?

[radcortez]

I guess the issue is that Quarkus @ConfigProperties has some kind of issue with this.

It is tricky to support @ConfigProperties when the code is in Quarkus and most of the config resolution code is in SR Config. Ideally we would like to move @ConfigProperties to SR Config. Actually we may replace it with @ConfigMapping from SR: https://smallrye.io/docs/smallrye-config/mapping/mapping.html.

Anyway, let me try to have a look.

[radcortez]

but if I change it to a map structure:

      users:
        bob: {}
        joe:
          age: 45

then I get an Optional.empty

why would not this work?

The issue here is that you need to tell the YAML parser where the list starts and you need to use a dash for it. For instance, parsing the original structure with snakeyaml will yield the following:

{users={bob=null, joe=null, age=45}}

They are just treated as different objects.

If you add a single dash:

users:
      - bob:
        joe:
        age: 45

Then you get the expected result: {users=[{bob=null, joe=null, age=45}]} and the objects are now inside a list. And it is translated to {"users": [{"bob": !!null "null", "joe": !!null "null", "age": !!int "45"}]}in the config properties representation. Hope it helps!

[vsevel]

The issue here is that you need to tell the YAML parser where the list starts

hi @radcortez . there are not lists in this example, just structs. in json this would be:

{
  "users": {
    "bob": {},
    "joe": {
      "age": 45
    }
  }
}

[radcortez]

Ah, probably because we only assign the outer most key if the inner keys are part of a list. Let me have a look.