Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Possibility to nullify config used in ConfigMapping #1291

Open
manofthepeace opened this issue Jan 14, 2025 · 12 comments
Open

Possibility to nullify config used in ConfigMapping #1291

manofthepeace opened this issue Jan 14, 2025 · 12 comments

Comments

@manofthepeace
Copy link
Contributor

This heavily relates to; #845

I have some ConfigMapping that are feature related. Those feature are enabled/disabled via .enabled=[true|false]

those ConfigMapping are used "lazily" in ApplicationScoped bean that are using Instance<> and LookupIfProperty() mechanism in quarkus.

I wish that if the enabled flag is false, all of feature.**.* get nullified to I don't have a startup error. I do not know / want to specify manually all the configs to be nullyfied, nor I want to force people to go and remove all their feature related config, they should just toggle the enabled/disable so they don't have to fully reconfigure the feature.

I tried using an interceptor, but it does not work, as the ConfigMapping does not "load" values "/ usees interceptor..

but in soft the logic I had taht I would kike somehow to apply to the config mapping is the following;

public class SubSystemDisabler implements ConfigSourceInterceptor {
    private static final long serialVersionUID = 1L;
  
    private static final List<String> subSystems = List.of("feature", "feature_b", "feature_b");
    private static final UnaryOperator<String> ENABLED = name -> name.concat(".enabled");
    private static final UnaryOperator<String> STARTS_WITH_ANY = name -> subSystems.stream()
            .filter(name::startsWith).findFirst().orElse(null);
    @Override
    public ConfigValue getValue(ConfigSourceInterceptorContext context, String name) {
        String handle = STARTS_WITH_ANY.apply(name);
        if (handle != null) {
            String enabled = ENABLED.apply(handle);
            if (enabled.equals(name)) {
                return context.proceed(name);
            }
            ConfigValue subsystemEnabled = context.proceed(enabled);
            if (subsystemEnabled != null && !Boolean.parseBoolean(subsystemEnabled.getValue())) {
                log.debug("{} is false, disabling {}", enabled, name);
                return null;
            }
        }
        return context.proceed(name);
    }
}

Hope it is somewhat clear, I have created a little app here https://github.com/manofthepeace/opt-configmapping to show the structure, with a super simple mapping, in my app the config are far more complex with lots of nesting.

Obviously the above interceptor is wrong for the job, but its what I would like to achieve in a nutshell. Options welcome :) thanks

@radcortez
Copy link
Member

In Quarkus you could wrap the mapping in a CDI Bean and use the annotations @IfBuildProfile or @IfBuildProperty to enable / disable the bean. This will also affect the declared mapping as:

@IfBuildProperty(name = "please.enable", stringValue = "yes")
@ApplicationScoped
public class MyMappingBean {
    @Inject
    MyMapping myMapping;
}

@ConfigMapping(prefix = "my.mapping")
public interface MyMapping {
    String value();
}

Would this work for you?

Another option is to use a SmallRyeConfigBuilderCustomizer and directly control which mappings are available in

public Map<Class<?>, Set<String>> getMappings() {
return mappings;
}
, but you are not going to have access to the full config context, because sources are not yet available here. We would require some sort of hook here:
public ConfigMappingContext(
final SmallRyeConfig config,
final Map<Class<?>, Set<String>> mappings) {
this.config = config;
matchPropertiesWithEnv(mappings);
for (Map.Entry<Class<?>, Set<String>> mapping : mappings.entrySet()) {
Map<String, Object> mappingObjects = new HashMap<>();
for (String prefix : mapping.getValue()) {
applyPrefix(prefix);
mappingObjects.put(prefix, constructMapping(mapping.getKey(), prefix));
}
this.mappings.put(mapping.getKey(), mappingObjects);
}
}

@manofthepeace
Copy link
Contributor Author

I am using quarkus, and I cannot use build time stuff, as I cannot start supporting X builds with X different combinations of feature enablement.

Another thing is that when let say feature X is enabled, a ConfigSourceFactory loads other configs that fills the config mapping (+ user stuff configs), hence why when feature gets disabled, the ConfigMapping becomes partially filled, and complains about the missing stuff.

That's why my idea was to wrap all the configs in an optional, like in the example repo, and have a way to sort of mark them all as empty, so the config looks "ok" i.e. optional is empty, and feature.enabled=false.

@manofthepeace
Copy link
Contributor Author

In relation to #845 I wonder if there could be something like this;

@ConfigMapping(prefix="conf", optional=true)

and a api sort of Optional<Server> serverConfig = config.getOptionalConfigMapping(Server.class);

Unsure how that work work for quarkus/cdi cases though..

@radcortez
Copy link
Member

I am using quarkus, and I cannot use build time stuff, as I cannot start supporting X builds with X different combinations of feature enablement.

The customizer is SmallRye Config only, so it would work independently of Quarkus.

@ConfigMapping(prefix="conf", optional=true)

This may not be a good idea. We use the declared types to determine the behaviour of the mapping. CDI would be a problem, because you cannot use Optional at the injection point, so you could only retrieve these optional mappings with the programmatic API.

Also, when a declaration is reused in a tree as:

@ConfigMapping(optional = true)
interface OptionalMapping {

}

@ConfigMapping
interface NonOptionalMapping {
  OptionalMapping optionalMapping();

  Optional<OptionalMapping> optionalMappingAsOptional();
}

This looks a bit weird to me. Would the type declaration override the annotation or we have to validate that you always wrap the mapping in an Optional.

@manofthepeace
Copy link
Contributor Author

Thanks for your answer.

The customizer is SmallRye Config only, so it would work independently of Quarkus.

I actually missed that from your previous message. That seems like it could be a really good solution, but as you mentioned if I do not have access to the raw config even, I cannot really know if a certain feature is available or not. When you said some hook in ConfigMappingContext.java I am not sure what it actually means though, does it mean it could be a way to get the config? A change in SR Config?

about the optional I said above;

CDI would be a problem, because you cannot use Optional at the injection point

Agree, no idea on that. I was naively thinking about something like Instance and use something like isResolvable or use something like Provider<> with it.

This looks a bit weird to me. Would the type declaration override the annotation or we have to validate that you always wrap the mapping in an Optional.

it does look weird. and I do not exactly know how it work work in a fully logical way. My goal is just to prevent half-provided config to break the app, i.e. make a half present optional be simply sort of ignored even if some configSource provides some values.

From what I understand and read, looks like the customizer is quite close of being able to make this work, and actually ignore configMappings.

@radcortez
Copy link
Member

I actually missed that from your previous message. That seems like it could be a really good solution, but as you mentioned if I do not have access to the raw config even, I cannot really know if a certain feature is available or not. When you said some hook in ConfigMappingContext.java I am not sure what it actually means though, does it mean it could be a way to get the config? A change in SR Config?

Yes, that does not exist now. We would need to add it.

How is the configuration to enable / disable is defined? Is it the user that provides the configuration? Or it is some other logic that sets that configuration to enable / disable? Couldn't that logic be written in the customizer to add / remove mappings?

@manofthepeace
Copy link
Contributor Author

It's a little convoluted but let me try to explain my flow;

1- there are features, feature can be enabled/disabled via a <feature>.enabled=<true|false> via env var in k8s
2- via a ConfigSourceFactory, check enabled feature isEnabled (via context.getValue and load their correspondent config from outside (or just external featureX.properties). If feature is disabled we dont bother loading the configs for the feature
3-User can override any properties via env_var or provide extra ones, or disable the feature altogether

My goal is that if the user wants to temporarily disable a feature he can jsut change the env var to featureX.disabled=false. Without bothering removing his extra featureX.customizaions="XXX". so when ready it can just be toggled back to true.

Currently if the feature is toggled FALSE, the ConfigSourceFactory will skip getting the config for that service, and the extra params that remain in a user conf will fill some of the things in the Optional (see linked project in OP) then startup will fail, until the user redeploys by commenting all the FeatureX properties in its env var.

Its not the end of the world obviously, but it would be nice to not have the risk of a feature changing because params were not reapplied properly, besides the basic enabled true false. Basically having a single feature flag, and not 1+ whatever is manually configured and/or overridden

@radcortez
Copy link
Member

1- there are features, feature can be enabled/disabled via a <feature>.enabled=<true|false> via env var in k8s

Can you only enable / disable from env var?

@manofthepeace
Copy link
Contributor Author

Yes, some are by default to "true" in the application.properties. Some needs to be toggled via env var.

@radcortez
Copy link
Member

Ok, for now, you can use the same validation trick we discussed in Quarkus. Create your customizer and internally, create a new SmallRyeConfig instance from SmallRyeConfigBuilder. You just need to be careful with the internal instance and not discover / use the current customizer, or you will get in an endless loop.

With the config, you can now query for the expected configuration and add / remove mappings from the original builder. The downside is that you create some additional objects (and read the sources multiple times), but it is usually fast and maybe good enough until we have a proper solution.

@manofthepeace
Copy link
Contributor Author

I tried the above and unfortunately I cannot remove the Customizer. I tried a little something like this

public class CustomConfigBuilder implements SmallRyeConfigBuilderCustomizer {
    @Override
    public void configBuilder(SmallRyeConfigBuilder builder) {
        var test = builder.isAddDiscoveredCustomizers();
        var conf = builder.build();
        boolean isEnabled = conf.getValue("feature.enabled", boolean.class);
        builder.withMappingIgnore("feature");
    }
}

compare to other the customizer does not have a setAddDiscoveredCustomizers() and the variable above it "true" so I get into the stackoverflow.

@manofthepeace
Copy link
Contributor Author

I tried a bit more by modifying SR Config without much success;;

changes in SR config
1- customizers ArrayList to CopyOnWriteArrayList (customizer called 6 times and looks like it triggers a CoucurrentModification exception)
2- add api setAddDiscoveredCustomizers and withCustomizerIgnore

end code looks like this;

    public void configBuilder(SmallRyeConfigBuilder builder) {
        log.error("xxxxxxxx I AM HERE xxxxxxx ");
        var conf = builder.setAddDiscoveredCustomizers(false).withCustomizerIgnore(this).build();
        builder.setAddDiscoveredCustomizers(true);
        boolean isEnabled = conf.getValue("feature.enabled", boolean.class);
        if (!isEnabled) {
            builder.withMappingIgnore("feature.**");
        }
    }

I am using withMappingIgnore as noted in the javadoc, but still getting java.util.NoSuchElementException for each and every props inside the configmapping it is still trying to build via "buildMappings"

logs

2025-01-24 21:23:49,495 ERROR [org.acme.con.CustomConfigBuilder] (main) xxxxxxxx I AM HERE xxxxxxx
2025-01-24 21:23:49,577 INFO  [org.acme.con.SubSystemsConfigSourceFactory] (main) Loading feature 1 configurations
2025-01-24 21:23:56,468 INFO  [org.acme.con.SubSystemsConfigSourceFactory] (main) Loading feature 1 configurations
2025-01-24 21:23:56,517 ERROR [org.acme.con.CustomConfigBuilder] (main) xxxxxxxx I AM HERE xxxxxxx
2025-01-24 21:24:00,165 ERROR [org.acme.con.CustomConfigBuilder] (main) xxxxxxxx I AM HERE xxxxxxx
2025-01-24 21:24:03,602 ERROR [org.acme.con.CustomConfigBuilder] (main) xxxxxxxx I AM HERE xxxxxxx
2025-01-24 21:24:09,548 DEBUG [org.hib.val.int.uti.Version] (Quarkus Main Thread) HV000001: Hibernate Validator 8.0.2.Final
2025-01-24 21:24:09,873 ERROR [org.acme.con.CustomConfigBuilder] (Quarkus Main Thread) xxxxxxxx I AM HERE xxxxxxx
2025-01-24 21:24:09,873 ERROR [org.acme.con.CustomConfigBuilder] (Quarkus Main Thread) xxxxxxxx I AM HERE xxxxxxx
2025-01-24 21:24:09,880 INFO  [org.acme.con.SubSystemsConfigSourceFactory] (Quarkus Main Thread) Loading feature 1 configurations
2025-01-24 21:24:09,880 INFO  [org.acme.con.SubSystemsConfigSourceFactory] (Quarkus Main Thread) Loading feature 1 configurations
2025-01-24 21:24:13,994 INFO  [org.acme.con.SubSystemsConfigSourceFactory] (Quarkus Main Thread) Loading feature 1 configurations
2025-01-24 21:24:13,994 INFO  [org.acme.con.SubSystemsConfigSourceFactory] (Quarkus Main Thread) Loading feature 1 configurations
2025-01-24 21:24:14,023 ERROR [io.qua.run.Quarkus] (Quarkus Main Thread) Error running Quarkus: java.lang.ExceptionInInitializerError
Caused by: io.smallrye.config.ConfigValidationException: Configuration validation failed:
    java.util.NoSuchElementException: SRCFG00014: The config property feature.test is required but it could not be found in any config source

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants