Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support hot reloading of configuration #67

Open
5 of 6 tasks
JelteF opened this issue Nov 1, 2021 · 13 comments
Open
5 of 6 tasks

Support hot reloading of configuration #67

JelteF opened this issue Nov 1, 2021 · 13 comments
Assignees
Labels
enhancement New feature or request 🔥Hot Reload Tasks related to DAB's Hot Reload feature proposal

Comments

@JelteF
Copy link
Contributor

JelteF commented Nov 1, 2021

Allow specifiying something like a --watch to make the runtime watch the provided configs and reload when they change.

Relevant comment from @jarupatj in #50:

You should look at using IOptionsMonitor
https://docs.microsoft.com/en-us/aspnet/core/fundamentals/configuration/options?view=aspnetcore-5.0#options-interfaces

#892

Tasks

Preview Give feedback
  1. do-not-merge refactor
    ayush3797 seantleonard
  2. enhancement 🔥Hot Reload
    seantleonard
  3. enhancement 🔥Hot Reload
    aaronburtle
  4. enhancement 🔥Hot Reload
    RubenCerna2079
  5. refactor
    aaronburtle
  6. refactor 🔥Hot Reload
    aaronburtle
@seantleonard
Copy link
Contributor

can this be closed with #50 merged? @JelteF

@JelteF
Copy link
Contributor Author

JelteF commented Nov 29, 2021

I don't think this should be closed, because it is about the more user facing version of hot roloading configs. One that does not require dotnet watch but simply run the binary with a --watch flag.

However, I don't think this should be an M0 milestone. Work on dev tooling team can start just fine without this support. So I'm marking this as M1 instead now. @jarupatj please comment on this if you disagree with that for some reason.

@JelteF JelteF modified the milestones: M0, M1 Nov 29, 2021
@Aniruddh25
Copy link
Contributor

Solving this #189 will help here.

@Aniruddh25 Aniruddh25 modified the milestones: M1, M2 Apr 16, 2022
@Aniruddh25
Copy link
Contributor

Should we support hot reloading at all? Once runtime is bootstrapped, should we allow modifications to it?
In M1, we might expect customers to restart the docker image.
For M2, we will throw away the container and use a different one from the warm pool whenever any config changes.

Moving to M2 for now so we can discuss more.

@aaronburtle
Copy link
Contributor

Modified this issue to cove hot reloading of the configuration file. This will clear duplicate issues, which will all be tracked here.

@aaronburtle aaronburtle changed the title Support hot reloading Support hot reloading of configuration Apr 3, 2023
@aaronburtle
Copy link
Contributor

Would be great to have a switch on the CLI, something like --watch that will automatically reload the configuration file (and thus re-generate the in-memory schema) whenever the file is changed. Something like https://learn.microsoft.com/en-us/dotnet/core/tools/dotnet-watch

That would greatly improve development velocity.

@aaronburtle
Copy link
Contributor

In order to set the LogLevel based on the mode in the runtimeconfig we need to use a configuration provider that can reload the configuration after updating the Loglevel.

More information can be found here: https://learn.microsoft.com/en-us/dotnet/core/extensions/logging?tabs=command-line

@aaronburtle aaronburtle removed this from the Mar2023 milestone Apr 3, 2023
@Aniruddh25 Aniruddh25 added the enhancement New feature or request label Jul 10, 2023
@aaronburtle aaronburtle modified the milestones: 0.9rc, 0.10rc Sep 25, 2023
@Aniruddh25 Aniruddh25 modified the milestones: 0.10rc, 0.11rc Nov 7, 2023
aaronburtle added a commit that referenced this issue Nov 11, 2023
## Why make this change?

Related to, covering the runtime section of the config only
#67

Because we are not able to make easy use of the `IOptionsMonitor` which
would have allowed for the updating of individual sections of the config
automatically (ie: just refreshing the `runtime options`), we monitor
and reload the entire file. However, we currently do not have support
for reloading the sections outside of the `Runtime` section of the
config. Because of this, we will not document and announce this feature
until the other sections can be safely reloaded.

Additionally, certain options are used to configure services at startup,
and despite refreshing the `RuntimeConfig`, those services will still be
configured with the values from initialization on startup. This means
adding a mechanism for refreshing those services to make use of the
updated options after a Hot Reload, and that work will come in a future
PR.



## What is this change?

Adds a config file watcher during startup which will monitor for any
file changes to the config and then trigger an update event in the case
that the file is changed.

We instantiate the config file watcher when the runtime provider is
first asked to get a config. We then register a method owned by the
runtime config provider that will be called back to in the case that we
wish to hot reload. However, we only do this for when we start the
service in local development. So that means when we are `not` late
configured, and we are in a Host Mode of Development on startup. In
other cases we do not watch the config file for changes, and do not make
this callback in the case that the config file does change.

To get the location of the file that we are going to watch we use the
config loader's file system and stored data regarding the config file.

## How was this tested?

Unit testing was added to verify correct hot reloading of the runtime
options that we intend to have changing.

## Sample Request(s)

Any valid request after the change ought to work.

---------

Co-authored-by: Sean Leonard <[email protected]>
aaronburtle added a commit that referenced this issue Nov 15, 2023
## Why make this change?

This is related to the following Hot Reload PR
#1726

and this issue #67


It adds a design doc to get feedback and iterate on how to implement the
Hot Reloading feature. Hot Reloading will be on going work as all of the
needed features to support it entirely are added.

## What is this change?

Adds the design doc and a basic diagram for the Hot Reload feature.

---------

Co-authored-by: Aniruddh Munde <[email protected]>
@seantleonard
Copy link
Contributor

hot-reload property in config?

@aaronburtle
Copy link
Contributor

I have created a number of sub-tasks to help track the progress on Hot Reload.

Currently, we are refreshing the RuntimeConfig to match any updates that we see in the associated config file when we are in the local scenario and start the engine in developer mode.

To facilitate extending Hot Reload in the future, a refactor of the RuntimeConfigProvider is taking place

#1966

as well as some changes to make hot reload work with the new multi config code

#1967

Additionally, one of the key features we want support for hot reloading is reloading of the log-level, and so this property is being added to the config and we have a task to track being able to hot reload this property in both dev and production mode

#1968

Additionally, there are some tasks for updating those settings which depend on the config only at startup, such as the OpenApiDoc and Auth settings

#1869

#1969

@seantleonard
Copy link
Contributor

Moving this + hot reload related tasks to post GA -> tracking via 1.2rc milestone.

@seantleonard seantleonard added the 🔥Hot Reload Tasks related to DAB's Hot Reload feature proposal label May 14, 2024
@seantleonard seantleonard modified the milestones: 1.2rc, Feature Backlog May 17, 2024
@seantleonard
Copy link
Contributor

The top-level task tracking hot reload can remain in backlog as we address sub-tasks. Ideally a sprint's tracked work items should be resolvable. This task has so many pre-reqs , we need to resolve those first.

@aaronburtle aaronburtle modified the milestones: Backlog, 1.3 Sep 4, 2024
aaronburtle added a commit that referenced this issue Oct 11, 2024
## Why make this change?

closes #2415
Relates to #67
Relates to #2323
Relates to #1869

This change adds event handling for the dependencies which are set
during startup, and refreshing their internal state to match the states
that are updated during a hot-reload. In order to help facilitate the
final portions of the configuration file which need to be handled in
hot-reload scenarios, we need these dependencies to be updated without
any stale values from before the hot-reload. And we enforce an order of
this update to make sure that if B depends on A, that the refresh
happens first with A, then with B.

## What is this change?

There are a number of classes which are instantiated and registered with
the Service Provider on startup using values that depend on the
'RuntimeConfig'. These are registered as singletons. Therefore, during a
hot-reload scenario, we need to update those states because the
'RuntimeConfig' has changed, and the values that those objects were
instantiated with are now potentially stale. To handle this updating,
when one of these singleton services is constructed, we subscribe an
update function with an Event Handler that is provided to the
constructor. We also save internally any arguments provided to the
constructor that were needed during construction so that they will be
available and usable during the updating.

The RuntimeConfig Loader then sequentially sends notifications for all
of these subscribed events to inform them of the hot-reload process.
This happens in a single function with a single thread so that the order
of operation is enforced, with the thread executing each update one at a
time.


Note that we save the handler internally in some of the classes that we
will be calling back to do configurations during hot-reload. Generally
speaking, this happens in the factory classes when their
re-configuration during hot-reload requires them to create other classes
that will need to be re-configured during a hot-reload. Those classes
then need to have a handler provided to their constructor so that they
can properly subscribe to hot-reload events.

## How was this tested?

Manually verified the updating process by modifying the config while DAB
is running and then verifying that the dependent objects successfully
refresh to match the new values in the config, and do so in the desired
order. The following issue tracks specifically testing this behavior:
#2416

## Sample Request(s)

Any change to the config used on startup will demonstrate the hot-reload
functionality added in this PR.

---------

Co-authored-by: Sean Leonard <[email protected]>
@abhishekkumams abhishekkumams modified the milestones: 1.3, 1.4 Nov 6, 2024
@abhishekkumams abhishekkumams removed this from the 1.4 milestone Nov 25, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request 🔥Hot Reload Tasks related to DAB's Hot Reload feature proposal
Projects
None yet
Development

No branches or pull requests

8 participants