data-api-builder
data-api-builder copied to clipboard
Azure
Support hot reloading of configuration
Allow specifiying something like a --watch to make the runtime watch the provided configs and reload when they change.
Relevant comment from @jarupatj in #50:
You should look at using IOptionsMonitor https://docs.microsoft.com/en-us/aspnet/core/fundamentals/configuration/options?view=aspnetcore-5.0#options-interfaces
#892
### Tasks
- [ ] https://github.com/Azure/data-api-builder/pull/1981
- [ ] https://github.com/Azure/data-api-builder/issues/1969
- [ ] https://github.com/Azure/data-api-builder/issues/1869
- [ ] https://github.com/Azure/data-api-builder/issues/1968
- [ ] https://github.com/Azure/data-api-builder/issues/1967
- [ ] https://github.com/Azure/data-api-builder/issues/1966
I don't think this should be closed, because it is about the more user facing version of hot roloading configs. One that does not require dotnet watch but simply run the binary with a --watch flag.
However, I don't think this should be an M0 milestone. Work on dev tooling team can start just fine without this support. So I'm marking this as M1 instead now. @jarupatj please comment on this if you disagree with that for some reason.
Solving this https://github.com/Azure/hawaii-gql/issues/189 will help here.
Should we support hot reloading at all? Once runtime is bootstrapped, should we allow modifications to it? In M1, we might expect customers to restart the docker image. For M2, we will throw away the container and use a different one from the warm pool whenever any config changes.
Moving to M2 for now so we can discuss more.
Modified this issue to cove hot reloading of the configuration file. This will clear duplicate issues, which will all be tracked here.
Would be great to have a switch on the CLI, something like --watch that will automatically reload the configuration file (and thus re-generate the in-memory schema) whenever the file is changed. Something like https://learn.microsoft.com/en-us/dotnet/core/tools/dotnet-watch
That would greatly improve development velocity.
In order to set the LogLevel based on the mode in the runtimeconfig we need to use a configuration provider that can reload the configuration after updating the Loglevel.
More information can be found here: https://learn.microsoft.com/en-us/dotnet/core/extensions/logging?tabs=command-line
Form the GraphQL side you need to implement an ITypeModule ... this will manage the phase out of schemas...
Here you can see an example. https://github.com/ChilliCream/graphql-platform/blob/13349fa4af03514507ed5e3aaf9a771c696b56a6/src/HotChocolate/Fusion/src/Core/DependencyInjection/FileWatcherTypeModule.cs
The TypeModule is registered with the schema.
I have created a number of sub-tasks to help track the progress on Hot Reload.
Currently, we are refreshing the RuntimeConfig to match any updates that we see in the associated config file when we are in the local scenario and start the engine in developer mode.
To facilitate extending Hot Reload in the future, a refactor of the RuntimeConfigProvider is taking place
https://github.com/Azure/data-api-builder/issues/1966
as well as some changes to make hot reload work with the new multi config code
https://github.com/Azure/data-api-builder/issues/1967
Additionally, one of the key features we want support for hot reloading is reloading of the log-level, and so this property is being added to the config and we have a task to track being able to hot reload this property in both dev and production mode
https://github.com/Azure/data-api-builder/issues/1968
Additionally, there are some tasks for updating those settings which depend on the config only at startup, such as the OpenApiDoc and Auth settings
https://github.com/Azure/data-api-builder/issues/1869
https://github.com/Azure/data-api-builder/issues/1969
Moving this + hot reload related tasks to post GA -> tracking via 1.2rc milestone.
The top-level task tracking hot reload can remain in backlog as we address sub-tasks. Ideally a sprint's tracked work items should be resolvable. This task has so many pre-reqs , we need to resolve those first.