ChilliCream
Fusion gateway schema hot reload not working with ConfigureFromFile
Is there an existing issue for this?
- [X] I have searched the existing issues
Product
Hot Chocolate
Describe the bug
When I configure my fusion gateway server from a file, changes to the file don't take effect until the server is restarted.
services
.AddFusionGatewayServer()
.ConfigureFromFile("gateway.fgp", watchFileForUpdates: true)
The issue occurred in version 13.5.1. After downgrading to 13.2.1, when the configuration looked slightly differently, hot reload worked without needing to restart.
Steps to reproduce
- Set up a HotChocolate Fusion gateway server with
services.AddFusionGatewayServer().ConfigureFromFile("gateway.fgp", watchFileForUpdates: true). - Create a valid gateway.fgp file.
- Start the GraphQL Fusion server.
- Confirm in BananaCakePop that the schema was loaded correctly from the gateway.fgp file.
- Replace the gateway.fgp file with a different valid configuration.
- Reload the schema in BananaCakePop and notice that the schema hasn't changed.
- Restart the GraphQL Fusion server.
- Confirm that BananaCakePop now shows the new schema.
Relevant log output
No response
Additional Context?
.ConfigureFromCloud() seems to work. So I assume the error might be related to the file watcher.
Version
13.5.1
Just wanted to add that we have the same issue. I can even delete the gateway.fgp file and it keeps working.
However if I restart it after deleting the file it can't load the schema, so I'm sure that the same file is being loaded and the correct directory is being used.
@michaelstaib any update on this? This problem has been in the package since sep 2023, was wondering when this will be scheduled or fixed.
Right now I don't think there is any out of the box way to make it work, exect for the hochocolate cloud or by restarting the entire gateway every time.
Right now I don't think there is any out of the box way to make it work, exect for the hochocolate cloud or by restarting the entire gateway every time.
@furystormss for us, we decided that restarting the gateway is a sufficient workaround until someone gets around to fixing the bug. As far as I know, hot reloading works on Mac, but not on Windows. So switching OS could fix it for you xD
If that is not an option for you, you can implement your own file watcher and use the ConfigureFromDocument() method instead of ConfigureFromFile() You can use ChilliCreams implementation of the https://github.com/ChilliCream/graphql-platform/blob/14.3.0/src/HotChocolate/Fusion/src/Core/DependencyInjection/GatewayConfigurationFileObserver.cs as inspiration. Just be aware that the bug causing this issue is also somewhere in that class or its dependencies, I think 😅 Although, now that I think about it, the bug could also be somewhere else in which case, writing your own file watcher and registering it with ConfigureFromDocument wouldn't even help.
I heard HC v15 will come with a lot of changes for Fusion. Maybe this will be fixed too.
Same issue here. It looks like at lease the file-watcher works, but the schema does not get refreshed.
This error seems to happen because the file is locked. I wrote a custom implementation, just changing the loading of the document to handle this. This works usually on the third load attempt and is working great for us. :)
Startup.cs
builder.Services
.AddFusionGatewayServer()
.RegisterGatewayConfiguration(
_ => new CustomFileWatcher("./gateway.fgp"))
CustomFileWatcher.cs
using HotChocolate.Fusion;
using HotChocolate.Language;
using HotChocolate.Utilities;
namespace namespace;
public class CustomFileWatcher(string fileName) : IObservable<GatewayConfiguration>
{
public IDisposable Subscribe(IObserver<GatewayConfiguration> observer)
=> new CustomFileConfigurationSession(observer, fileName);
}
public class CustomFileConfigurationSession : IDisposable
{
private readonly IObserver<GatewayConfiguration> _observer;
private readonly string _fileName;
private readonly FileSystemWatcher _watcher;
public CustomFileConfigurationSession(IObserver<GatewayConfiguration> observer, string fileName)
{
var fullPath = System.IO.Path.GetFullPath(fileName);
var directory = System.IO.Path.GetDirectoryName(fullPath) ?? throw new ArgumentException($"Could not find {fileName}");
_observer = observer;
_fileName = fullPath;
BeginLoadConfig();
_watcher = new FileSystemWatcher
{
Path = directory,
Filter = "*.*",
NotifyFilter =
NotifyFilters.FileName |
NotifyFilters.DirectoryName |
NotifyFilters.Attributes |
NotifyFilters.CreationTime |
NotifyFilters.FileName |
NotifyFilters.LastWrite |
NotifyFilters.Size
};
_watcher.Created += (_, e) =>
{
if (fullPath.EqualsOrdinal(e.FullPath))
{
BeginLoadConfig();
}
};
_watcher.Changed += (_, e) =>
{
if (fullPath.EqualsOrdinal(e.FullPath))
{
BeginLoadConfig();
}
};
_watcher.EnableRaisingEvents = true;
}
private void BeginLoadConfig()
=> LoadConfig().FireAndForget();
private async Task LoadConfig()
{
int retryCount = 0;
int maxRetries = 20;
int delayMs = 100; // Start with a small delay
while (retryCount < maxRetries)
{
try
{
// Check if file is locked
if (IsFileLocked(_fileName))
{
await Task.Delay(delayMs);
delayMs *= 2; // Exponential backoff
retryCount++;
continue;
}
var document = await GatewayConfigurationFileUtils.LoadDocumentAsync(_fileName, CancellationToken.None);
_observer.OnNext(new GatewayConfiguration(document));
Console.WriteLine("Configuration loaded successfully");
return;
}
catch (IOException ioEx) when (IsFileLockedException(ioEx))
{
Console.WriteLine($"File access error (likely locked): {ioEx.Message}. Retry {retryCount + 1}/{maxRetries}...");
await Task.Delay(delayMs);
delayMs *= 2; // Exponential backoff
retryCount++;
}
catch (Exception ex)
{
Console.WriteLine($"Failed to load configuration: {ex.Message}");
await Task.Delay(1000); // Delay one second before retrying on general errors
LoadConfig().FireAndForget();
return;
}
}
Console.WriteLine($"Failed to load configuration after {maxRetries} attempts due to file locks");
}
private static bool IsFileLocked(string filePath)
{
try
{
using var fileStream = new FileStream(
filePath,
FileMode.Open,
FileAccess.ReadWrite,
FileShare.None);
return false; // File is not locked
}
catch (IOException)
{
return true; // File is locked
}
}
private static bool IsFileLockedException(IOException exception)
{
int errorCode = exception.HResult & 0xFFFF;
return errorCode == 32 || // ERROR_SHARING_VIOLATION
errorCode == 33; // ERROR_LOCK_VIOLATION
}
public void Dispose()
=> _watcher.Dispose();
}
internal static class GatewayConfigurationFileUtils
{
public static async ValueTask<DocumentNode> LoadDocumentAsync(
string fileName,
CancellationToken cancellationToken)
{
try
{
// We first try to load the file name as a fusion graph package.
// This might fail as the file that was provided is a fusion graph document.
await using var package = FusionGraphPackage.Open(fileName, FileAccess.Read);
return await package.GetFusionGraphAsync(cancellationToken);
}
catch
{
// If we fail to load the file as a fusion graph package we will
// try to load it as a GraphQL schema document.
var sourceText = await File.ReadAllTextAsync(fileName, cancellationToken);
return Utf8GraphQLParser.Parse(sourceText);
}
}
}
I am closing this issue as we have a new implementation of the configuration observer in version 16
