[QUESTION] Getting Eureka port registration correct with dynamic port allocation

[osmedd]

Question

I have a .NET Core service I need to register in Eureka. In some use cases, it needs to run using automatic port assignment to let .NET pick an open port (like: Blah.exe --urls "http://*:0"). In this case, Steeltoe Discovery registers the application as using port 0, which saddens anybody who looks up this entry. I am sure there is a way to configure and force Steeltoe to look at what port was assigned, but combing Google did not come up with any good answers. Pointers, please?

Environment (please complete the following information):

• .NET Version: 6.0.13
• Steeltoe Version: 3.2.2

Additional context or links

I have a hack using EurekaApplicationInfoManager where I update the port information after the ApplicationStarted event is fired. This seemed to work okay in Steeltoe 2.5.5, but now in 3.2.2 there is a 404 error from Eureka (1.10.17) on the first refresh after updating the port information... this causes a re-registration which is correct but that leaves both instances registered for a period of time opening the door to breaking things.

[osmedd]

It seems deciding that port 0 is the correct port to advertise to Eureka is incorrect and I should refile this as a defect?

[TimHess]

Hi @osmedd,

Thanks for opening the issue, and sorry for a bit of a delayed response. I think this ultimately may be a feature request. Right now, the port selected for registration is much more configuration-based. In order to get the port ASP.NET Core is actually listening on, we will need to do something like this, which might be a great solution, or it might cause issues (I don't know off the top of my head, we might have to try it out).

[osmedd]

I have for a few years now used Steeltoe 2.5.5 with the approach (wrenching the values in the eureka instance config in an ApplicationStarted event handler) that is described in that article. Code below... it seemed like a hack but it worked. However, when I tried to update to 3.2.2, I tried something similar but the results are not so pretty... I end up with a port 0 registration which is then abandoned (but still live until timeout) with a parallel "correct" registration that happens after an exception and a re-registration. I was thinking I am just missing the correct way to do this in current code.

Here is an abridged version of the hack I use in 2.5.5 in the ApplicationStarted event. It isn't pretty I know but I was hoping to clean things up with the move to 3.2.2.

var addressFeature = app.ServerFeatures.Get<IServerAddressesFeature>();
// Select the first address in the list to register
var hostUri = new Uri(addressFeature.Addresses.First());

var eurekaInstanceOptions = new EurekaInstanceOptions();
ConfigurationBinder.Bind(configuration.GetSection(EurekaInstanceOptions.EUREKA_INSTANCE_CONFIGURATION_PREFIX), eurekaInstanceOptions);
eurekaInstanceOptions.InstanceId = sharedState.ServerId.ToString();
if (UseSSL)
{
    eurekaInstanceOptions.SecurePort = hostUri.Port;
    eurekaInstanceOptions.SecurePortEnabled = true;
    eurekaInstanceOptions.NonSecurePortEnabled = false;
}
else
{
    eurekaInstanceOptions.Port = hostUri.Port;
}
eurekaInstanceOptions.StatusPageUrlPath = "/app/status";
eurekaInstanceOptions.HealthCheckUrlPath = "/app/hc";
Log.Information($"Registering port {hostUri.Port} with Eureka (with at least host {hostUri.Host})...");
ApplicationInfoManager.Instance.Initialize(eurekaInstanceOptions, loggerFactory);

Owen

[TimHess]

This seems like a fairly complicated issue that doesn't exactly align with how the Steeltoe Eureka client was originally built. I've been looking at this for a couple hours now, but I don't have an easy answer jumping out at me. It would help a smidge if there was a repro we were both looking at - maybe a fork of this service would be easy enough to work with, would you want to take a stab at that?.

One (less than ideal) option to help with cleaning up the bad registration might be to get EurekaDiscoveryClient from the DI container and call .HttpClient.StatusUpdateAsync() (or other combinations of methods) on it.

[bart-vmware]

@osmedd Supporting automatic port assignment greatly complicates the internals of Steeltoe discovery. We can't know the actual port until the app is fully started. Because the Instance ID depends on the port number, we'll need to delay the initialization of discovery. Otherwise, multiple app instances would register under the same instance ID. As the port is unknown until fully started, there's no point in registering with Eureka until then. But that also means that getting remote instances won't work, as these things are intertwined. So another hosted service that wants to issue an HTTP request would fail. We could use a random GUID for the Instance ID when automatic port assignment is used, but the problem is we can't reliably determine that we should until IServerAddressesFeature is populated. Another issue is that enabling external code to change the port (as in your workaround) would be overwritten any time the configuration reloads at a later time. There are also race conditions, ie you'd set .SecurePort, the heartbeat kicks in from another thread, which issues an HTTP request to Eureka to re-register because the instance is dirty, then your next line .SecurePortEnabled = true executes, then the Eureka HTTP request completes and clears the dirty flag. So now your second change will never be sent to Eureaka, except as a side effect if something else changes (which may never happen).

So I'm curious if there's a strong use case for automatic port assignment. Can you elaborate on that? Otherwise, I'm afraid it's just not worth all the trouble.

[bart-vmware]

Automatic port assignment is supported in #1280 and works in the following way:

• When binding to zero port number(s), the local Instance ID is suffixed with a random number to make it unique, and then registered in Eureka as usual. This happens before the app has fully started.
• Once the app has fully started, the local Instance is updated with the assigned port number(s) and Eureka gets updated.
  • If the update results in a change of the Instance ID, an error is thrown and all changes are discarded. To prevent that from happening, explicitly set the Instance ID in configuration at key eureka:instance:instanceId.