azure-functions-host
azure-functions-host copied to clipboard
Azure
HostID Truncation can cause collisions
Currently when generating a default host ID we use the host name (slot host name) and we truncate to 32 characters max (code here). This ensures that the generated ID conforms to the core SDK length restrictions (code here).
This truncation can of course open the possibility for naming collisions, particularly in the case of slots. In slot scenarios, if the site name is over 32 characters long and a slot is created that starts with the same 32 characters and is only disambiguated in later characters, both the production and the slot site will be using the same host ID. This can lead to issues. For example, TimerTrigger uses the host ID as a component of the blob lease path. In this case the timer function will only be able to run in one of the sites because they're competing for the same lock. Similarly, customers often have apps deployed to different regions using the same long naming path, varying only in later name components (e.g. region/environment), and can run into this.
More information on can be found in Host ID Collisions.
This is a dangerous change to make for v1 but we should fix it in v2. This would require some updates to the scale controller as I believe it has this logic duplicated.
I've confirmed the scale controller will need an update, but it should be straightforward once we know what the new logic for generating the host ID is.
We had to punt this due to higher priority issues and the need to coordinate the change across multiple components. Revisit again in V3.
As a workaround for customers running into this issue, in Functions v2, you can set an explicit HostID in app settings, using a different ID for each environment. The app setting name to use is AzureFunctionsWebHost:hostId. For Functions v1 you can specify the ID in host.json via the hostId property. The host ID value should be unique for all apps/slots you're running. The important thing is that the IDs are under 32 characters. The restrictions for HostIds that the value must satisfy are here. Another way to generate an ID would be to take a GUID, remove the dashes and make it lower case, e.g. 1835D7B5-5C98-4790-815D-072CC94C6F71 => 1835d7b55c984790815d072cc94c6f71
Had a customer issue last week, where this truncation caused collision between multiple function apps sharing the same storage account.
@fabiocav, is this already planned for V3?
Deferring this work as it would have scale controller dependencies and does not align with the timing.
@paulbatum @fabiocav Per #1904 regarding the same detail on architecture center doc, we communicate that the Function App name length to be 1-60 - could you advise what the appropriate length that we should update it to?
@mike-urnun-msft good catch. staying in the range of 1-32 will avoid this bug.
Though there is no specific mention in this issue, I assume that the solution of explicitly setting the host ID via the 'AzureFunctionsWebHost:hostId' App Setting is still valid for v3 Function Apps?
If so, does the host ID have to remain static, or can it change after every deployment? I ask because our CD pipeline updates App Settings through an ARM template, so if the host ID has to remain static we'd first need to query the App Settings to get the current value of 'AzureFunctionsWebHost:hostId' for the slot we are deploying to so that it can be set back with the same value.
Also, does this only apply to a Function App with a slot, or does the value of 'AzureFunctionsWebHost:hostId' have to be unique across Function Apps? For example, if you have these two Function Apps with no slots...
- my-really-really-long-functionapp-uksouth
- my-really-really-long-functionapp-ukwest
...could both have the same truncated value of 'my-really-really-long-functionap'?
Why is this not mentioned in the docs or warned upon creation? This has been known since 2017, but no obvious mention of it anywhere that I can find. ( I very well could be blind, so if so please correct me. )
+1 I just recently got hit with a case that appears to be, at least partially, caused by this limitation.
@mathewc Is the app setting still a viable approach for v3 function apps? I also was told from an open MS ticket this when running deployment and swap operations, Azure is unable to differentiate between your deployment slots.
We are very interested with this approach because we use a specific naming convention with our resource naming and this limit does not give a lot of wiggle room.
@mpaul31 yes, the app setting is still an option, so setting the host ID using AzureFunctionsWebHost__hostId(Windows and Linux) or AzureFunctionsWebHost:hostId (Windows only) is supported.
@mpaul31 yes, the app setting is still an option, so setting the host ID using
AzureFunctionsWebHost__hostId(Windows and Linux) orAzureFunctionsWebHost:hostId(Windows only) is supported.
@fabiocav Just to confirm, does the casing of the app setting matter? When looking at the Diagnose and Solve blade for my function app, it mentions this app setting (note it's all lowercase): AzureFunctionsWebHost__hostid
Also, after I added the setting AzureFunctionsWebHost__hostId Diagnose and Solve still shows a warning about the function name collision. Should I just ignore this?
@fabiocav
We added the AzureFunctionsWebHost__hostid (lowercase) and we still get critical error in the portal:
Also the timer trigger stops working randomly from time to time.
Could you please provide correct unambiguous documentation on how to solve this issue.
Is there also a procedure on how detect if our timer trigger issue comes from the hostId issue? Where do need to check the logs? Because everything is successful.
It just stops firing.
Many thanks.
@cmenzi thanks for highlighting this behavior, we are modifying the logic to check if the HostId is set.
We added code to the host to check for this issue and log a warning in Functions v3. In Functions v4 when detected it's an error and we prevent the host from starting. See https://github.com/Azure/Azure-Functions/issues/2049 for details.
@mathewc: Should the AzureFunctionsWebHost:hostId setting be slot-sticky ("deployment slot setting")?
Important information AzureFunctionsWebHost:hostId should be lower case. Else in some case it can crash your function host
Important information AzureFunctionsWebHost:hostId should be lower case. Else in some case it can crash your function host
Can we confirm this please. This is very important.
It's only the case when you override and I think it's related to a specific trigger. I had this error when used an uppercased hostid. That is not the case when hostid is not override or when it's lowercased
The specifed resource name contains invalid characters.
RequestId:d154be16-6003-006a-7eff-0d7701000000
Time:2022-01-20T13:14:27.4462948Z
Status: 400 (The specifed resource name contains invalid characters.)
ErrorCode: InvalidResourceName
Content:
<?xml version="1.0" encoding="utf-8"?><Error><Code>InvalidResourceName</Code><Message>The specifed resource name contains invalid characters.
RequestId:d154be16-6003-006a-7eff-0d7701000000
Time:2022-01-20T13:14:27.4462948Z</Message></Error>
Headers:
Server: Windows-Azure-Queue/1.0,Microsoft-HTTPAPI/2.0
x-ms-request-id: d154be16-6003-006a-7eff-0d7701000000
x-ms-version: 2018-11-09
x-ms-error-code: InvalidResourceName
Date: Thu, 20 Jan 2022 13:14:27 GMT
Content-Length: 243
Content-Type: application/xml
@mathewc: Should the
AzureFunctionsWebHost:hostIdsetting be slot-sticky ("deployment slot setting")?
we've added this property for each slot (deployment slot setting) and the issues disappear. (windows function)
{
"name": "AzureFunctionsWebHost:hostId",
"value": "my-function-slotname",
"slotSetting": true
}
I think the issue still exist at the v4 SDK https://github.com/Azure/azure-functions-host/wiki/Host-IDs#host-id-collisions
I was under the impression this was actually resolved in V4. Can @fabiocav or @mathewc confirm? It may help to close this thread, to communicate this behavior has been established, if that's the case.
@AlphaWong is correct - the restriction still exists, however in v4 we added detection and prevent the host from starting up in this state as described here.
@mathewc we just ran into this host id truncation issue out of no where, it was working up until last friday July 1, 2022. Our v4 application did start. It also read and processed messages from a incoming queue. However, it would fail writing to an output queue. We have resolved it by changing our site names.
Running a v4 function, we're still facing the Function App Name Collision Found error when running function configuration diagnostic even though hostid is setup at slot level (linux consumption plan) using a random lowercase guid without dashes. Should we just ignore this error message? Should I setup FUNCTIONS_HOSTID_CHECK_LEVEL to Warning level?
It had also already been asked if we must keep a static hostId at each deployment or if we can generate a new one but I didn't see any answer to this question.
Thanks in advance for the clarifications.
@AlphaWong is correct - the restriction still exists, however in v4 we added detection and prevent the host from starting up in this state as described here.
This error message appeared for me, even though there was no conflict. It seems this gets triggered if there is any truncation. It does not seem to be preventing my apps from running, though. I think it may be more appropriate for this to appear as a warning when there is truncation, but no conflict detected.
On a related note, why is the limit only 32 characters? That seems rather short when basing things on names assigned by humans. I guess this is really all a result of the fatal flaw in Azure where the name of a resource doubles as the ID of the resource. This decision has created a lot of inconveniences within Azure.
