components-contrib
components-contrib copied to clipboard
Dapr sidecar does not crash after failure to create infrastructure
In what area(s)?
/area runtime
What version of Dapr?
1.14
Expected Behavior
The Dapr sidecar crashes and causes the associated application to restart as well
Actual Behavior
The Dapr sidecar logs a warning but continues to run without being subscribed
Steps to Reproduce the Problem
Theoretically this can happen in a variety of ways where the underlying infrastructure fails to create. The way I encountered the issue was by having 2 applications that publish and subscribe to the same topic. They get deployed together so at roughly the same time they would try and create the same topic.
I use Azure Service Bus so while that is not the root cause of the problem, it is a contributor in that if you get 2 requests close to each other to create the same resource, one of the requests may fail.
failed to subscribe to topics: failed to subscribe to topic {topic name}: could not create topic {topic name}
RESPONSE 409: 409 Conflict\nERROR CODE: 409\n--------------------------------------------------------------------------------\n<Error><Code>409<Detail>SubCode=40900. Conflict.
You're requesting an operation that isn't allowed in the resource's current state. To know more visit https://aka.ms/sbResourceMgrExceptions.
Release Note
RELEASE NOTE: FIX Failure to create pubsub infrastructure will no longer leave the sidecar running.
This issue has been automatically marked as stale because it has not had activity in the last 30 days. It will be closed in the next 7 days unless it is tagged (pinned, good first issue, help wanted or triaged/resolved) or other activity occurs. Thank you for your contributions.
I guess if dapr recognize those errors and just logs a warning, its only a matter of changing the log level of those kinds of errors from warning to critical...
Looks to me like the desired behavior should be that we log as fatal in this case. @JoshVanL thoughts?
This issue has been automatically marked as stale because it has not had activity in the last 30 days. It will be closed in the next 7 days unless it is tagged (pinned, good first issue, help wanted or triaged/resolved) or other activity occurs. Thank you for your contributions.
This issue has been automatically closed because it has not had activity in the last 37 days. If this issue is still valid, please ping a maintainer and ask them to label it as pinned, good first issue, help wanted or triaged/resolved. Thank you for your contributions.