Azure
Investigate the ordering of Init request and Metadata request in multi-worker scenarios
Ideally, it is expected that worker will receive WorkerInitRequest first and then FunctionMetadataRequest (in case of Stein) or FunctionLoadRequest (legacy scenario). However, by random chance, it seems the second worker may receive a function load request or a metadata request as its first request.
Need to investigate this. Ideally, these messages should be in predictable order (where WorkerInitRequest will be the first one).
Case A
Worker 1 gets an init request and then metadata request (as expected). However, worker 2 receives a function load request before an init request.
[2022-10-06T02:42:39.129Z] Worker 71780151-7b5a-4152-af87-95168dc0aacf received WorkerInitRequest [2022-10-06T02:42:39.464Z] Worker 71780151-7b5a-4152-af87-95168dc0aacf received FunctionsMetadataRequest [2022-10-06T02:42:39.514Z] Worker 71780151-7b5a-4152-af87-95168dc0aacf received FunctionLoadRequest [2022-10-06T02:42:39.878Z] Worker bffcf551-f64f-48c7-808d-fba0659b37a1 received FunctionLoadRequest [2022-10-06T02:42:39.886Z] Worker bffcf551-f64f-48c7-808d-fba0659b37a1 received WorkerInitRequest
Case B
Worker 1 is initialized, but worker 2 receives the metadata request before it is initialized.
[2022-10-06T02:36:12.135Z] Worker 554e325b-3cef-4db0-8626-6a32a4a67e7e received WorkerInitRequest [2022-10-06T02:36:12.737Z] Worker d43a5b45-cb3e-45a8-ba89-a53548271186 received FunctionsMetadataRequest [2022-10-06T02:36:12.741Z] Worker d43a5b45-cb3e-45a8-ba89-a53548271186 received WorkerInitRequest
