[Bug] Core may sometimes issue an activation for a run ID that lang does not know about

[bergundy]

This error is seen on rare occasions in both the TypeScript and Python SDKs:

TypeScript:

IllegalStateError: Received workflow activation for an untracked workflow with no start workflow job

Python:

tests/worker/test_workflow.py::test_workflow_uuid 
-------------------------------- live log call ---------------------------------
21:20:22 [   DEBUG] Evicting workflow with run ID f9b1436c-2766-47d3-be18-0a330e1e398c, message: Workflow cache full (_workflow.py:266)
21:20:22 [   ERROR] Failed handling activation on workflow with run ID f9b1436c-2766-47d3-be18-0a330e1e398c (_workflow.py:216)
Traceback (most recent call last):
  File "/home/runner/work/sdk-python/sdk-python/temporalio/worker/_workflow.py", line 196, in _handle_activation
    workflow = self._create_workflow_instance(act)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/runner/work/sdk-python/sdk-python/temporalio/worker/_workflow.py", line 294, in _create_workflow_instance
    raise RuntimeError(
RuntimeError: Missing start workflow, workflow could have unexpectedly been removed from cache

It seems to happen with "legacy" queries and is likely to do with evictions but more investigation is required.

This causes a runtime error as it violates the Core-Lang contract. Core should avoid sending activations to Lang for run IDs Lang is not aware of. If this is related to queries, Core should fetch the history for the run ID and recreate the workflow state so the query can be responded to.

[bergundy]

This may only happen in older versions of Core. Leaving it open for a while longer. Let's close if nobody encounters this again with newer versions in a few months.