PrefectHQ
PREFECT_AUTH_STRING is not injected when Azure container is spawn
Bug summary
Dears, I'm facing issues with executing flows on Azure Container Interface. I (partly) deployed a worker on Azure and made it connected to a self-hosted server on a personal Virtual Private Server.
For a non-protected API server, everything works, container is spawn on Azure through the woker, flow is executed and result reported correctly to the server. For basic authentication protected API server, the spawned container fails upon starting. I checked the env variables and saw that the following ones
PREFECT_AUTH_STRING PREFECT_API_KEY
are empty. Even if I set them explicitely via the base-template, they got empty on the target container.
Getting deeper in the logs, I noticed that the worker deployed in ACI works almost correctly, except for the websocket part:
14:27:53.615 | WARNING | prefect.events.clients - Unable to connect to 'ws://XXXX/api/events/in'. Please check your network settings to ensure websocket connections to the API are allowed. Otherwise event data (including task run data) may be lost. Reason: server rejected WebSocket connection: HTTP 401. Set PREFECT_DEBUG_MODE=1 to see the full error.
14:27:53.619 | ERROR | GlobalEventLoopThread | prefect._internal.concurrency - Service 'EventsWorker' failed with 1 pending items.
which seems related to the error I see on spawned containers:
File "/usr/local/lib/python3.12/site-packages/prefect/cli/_utilities.py", line 44, in wrapper
return fn(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/prefect/cli/_types.py", line 159, in sync_fn
return asyncio.run(async_fn(*args, **kwargs))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/asyncio/runners.py", line 194, in run
return runner.run(main)
^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/asyncio/runners.py", line 118, in run
return self._loop.run_until_complete(task)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/asyncio/base_events.py", line 687, in run_until_complete
return future.result()
^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/prefect/cli/flow_run.py", line 428, in execute
await runner.execute_flow_run(id)
File "/usr/local/lib/python3.12/site-packages/prefect/runner/runner.py", line 601, in execute_flow_run
async with context:
File "/usr/local/lib/python3.12/site-packages/prefect/runner/runner.py", line 1571, in __aenter__
self._cancelling_observer = await self._exit_stack.enter_async_context(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/contextlib.py", line 659, in enter_async_context
result = await _enter(cm)
^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/prefect/runner/_observers.py", line 157, in __aenter__
self._events_subscriber = await self._exit_stack.enter_async_context(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/contextlib.py", line 659, in enter_async_context
result = await _enter(cm)
^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/prefect/events/clients.py", line 555, in __aenter__
await self._reconnect()
File "/usr/local/lib/python3.12/site-packages/prefect/events/clients.py", line 566, in _reconnect
self._websocket = await self._connect.__aenter__()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/websockets/asyncio/client.py", line 587, in __aenter__
return await self
^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/websockets/asyncio/client.py", line 543, in __await_impl__
await self.connection.handshake(
File "/usr/local/lib/python3.12/site-packages/websockets/asyncio/client.py", line 114, in handshake
raise self.protocol.handshake_exc
File "/usr/local/lib/python3.12/site-packages/websockets/client.py", line 325, in parse
self.process_response(response)
File "/usr/local/lib/python3.12/site-packages/websockets/client.py", line 142, in process_response
raise InvalidStatus(response)
websockets.exceptions.InvalidStatus: server rejected WebSocket connection: HTTP 401
An exception occurred.
Upon deploying the worker, I deliberately set both the PREFECT_API_KEY and PREFECT_API_AUTH_STRING and they got correctly injected in the spawned container in ACI and set values are visible through the UI.
Best, FB
Version info
Version: 3.6.4
API version: 0.8.4
Python version: 3.12.12
Git commit: d3c3ed50
Built: Fri, Nov 21, 2025 06:07 PM
OS/Arch: linux/x86_64
Profile: ephemeral
Server type: server
Pydantic version: 2.12.4
Server:
Database: postgresql
PostgreSQL version: 14.20 (Debian 14.20-1.pgdg13+1)
Integrations:
prefect-redis: 0.2.6
Additional context
Worker version deployed on ACI, via Docker: prefecthq/prefect:3-python3.12 Integrations: prefect-docker: 0.6.6 prefect-azure: 0.4.6
I had a similar problem with running Cloud Run jobs in Google Cloud. In my case the PREFECT_API_AUTH_STRING was getting included in my container, but for some reason it wasn't being used. I got round it by putting the basic auth string in the PREFECT_API_URL env variable, e.g. PREFECT_API_URL=https://username:[email protected]/api.
