pygeoapi icon indicating copy to clipboard operation
pygeoapi copied to clipboard

Starlette server fails with top level async-await

Open C-Loftus opened this issue 8 months ago • 5 comments
trafficstars

Description

When launching pygeoapi with pygeoapi serve --starlette pygeoapi throws an error if you try to use top level async functions. I expected to be able to use async functions without needing to use asyncio.run since the web server should initialize the async event loop.

If you are developing a large pygeoapi plugin with lots of fetch-based ETL, async is often needed and using asyncio.run over every async function adds a fair bit of boilerplate (and requires you to make sure another event loop isn't running / created by another plugin)

The error seems to signify that the content is not being awaited properly

Steps to Reproduce

  1. Create a custom provider plugin
  2. Create an async function like get inside the provider
    async def get(self, identifier, **kwargs):
        await asyncio.sleep(1)
    # rest of geojson response omitted here for brevity ...
  1. When you call the oaf endpoint and thus call get() it will give the error:
Traceback (most recent call last):
  File "/Users/cloftus/github/RISE-EDR/.venv/lib/python3.12/site-packages/uvicorn/protocols/http/h11_impl.py", line 403, in run_asgi
    result = await app(  # type: ignore[func-returns-value]
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/cloftus/github/RISE-EDR/.venv/lib/python3.12/site-packages/uvicorn/middleware/proxy_headers.py", line 60, in __call__
    return await self.app(scope, receive, send)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/cloftus/github/RISE-EDR/.venv/lib/python3.12/site-packages/starlette/applications.py", line 112, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/Users/cloftus/github/RISE-EDR/.venv/lib/python3.12/site-packages/starlette/middleware/errors.py", line 187, in __call__
    raise exc
  File "/Users/cloftus/github/RISE-EDR/.venv/lib/python3.12/site-packages/starlette/middleware/errors.py", line 165, in __call__
    await self.app(scope, receive, _send)
  File "/Users/cloftus/github/RISE-EDR/.venv/lib/python3.12/site-packages/starlette/middleware/exceptions.py", line 62, in __call__
    await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
  File "/Users/cloftus/github/RISE-EDR/.venv/lib/python3.12/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
    raise exc
  File "/Users/cloftus/github/RISE-EDR/.venv/lib/python3.12/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app
    await app(scope, receive, sender)
  File "/Users/cloftus/github/RISE-EDR/.venv/lib/python3.12/site-packages/starlette/routing.py", line 714, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/Users/cloftus/github/RISE-EDR/.venv/lib/python3.12/site-packages/starlette/routing.py", line 734, in app
    await route.handle(scope, receive, send)
  File "/Users/cloftus/github/RISE-EDR/.venv/lib/python3.12/site-packages/starlette/routing.py", line 460, in handle
    await self.app(scope, receive, send)
  File "/Users/cloftus/github/RISE-EDR/.venv/lib/python3.12/site-packages/starlette/routing.py", line 714, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/Users/cloftus/github/RISE-EDR/.venv/lib/python3.12/site-packages/starlette/routing.py", line 734, in app
    await route.handle(scope, receive, send)
  File "/Users/cloftus/github/RISE-EDR/.venv/lib/python3.12/site-packages/starlette/routing.py", line 288, in handle
    await self.app(scope, receive, send)
  File "/Users/cloftus/github/RISE-EDR/.venv/lib/python3.12/site-packages/starlette/routing.py", line 76, in app
    await wrap_app_handling_exceptions(app, request)(scope, receive, send)
  File "/Users/cloftus/github/RISE-EDR/.venv/lib/python3.12/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
    raise exc
  File "/Users/cloftus/github/RISE-EDR/.venv/lib/python3.12/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app
    await app(scope, receive, sender)
  File "/Users/cloftus/github/RISE-EDR/.venv/lib/python3.12/site-packages/starlette/routing.py", line 73, in app
    response = await f(request)
               ^^^^^^^^^^^^^^^^
  File "/Users/cloftus/github/RISE-EDR/.venv/lib/python3.12/site-packages/pygeoapi/starlette_app.py", line 360, in collection_items
    return await execute_from_starlette(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/cloftus/github/RISE-EDR/.venv/lib/python3.12/site-packages/pygeoapi/starlette_app.py", line 133, in execute_from_starlette
    headers, status, content = await loop.run_in_executor(
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/cloftus/.local/share/uv/python/cpython-3.12.8-macos-aarch64-none/lib/python3.12/concurrent/futures/thread.py", line 59, in run
    result = self.fn(*self.args, **self.kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/cloftus/github/RISE-EDR/.venv/lib/python3.12/site-packages/pygeoapi/starlette_app.py", line 104, in call_api_threadsafe
    return api_call(*args)
           ^^^^^^^^^^^^^^^
  File "/Users/cloftus/github/RISE-EDR/.venv/lib/python3.12/site-packages/pygeoapi/api/itemtypes.py", line 856, in get_collection_item
    if 'links' not in content:
       ^^^^^^^^^^^^^^^^^^^^^^
TypeError: argument of type 'coroutine' is not iterable

Expected behavior

I expect to be able to have starlette itself initialize the async event loop. I expected to be able to use async functions as it mentions in the starlette docs

from starlette.applications import Starlette
from starlette.responses import JSONResponse
from starlette.routing import Route


async def homepage(request):
    return JSONResponse({'hello': 'world'})


app = Starlette(debug=True, routes=[
    Route('/', homepage),
])

Environment

  • OS: Macos Sonoma 14.3
  • Python version: 3.12.8
  • pygeoapi version: pygeoapi, version 0.20.dev0

C-Loftus avatar Mar 12 '25 17:03 C-Loftus