edgedb-python icon indicating copy to clipboard operation
edgedb-python copied to clipboard

Error when a gel-python app is terminated or killed

Open gscho opened this issue 2 months ago • 0 comments

Describe the bug

I'm running a gel-python application on kubernetes. The application was failing its liveness checks and getting restarted by kubernetes and it produced this error on shutdown:

     ERROR   Traceback (most recent call last):                                 
               File                                                             
             "/app/.venv/lib/python3.13/site-packages/starlette/routing.py",    
             line 694, in lifespan                                              
                 async with self.lifespan_context(app) as maybe_state:          
                            ~~~~~~~~~~~~~~~~~~~~~^^^^^                          
               File "/usr/local/lib/python3.13/contextlib.py", line 221, in     
             __aexit__                                                          
                 await anext(self.gen)                                          
               File                                                             
             "/app/.venv/lib/python3.13/site-packages/fastapi/routing.py", line 
             209, in merged_lifespan                                            
                 async with original_context(app) as maybe_original_state:      
                            ~~~~~~~~~~~~~~~~^^^^^                               
               File "/usr/local/lib/python3.13/contextlib.py", line 221, in     
             __aexit__                                                          
                 await anext(self.gen)                                          
               File                                                             
             "/app/.venv/lib/python3.13/site-packages/fastapi/routing.py", line 
             209, in merged_lifespan                                            
                 async with original_context(app) as maybe_original_state:      
                            ~~~~~~~~~~~~~~~~^^^^^                               
               File "/usr/local/lib/python3.13/contextlib.py", line 221, in     
             __aexit__                                                          
                 await anext(self.gen)                                          
               File                                                             
             "/app/.venv/lib/python3.13/site-packages/fastapi/routing.py", line 
             210, in merged_lifespan                                            
                 async with nested_context(app) as maybe_nested_state:          
                            ~~~~~~~~~~~~~~^^^^^                                 
               File                                                             
             "/app/.venv/lib/python3.13/site-packages/gel/_internal/_integration
             /_fastapi/_client.py", line 224, in __aexit__                      
                 await concurrency.run_in_threadpool(                           
                     self._bio_client.close, timeout=timeout                    
                 )                                                              
               File                                                             
             "/app/.venv/lib/python3.13/site-packages/starlette/concurrency.py",
             line 38, in run_in_threadpool                                      
                 return await anyio.to_thread.run_sync(func)                    
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^                    
               File                                                             
             "/app/.venv/lib/python3.13/site-packages/anyio/to_thread.py", line 
             56, in run_sync                                                    
                 return await get_async_backend().run_sync_in_worker_thread(    
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^    
                     func, args, abandon_on_cancel=abandon_on_cancel,           
             limiter=limiter                                                    
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
             ^^^^^                                                              
                 )                                                              
                 ^                                                              
               File                                                             
             "/app/.venv/lib/python3.13/site-packages/anyio/_backends/_asyncio.p
             y", line 2485, in run_sync_in_worker_thread                        
                 return await future                                            
                        ^^^^^^^^^^^^                                            
               File                                                             
             "/app/.venv/lib/python3.13/site-packages/anyio/_backends/_asyncio.p
             y", line 976, in run                                               
                 result = context.run(func, *args)                              
               File                                                             
             "/app/.venv/lib/python3.13/site-packages/gel/blocking_client.py",  
             line 788, in close                                                 
                 iter_coroutine(self._impl.close(timeout))                      
                 ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^                      
               File                                                             
             "/app/.venv/lib/python3.13/site-packages/gel/blocking_client.py",  
             line 75, in iter_coroutine                                         
                 coro.send(None)                                                
                 ~~~~~~~~~^^^^^^                                                
               File                                                             
             "/app/.venv/lib/python3.13/site-packages/gel/blocking_client.py",  
             line 372, in close                                                 
                 await ch.close()                                               
               File                                                             
             "/app/.venv/lib/python3.13/site-packages/gel/blocking_client.py",  
             line 274, in close                                                 
                 await self._con.close(timeout=timeout)                         
               File                                                             
             "/app/.venv/lib/python3.13/site-packages/gel/blocking_client.py",  
             line 230, in close                                                 
                 await self._protocol.wait_for_disconnect()                     
               File "gel/protocol/blocking_proto.pyx", line 127, in             
             wait_for_disconnect                                                
               File "gel/protocol/protocol.pyx", line 1354, in                  
             gel.protocol.protocol.SansIOProtocol.fallthrough                   
             gel.errors.ProtocolError: unexpected message type 'E'
     ERROR   Application shutdown failed. Exiting.

Reproduction

I haven't tried to recreate it but I think if you have a fastAPI application using gel-python and you send a SIGTERM or SIGKILL it would likely be recreated.

Expected behavior

Graceful shutdown.

Versions (please complete the following information):

  • OS: Debian 12
  • EdgeDB version: 7.0
  • EdgeDB CLI version: Gel CLI 7.7.0+3788f53
  • edgedb-python version: Latest -> The pyproject.toml points to main branch in gel-python
  • Python version: 3.13.5

gscho avatar Nov 10 '25 17:11 gscho