[Improvement]: Make sqlalchemy create_engine parameters configurable
Currently we override sqlalchemy's default engine pool_size from 5 to 60 in the ENGINE_ARGUMENTS dictionary in orchestrator/db/database.py. Configuring it to a different value in a WFO implementation requires patching said dictionary before initializing the OrchestratorCore app.
It would be nice if there was an environment variable to configure this alongside with 2 other parameters of create_engine():
pool_size: max number of persistent connectionsmax_overflow: max number of additional connections. Added on top ofpool_size, however these are closed immediately after use. (sqlalchemy default = 10)timeout: seconds to wait before giving up on getting a connection from the pool (sqlalchemy default = 30 seconds)
The settings can be added to the AppSettings classed and used in the Database.__init__ function - taking precedence over ENGINE_ARGUMENTS unless None. Please document these settings along with suggestions on reasonable values to use - also considering what happens if you run multiple instances of the API, and if the number of connections is exhausted.
Also, please test what happens when the number of connections is exhausted; what kind of response is returned by the REST/GraphQL API, and if this could be improved by catching the sqlalchemy timeout.