python-swat icon indicating copy to clipboard operation
python-swat copied to clipboard

Await for CAS actions execution

Open gygabyte017 opened this issue 4 years ago • 3 comments
trafficstars

Hi, I am developing a backend API which extract data from CAS and returns it to the caller. Since from the backend point of view every CAS action called is an I/O blocking statement, I would like very much to be able to await for them using asyncio.

For instance something like:

async def calculate_data(...):
  await cas.dataStep.runCode(code="...")
  result = await cas.table.fetch(.....)
  return result

Is that something achievable?

gygabyte017 avatar Oct 05 '21 13:10 gygabyte017

I'm investigating your question, I'll let you know what I find out

bkemper24 avatar Oct 11 '21 11:10 bkemper24

I did some investigating into asyncio. One thing I read was that await can only be called on coroutine functions, so I don't think it would be as simple as "await cas.dataStep.runCode(code="...")". However, it looks like you may be able to use the run_in_executor() loop function to run it. I haven't had a chance to actually try it out yet, though.

bkemper24 avatar Nov 02 '21 11:11 bkemper24

Thank you it works great, however when two concurrent calls use the same cas connection I get unexpected results. What would you recommend using some kind of copy or fork strategy? Or to maintain some kind of pool?

Basically I am building an API, where when a user calls an endpoint that endpoint fetches a slice of a table on cas and returns it to the user. Therefore there may be a lot of parallel requests to the endpoint, and the data from cas should be retrieved as fast and as parallel as possible.

Any advice on that? Thank you very much

gygabyte017 avatar Dec 10 '21 11:12 gygabyte017