tinygrad threading issue: SQLite objects created in a thread can only be used in that same thread

[AlexCheema]

How to reproduce:

• Run a cluster on 2 nodes
• Run a request
• Restart one node
• Run another request

Traceback (most recent call last):
  File "/Users/alex/exo/exo/api/chatgpt_api.py", line 288, in handle_post_chat_completions
    await asyncio.wait_for(asyncio.shield(asyncio.create_task(self.node.process_prompt(shard, prompt, request_id=request_id))), timeout=self.response_timeout)
  File "/usr/local/python-X.Y.Z-static/lib/python3.12/asyncio/tasks.py", line 520, in wait_for
    return await fut
           ^^^^^^^^^
  File "/Users/alex/exo/exo/orchestration/standard_node.py", line 155, in process_prompt
    resp = await self._process_prompt(base_shard, prompt, request_id)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/alex/exo/exo/orchestration/standard_node.py", line 184, in _process_prompt
    resp = await self.forward_prompt(shard, prompt, request_id, 0)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/alex/exo/exo/orchestration/standard_node.py", line 272, in forward_prompt
    await target_peer.send_prompt(next_shard, prompt, request_id=request_id)
  File "/Users/alex/exo/exo/networking/grpc/grpc_peer_handle.py", line 81, in send_prompt
    response = await self.stub.SendPrompt(request)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/alex/exo/.venv/lib/python3.12/site-packages/grpc/aio/_call.py", line 327, in __await__
    raise _create_rpc_error(
grpc.aio._call.AioRpcError: <AioRpcError of RPC that terminated with:
        status = StatusCode.UNKNOWN
        details = "Unexpected <class 'sqlite3.ProgrammingError'>: SQLite objects created in a thread can only be used in that same thread. The object was created in thread id 6375518208 and this is thread id 18458554368."
        debug_error_string = "UNKNOWN:Error received from peer  {grpc_message:"Unexpected <class \'sqlite3.ProgrammingError\'>: SQLite objects created in a thread can only be used in that same thread. The object was created in thread id 
6375518208 and this is thread id 18458554368.", grpc_status:2, created_time:"2024-11-23T22:06:12.109965+04:00"}"
>