ib_async
ib_async copied to clipboard
ib-api-reloaded
Previously working implementations timeout as soon as TWS asks to allow
I had a previously working implementation of ib_insync that started giving "Failed to connect to IB gateway/ Timeout Error" as soon as the TWS popup asks if I want to allow the connection about 2 or 3 weeks ago. I've been going crazy trying to track down the exact cause and was shocked to find the news about ib_insync's creator, and then even more shock to have the same issue when using this new and updated library. Whitelisting doesn't help and as a way to rule out my implementation, I have the same issue with @mattsta 's icli (and also I ran the example code in a jupiter notebook to triple check, same results output follows:
2024-05-06 14:04:21,259 ib_async.client INFO Connecting to 192.168.50.203:7497 with clientId 5280...
2024-05-06 14:04:21,266 ib_async.client INFO Connected
2024-05-06 14:04:25,271 ib_async.client INFO Disconnecting
2024-05-06 14:04:25,274 ib_async.client ERROR API connection failed: TimeoutError()
---------------------------------------------------------------------------
CancelledError Traceback (most recent call last)
File [~/.pyenv/versions/3.11.0rc1/lib/python3.11/asyncio/tasks.py:495](http://localhost:8888/home/lc/.pyenv/versions/3.11.0rc1/lib/python3.11/asyncio/tasks.py#line=494), in wait_for(fut, timeout)
494 try:
--> 495 return fut.result()
496 except exceptions.CancelledError as exc:
File [~/.pyenv/versions/3.11.0rc1/lib/python3.11/asyncio/futures.py:204](http://localhost:8888/home/lc/.pyenv/versions/3.11.0rc1/lib/python3.11/asyncio/futures.py#line=203), in Future.result(self)
203 exc = self._make_cancelled_error()
--> 204 raise exc
205 if self._state != _FINISHED:
File [~/.pyenv/versions/3.11.0rc1/lib/python3.11/asyncio/tasks.py:274](http://localhost:8888/home/lc/.pyenv/versions/3.11.0rc1/lib/python3.11/asyncio/tasks.py#line=273), in Task.__step(***failed resolving arguments***)
273 else:
--> 274 result = coro.throw(exc)
275 except StopIteration as exc:
File [~/.pyenv/versions/3.11.0rc1/lib/python3.11/asyncio/tasks.py:689](http://localhost:8888/home/lc/.pyenv/versions/3.11.0rc1/lib/python3.11/asyncio/tasks.py#line=688), in _wrap_awaitable(awaitable)
684 """Helper for asyncio.ensure_future().
685
686 Wraps awaitable (an object with __await__) into a coroutine
687 that will later be wrapped in a Task by ensure_future().
688 """
--> 689 return (yield from awaitable.__await__())
File [~/.pyenv/versions/3.11.0rc1/lib/python3.11/asyncio/futures.py:293](http://localhost:8888/home/lc/.pyenv/versions/3.11.0rc1/lib/python3.11/asyncio/futures.py#line=292), in Future.__await__(self)
292 self._asyncio_future_blocking = True
--> 293 yield self # This tells Task to wait for completion.
294 if not self.done():
File [~/.pyenv/versions/3.11.0rc1/lib/python3.11/asyncio/tasks.py:344](http://localhost:8888/home/lc/.pyenv/versions/3.11.0rc1/lib/python3.11/asyncio/tasks.py#line=343), in Task.__wakeup(self, future)
343 try:
--> 344 future.result()
345 except BaseException as exc:
346 # This may also be a cancellation.
File [~/.pyenv/versions/3.11.0rc1/lib/python3.11/asyncio/futures.py:204](http://localhost:8888/home/lc/.pyenv/versions/3.11.0rc1/lib/python3.11/asyncio/futures.py#line=203), in Future.result(self)
203 exc = self._make_cancelled_error()
--> 204 raise exc
205 if self._state != _FINISHED:
CancelledError:
The above exception was the direct cause of the following exception:
TimeoutError Traceback (most recent call last)
Cell In[13], line 2
1 ib = IB()
----> 2 ib.connect('192.168.50.203', 7497, clientId=5280)
File [~/.pyenv/versions/3.11.0rc1/lib/python3.11/site-packages/ib_async/ib.py:337](http://localhost:8888/home/lc/.pyenv/versions/3.11.0rc1/lib/python3.11/site-packages/ib_async/ib.py#line=336), in IB.connect(self, host, port, clientId, timeout, readonly, account, raiseSyncErrors)
305 def connect(
306 self,
307 host: str = "127.0.0.1",
(...)
313 raiseSyncErrors: bool = False,
314 ):
315 """
316 Connect to a running TWS or IB gateway application.
317 After the connection is made the client is fully synchronized
(...)
335 When ``False`` the error will only be logged at error level.
336 """
--> 337 return self._run(
338 self.connectAsync(
339 host, port, clientId, timeout, readonly, account, raiseSyncErrors
340 )
341 )
File [~/.pyenv/versions/3.11.0rc1/lib/python3.11/site-packages/ib_async/ib.py:381](http://localhost:8888/home/lc/.pyenv/versions/3.11.0rc1/lib/python3.11/site-packages/ib_async/ib.py#line=380), in IB._run(self, *awaitables)
380 def _run(self, *awaitables: Awaitable):
--> 381 return util.run(*awaitables, timeout=self.RequestTimeout)
File [~/.pyenv/versions/3.11.0rc1/lib/python3.11/site-packages/ib_async/util.py:357](http://localhost:8888/home/lc/.pyenv/versions/3.11.0rc1/lib/python3.11/site-packages/ib_async/util.py#line=356), in run(timeout, *awaitables)
355 globalErrorEvent.connect(onError)
356 try:
--> 357 result = loop.run_until_complete(task)
358 except asyncio.CancelledError as e:
359 raise globalErrorEvent.value() or e
File [~/.pyenv/versions/3.11.0rc1/lib/python3.11/site-packages/nest_asyncio.py:98](http://localhost:8888/home/lc/.pyenv/versions/3.11.0rc1/lib/python3.11/site-packages/nest_asyncio.py#line=97), in _patch_loop.<locals>.run_until_complete(self, future)
95 if not f.done():
96 raise RuntimeError(
97 'Event loop stopped before Future completed.')
---> 98 return f.result()
File [~/.pyenv/versions/3.11.0rc1/lib/python3.11/asyncio/futures.py:209](http://localhost:8888/home/lc/.pyenv/versions/3.11.0rc1/lib/python3.11/asyncio/futures.py#line=208), in Future.result(self)
207 self.__log_traceback = False
208 if self._exception is not None:
--> 209 raise self._exception.with_traceback(self._exception_tb)
210 return self._result
File [~/.pyenv/versions/3.11.0rc1/lib/python3.11/asyncio/tasks.py:272](http://localhost:8888/home/lc/.pyenv/versions/3.11.0rc1/lib/python3.11/asyncio/tasks.py#line=271), in Task.__step(***failed resolving arguments***)
268 try:
269 if exc is None:
270 # We use the `send` method directly, because coroutines
271 # don't have `__iter__` and `__next__` methods.
--> 272 result = coro.send(None)
273 else:
274 result = coro.throw(exc)
File [~/.pyenv/versions/3.11.0rc1/lib/python3.11/site-packages/ib_async/ib.py:1973](http://localhost:8888/home/lc/.pyenv/versions/3.11.0rc1/lib/python3.11/site-packages/ib_async/ib.py#line=1972), in IB.connectAsync(self, host, port, clientId, timeout, readonly, account, raiseSyncErrors)
1970 timeout = timeout or None
1971 try:
1972 # establish API connection
-> 1973 await self.client.connectAsync(host, port, clientId, timeout)
1975 # autobind manual orders
1976 if clientId == 0:
File [~/.pyenv/versions/3.11.0rc1/lib/python3.11/site-packages/ib_async/client.py:228](http://localhost:8888/home/lc/.pyenv/versions/3.11.0rc1/lib/python3.11/site-packages/ib_async/client.py#line=227), in Client.connectAsync(self, host, port, clientId, timeout)
219 msg = b"API\0" + self._prefix(
220 b"v%d..%d%s"
221 % (
(...)
225 )
226 )
227 self.conn.sendMsg(msg)
--> 228 await asyncio.wait_for(self.apiStart, timeout)
229 self._logger.info("API connection ready")
230 except BaseException as e:
File [~/.pyenv/versions/3.11.0rc1/lib/python3.11/asyncio/tasks.py:497](http://localhost:8888/home/lc/.pyenv/versions/3.11.0rc1/lib/python3.11/asyncio/tasks.py#line=496), in wait_for(fut, timeout)
495 return fut.result()
496 except exceptions.CancelledError as exc:
--> 497 raise exceptions.TimeoutError() from exc
498 finally:
499 timeout_handle.cancel()
TimeoutError:
2024-05-06 14:04:25,965 ib_async.client INFO Disconnected.
Any feedback would be VERY appreciated, strangely the ruby IBKR library that hasn't been updated since 2016 connects properly to my TWS (and even submits trades successfully) so I know it TWS still set up correctly, but I am starting to think I need to reach out and see if I'm the only one.
P.S. I was previously using it with older python versions (3.10) but as you can see above, 3.11 also didn't work.
Oh also I did try hardcoding a longer timeout (double and triple), but that didn't make a difference at all since the Timeout comes the instant the connection opens (attempts to open) on TWS
Those do sound like odd errors. The systems are currently working for me, so I don't think anything is critically broken with the python libraries themselves. Sometimes the TWS or IBKR gateway breaks and requires a full exit/restart cycle to fix itself, but it sounds like you tried across multiple days already.
Are you sure the client id is unique everywhere?
Did you try maybe running the gateway only and not a full TWS?
I assume your ruby tests are also using the same network address? So you're connecting to a remote system and not just something over localhost?
Initial suggestions:
- verify the remote host is actually reachable (no firewall rules causing problems, etc)
- try different client ids (or even client id
0which controls everything) - upgrade your python version from
3.11.0rc1to the latest 3.11 or 3.12 and install the dependencies again:
$ pyenv install -l |grep 3.1\[12]
3.11.0
3.11-dev
3.11.1
3.11.2
3.11.3
3.11.4
3.11.5
3.11.6
3.11.7
3.11.8
3.11.9
3.12.0
3.12-dev
3.12.1
3.12.2
3.12.3
Thank you for the quick response, I had suspected I should/could be using a newer version of python but my pyenv even after updating (and installing again by following the docs) shows this as the result
3.11.0rc1
3.11-dev
3.12-dev
miniconda3-4.3.11
I'm going to keep digging, but if you have any idea why this would be happening. My guess was whatever debian or ubuntu distro I'm using here is too old and it was installed from the pkg manager, which is why I tried installing from scratch as per the docs. Maybe I'll nuke ~.pyenv and start over...
Ok, scratch that last comment, I have confirmed I get the same error on 3.11.9
Traceback (most recent call last):
File "/home/lou/.pyenv/versions/3.11.9/lib/python3.11/asyncio/tasks.py", line 500, in wait_for
return fut.result()
^^^^^^^^^^^^
File "/home/lou/.pyenv/versions/3.11.9/lib/python3.11/asyncio/tasks.py", line 694, in _wrap_awaitable
return (yield from awaitable.__await__())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
asyncio.exceptions.CancelledError
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/lou/.pyenv/versions/3.11.9/lib/python3.11/site-packages/ib_async/ib.py", line 337, in connect
return self._run(
^^^^^^^^^^
File "/home/lou/.pyenv/versions/3.11.9/lib/python3.11/site-packages/ib_async/ib.py", line 381, in _run
return util.run(*awaitables, timeout=self.RequestTimeout)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/lou/.pyenv/versions/3.11.9/lib/python3.11/site-packages/ib_async/util.py", line 357, in run
result = loop.run_until_complete(task)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/lou/.pyenv/versions/3.11.9/lib/python3.11/asyncio/base_events.py", line 654, in run_until_complete
return future.result()
^^^^^^^^^^^^^^^
File "/home/lou/.pyenv/versions/3.11.9/lib/python3.11/site-packages/ib_async/ib.py", line 1973, in connectAsync
await self.client.connectAsync(host, port, clientId, timeout)
File "/home/lou/.pyenv/versions/3.11.9/lib/python3.11/site-packages/ib_async/client.py", line 228, in connectAsync
await asyncio.wait_for(self.apiStart, timeout)
File "/home/lou/.pyenv/versions/3.11.9/lib/python3.11/asyncio/tasks.py", line 502, in wait_for
raise exceptions.TimeoutError() from exc
TimeoutError
I haven't yet tried the standalone gateway, I'd like to ideally avoid it since I would use my implementation side by side with TWS running.
Also, to answer your other questions I do usually use random to generate the clientId, and I always try a few times and also hard code it to something unique.
Yes, this is over my local network so not just localhost but I know there are no firewall issues because the ruby library connects from all the different machines I've tried (just like ib_insync did for over 2 years lol).
Looks like I need to really dig into the asyncio lib and figure this out :-|
I also tried python v 3.12.3 by using icli and poetry and reproduced the problem there.
2024-05-15 23:39:02.093 | INFO | icli.helpers:<module>:53 - Futures Next Roll-Forward Date: 2024-06-17
2024-05-15 23:39:02.922 | INFO | icli.cli:reconnect:2680 - Connecting to IBKR API...
2024-05-15 23:39:07.039 | ERROR | icli.cli:reconnect:2741 - [] Failed to connect to IB Gateway, trying again...
2024-05-15 23:39:14.050 | ERROR | icli.cli:reconnect:2741 - [] Failed to connect to IB Gateway, trying again...
2024-05-15 23:39:21.062 | ERROR | icli.cli:reconnect:2741 - [] Failed to connect to IB Gateway, trying again...
2024-05-15 23:39:28.072 | ERROR | icli.cli:reconnect:2741 - [] Failed to connect to IB Gateway, trying again...
^C2024-05-15 23:39:33.335 | ERROR | icli.cli:reconnect:2741 - [] Failed to connect to IB Gateway, trying again...
^C2024-05-15 23:39:34.196 | WARNING | icli.cli:reconnect:2751 - Exit requested during sleep. Goodbye.
Just baffled because it does seem like it should be some misconfiguration in my TWS but I haven't changed the settings in months (and they are backed up) plus the whole ruby library working in the same situation leads me to think its something in asyncio
maybe its better if you share part of your code that relates to connecting.. and also share your TWS API config settings..
from what i can gather above i'm still not very sure what's the steps to replicate your scenario.. is the prompt asking you to allow connection to TWS still popping up? or since 2 weeks ago you seen it its no longer showing?
Having the same problem here:
Ubuntu 24.04-amd64 Python 3.12.3 ib_async 1.0.3
`python3 main.py API connection failed: TimeoutError() Traceback (most recent call last): File "/usr/lib/python3.12/asyncio/tasks.py", line 520, in wait_for return await fut ^^^^^^^^^ File "/usr/lib/python3.12/asyncio/futures.py", line 287, in await yield self # This tells Task to wait for completion. ^^^^^^^^^^ File "/usr/lib/python3.12/asyncio/tasks.py", line 385, in __wakeup future.result() File "/usr/lib/python3.12/asyncio/futures.py", line 198, in result raise exc asyncio.exceptions.CancelledError
`
Not with my previously working TWS setup, but a fresh install of IB Gateway worked for me so I would suggest using that if at all possible for your implementation.
On Tue, Jul 23, 2024 at 12:12 PM Crespillo @.***> wrote:
@loucal https://github.com/loucal by any chance, did you solve this problem?
— Reply to this email directly, view it on GitHub https://github.com/ib-api-reloaded/ib_async/issues/18#issuecomment-2245664388, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAH2HMVJYPKLX7N3DM4TYXLZNZ6IRAVCNFSM6AAAAABHXDCJ46VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDENBVGY3DIMZYHA . You are receiving this because you were mentioned.Message ID: @.***>