unicorn-binance-websocket-api
unicorn-binance-websocket-api copied to clipboard
Published
20 hours ago •
LUCIT-Systems-and-Development
LUCIT-Systems-and-Development
Socket timeout and thread exception using unicorn-binance-websocket-api 1.41.0
Version of this library.
1.41.0
Solution to Issue cannot be found in the documentation or other Issues and also occurs in the latest version of this library.
- [X] I checked the documentation and other Issues. I am using the latest version of this library.
Hardware?
Local server/workstation
Operating System?
Windows
Python version?
Python3.9
Installed packages
absl-py 0.15.0
aiohttp 3.7.4
alpaca-trade-api 1.5.0
aniso8601 9.0.1
appdirs 1.4.4
astunparse 1.6.3
async-generator 1.10
async-timeout 3.0.1
attrs 21.4.0
backcall 0.2.0
beautifulsoup4 4.9.3
bpemb 0.3.3
bs4 0.0.1
cachetools 5.0.0
certifi 2021.10.8
cffi 1.15.0
chardet 3.0.4
charset-normalizer 2.0.10
cheroot 8.6.0
clang 5.0
click 8.0.3
cmdstanpy 0.9.68
colorama 0.4.4
commonmark 0.9.1
conllu 4.4.1
convertdate 2.4.0
cryptography 37.0.1
cssselect 1.1.0
cycler 0.11.0
Cython 0.29.23
dateparser 0.7.6
debugpy 1.5.1
decorator 5.1.1
Deprecated 1.2.13
deprecation 2.1.0
dnspython 1.16.0
entrypoints 0.3
ephem 4.1.3
fake-useragent 0.1.11
feedparser 6.0.8
filelock 3.4.2
fix-yahoo-finance 0.1.37
flair 0.10
Flask 2.1.1
Flask-RESTful 0.3.9
flatbuffers 1.12
fonttools 4.29.0
ftfy 6.0.3
gast 0.4.0
gdown 3.12.2
gensim 4.1.2
gnews 0.2.7
google-auth 2.5.0
google-auth-oauthlib 0.4.6
google-pasta 0.2.0
grpcio 1.43.0
h11 0.13.0
h5py 3.1.0
hijri-converter 2.2.2
holidays 0.12
huggingface-hub 0.4.0
idna 3.3
importlib-metadata 4.10.1
ipykernel 6.4.1
ipython 7.31.1
ipython-genutils 0.2.0
itsdangerous 2.1.2
Janome 0.4.1
jaraco.functools 3.5.0
jedi 0.18.0
Jinja2 3.1.1
joblib 1.1.0
jsonschema 4.4.0
jupyter-client 7.1.2
jupyter-core 4.9.1
kaleido 0.1.0.post1
keras 2.8.0
Keras-Preprocessing 1.1.2
keras-tuner 1.1.0
keybert 0.5.0
kiwisolver 1.3.2
konoha 4.6.5
korean-lunar-calendar 0.2.1
kt-legacy 1.0.4
langdetect 1.0.9
libclang 13.0.0
loguru 0.5.3
LunarCalendar 0.0.9
lxml 4.7.1
Markdown 3.3.6
MarkupSafe 2.1.1
matplotlib 3.5.1
matplotlib-inline 0.1.2
more-itertools 8.8.0
mpld3 0.3
mplfinance 0.12.8b9
msgpack 1.0.2
multidict 6.0.2
multitasking 0.0.10
munkres 1.1.4
nbformat 5.1.3
nest-asyncio 1.5.1
nltk 3.6.7
numpy 1.22.1
oauthlib 3.1.1
olefile 0.46
opt-einsum 3.3.0
outcome 1.1.0
overrides 3.1.0
packaging 21.3
pandas 1.4.0
pandas-datareader 0.10.0
pandas-ta 0.3.14b0
parse 1.19.0
parso 0.8.3
patsy 0.5.2
pickleshare 0.7.5
Pillow 9.0.0
pip 21.2.4
plotly 5.6.0
plotly-express 0.4.1
praw 7.5.0
prawcore 2.3.0
prompt-toolkit 3.0.20
prophet 1.0.1
protobuf 3.19.3
psutil 5.9.0
pyasn1 0.4.8
pyasn1-modules 0.2.8
pycparser 2.21
pydot 1.4.2
pyee 8.2.2
Pygments 2.10.0
PyMeeus 0.5.11
pymongo 3.12.3
pyOpenSSL 22.0.0
pyparsing 3.0.7
pyppeteer 1.0.2
PyQt5 5.12.3
PyQt5_sip 4.19.18
PyQtChart 5.12
PyQtWebEngine 5.12.1
pyquery 1.4.3
pyrsistent 0.18.1
PySocks 1.7.1
pystan 2.19.1.1
python-binance 1.0.16
python-dateutil 2.8.2
python-dotenv 0.19.2
python-engineio 3.14.2
python-socketio 4.6.0
pytz 2021.3
pytz-deprecation-shim 0.1.0.post0
pywin32 302
PyYAML 5.4.1
pyzmq 22.3.0
regex 2022.1.18
requests 2.26.0
requests-html 0.10.0
requests-oauthlib 1.3.0
rich 11.2.0
rsa 4.8
sacremoses 0.0.47
scikit-learn 1.0.2
scipy 1.7.3
seaborn 0.11.2
segtok 1.5.11
selenium 4.1.3
sentence-transformers 2.2.0
sentencepiece 0.1.95
setuptools 57.5.0
setuptools-git 1.2
sgmllib3k 1.0.0
simfin 0.8.4
six 1.16.0
smart-open 5.2.1
sniffio 1.2.0
sortedcontainers 2.4.0
soupsieve 2.3.1
sqlitedict 1.7.0
statsmodels 0.13.1
ta 0.9.0
TA-Lib 0.4.24
tabulate 0.8.9
tenacity 8.0.1
tensorboard 2.8.0
tensorboard-data-server 0.6.1
tensorboard-plugin-wit 1.8.1
tensorflow 2.6.0
tensorflow-estimator 2.7.0
tensorflow-gpu 2.8.0
tensorflow-io-gcs-filesystem 0.23.1
termcolor 1.1.0
tf-estimator-nightly 2.8.0.dev2021122109
threadpoolctl 3.0.0
tokenizers 0.10.3
torch 1.10.2
torchvision 0.11.3
tornado 6.1
tqdm 4.62.3
traitlets 5.1.1
transformers 4.15.0
trio 0.20.0
trio-websocket 0.9.2
tweepy 4.5.0
typing-extensions 3.7.4.3
tzdata 2022.1
tzlocal 4.1
ujson 5.1.0
unicodedata2 14.0.0
unicorn-binance-websocket-api 1.41.0
unicorn-fy 0.12.2
update-checker 0.18.0
urllib3 1.26.8
uuid 1.30
v 0.0.0
w3lib 1.22.0
wcwidth 0.2.5
websocket-client 1.2.3
websockets 10.3
Werkzeug 2.0.2
wheel 0.37.1
Wikipedia-API 0.5.4
win32-setctime 1.1.0
wincertstore 0.2
wrapt 1.12.1
wsproto 1.1.0
yahoo-fin 0.8.9.1
yarl 1.7.2
yfinance 0.1.70
zipp 3.7.0
Logging output
Exception in thread Thread-3:
Traceback (most recent call last):
File "C:\Users\dutch\anaconda3\envs\tradesystem1\lib\site-packages\urllib3\connectionpool.py", line 449, in _make_request
six.raise_from(e, None)
File "<string>", line 3, in raise_from
File "C:\Users\dutch\anaconda3\envs\tradesystem1\lib\site-packages\urllib3\connectionpool.py", line 444, in _make_request
httplib_response = conn.getresponse()
File "C:\Users\dutch\anaconda3\envs\tradesystem1\lib\http\client.py", line 1345, in getresponse
response.begin()
File "C:\Users\dutch\anaconda3\envs\tradesystem1\lib\http\client.py", line 307, in begin
version, status, reason = self._read_status()
File "C:\Users\dutch\anaconda3\envs\tradesystem1\lib\http\client.py", line 268, in _read_status
line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
File "C:\Users\dutch\anaconda3\envs\tradesystem1\lib\socket.py", line 704, in readinto
return self._sock.recv_into(b)
File "C:\Users\dutch\anaconda3\envs\tradesystem1\lib\ssl.py", line 1241, in recv_into
return self.read(nbytes, buffer)
File "C:\Users\dutch\anaconda3\envs\tradesystem1\lib\ssl.py", line 1099, in read
return self._sslobj.read(len, buffer)
socket.timeout: The read operation timed out
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Users\dutch\anaconda3\envs\tradesystem1\lib\site-packages\requests\adapters.py", line 439, in send
resp = conn.urlopen(
File "C:\Users\dutch\anaconda3\envs\tradesystem1\lib\site-packages\urllib3\connectionpool.py", line 785, in urlopen
retries = retries.increment(
File "C:\Users\dutch\anaconda3\envs\tradesystem1\lib\site-packages\urllib3\util\retry.py", line 550, in increment
raise six.reraise(type(error), error, _stacktrace)
File "C:\Users\dutch\anaconda3\envs\tradesystem1\lib\site-packages\urllib3\packages\six.py", line 770, in reraise
raise value
File "C:\Users\dutch\anaconda3\envs\tradesystem1\lib\site-packages\urllib3\connectionpool.py", line 703, in urlopen
httplib_response = self._make_request(
File "C:\Users\dutch\anaconda3\envs\tradesystem1\lib\site-packages\urllib3\connectionpool.py", line 451, in _make_request
self._raise_timeout(err=e, url=url, timeout_value=read_timeout)
File "C:\Users\dutch\anaconda3\envs\tradesystem1\lib\site-packages\urllib3\connectionpool.py", line 340, in _raise_timeout
raise ReadTimeoutError(
urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='testnet.binance.vision', port=443): Read timed out. (read timeout=10)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Users\dutch\anaconda3\envs\tradesystem1\lib\threading.py", line 954, in _bootstrap_inner
self.run()
File "C:\Users\dutch\anaconda3\envs\tradesystem1\lib\threading.py", line 892, in run
self._target(*self._args, **self._kwargs)
File "C:\dev\trading\tradesystem1\crypto-bots\binance_trading_bot2.py", line 510, in process_stream_data
handle_price_change(symbol=data['s'], timestamp=data['T'], price=float(data['p']))
File "C:\dev\trading\tradesystem1\crypto-bots\binance_trading_bot2.py", line 401, in handle_price_change
historic_prices_df = get_crypto_minute_prices(symbol)
File "C:\dev\trading\tradesystem1\crypto-bots\binance_trading_bot2.py", line 642, in get_crypto_minute_prices
df = pd.DataFrame(client.get_historical_klines(symbol, '1m', '300 minutes ago UTC'))
File "C:\Users\dutch\anaconda3\envs\tradesystem1\lib\site-packages\binance\client.py", line 934, in get_historical_klines
return self._historical_klines(
File "C:\Users\dutch\anaconda3\envs\tradesystem1\lib\site-packages\binance\client.py", line 973, in _historical_klines
first_valid_ts = self._get_earliest_valid_timestamp(symbol, interval, klines_type)
File "C:\Users\dutch\anaconda3\envs\tradesystem1\lib\site-packages\binance\client.py", line 904, in _get_earliest_valid_timestamp
kline = self._klines(
File "C:\Users\dutch\anaconda3\envs\tradesystem1\lib\site-packages\binance\client.py", line 883, in _klines
return self.get_klines(**params)
File "C:\Users\dutch\anaconda3\envs\tradesystem1\lib\site-packages\binance\client.py", line 868, in get_klines
return self._get('klines', data=params, version=self.PRIVATE_API_VERSION)
File "C:\Users\dutch\anaconda3\envs\tradesystem1\lib\site-packages\binance\client.py", line 371, in _get
return self._request_api('get', path, signed, version, **kwargs)
File "C:\Users\dutch\anaconda3\envs\tradesystem1\lib\site-packages\binance\client.py", line 334, in _request_api
return self._request(method, uri, signed, **kwargs)
File "C:\Users\dutch\anaconda3\envs\tradesystem1\lib\site-packages\binance\client.py", line 314, in _request
self.response = getattr(self.session, method)(uri, **kwargs)
File "C:\Users\dutch\anaconda3\envs\tradesystem1\lib\site-packages\requests\sessions.py", line 555, in get
return self.request('GET', url, **kwargs)
File "C:\Users\dutch\anaconda3\envs\tradesystem1\lib\site-packages\requests\sessions.py", line 542, in request
resp = self.send(prep, **send_kwargs)
File "C:\Users\dutch\anaconda3\envs\tradesystem1\lib\site-packages\requests\sessions.py", line 655, in send
r = adapter.send(request, **kwargs)
File "C:\Users\dutch\anaconda3\envs\tradesystem1\lib\site-packages\requests\adapters.py", line 529, in send
raise ReadTimeout(e, request=request)
requests.exceptions.ReadTimeout: HTTPSConnectionPool(host='testnet.binance.vision', port=443): Read timed out. (read timeout=10)
BinanceWebSocketApiSocket.start_socket(154321d1a832-0ec9-db58-20d6-1d999650, {'trade'}, ['btcusdt']) - Exception ConnectionClosed - error_msg: sent 1011 (unexpected error) keepalive ping timeout; no close frame received
BinanceWebSocketApiManager.stream_is_crashing(154321d1a832-0ec9-db58-20d6-1d999650)
BinanceWebSocketApiManager._create_stream_thread() stream_id=154321d1a832-0ec9-db58-20d6-1d999650 - RuntimeError `error: 11` - error_msg: cannot schedule new futures after interpreter shutdown - Please create an issue: https://github.com/LUCIT-Systems-and-Development/unicorn-binance-websocket-api/issues/new/choose
Processing method?
process_stream_data
Used endpoint?
binance.us
Issue
This happens with binance-us testnet. I ran the websocket connection for about 5 hours when the issue occurred. It may just be an issue on the Binance side. Can you tell me how I could handle this exception and restart the websocket listener? Thanks!
Please post your code!
I guess you have only one stream and are using high_performance=True? Try with setting to False!
Hi, thanks for the feedback. I'm not explicitly setting high_performance.
Here my code:
from unicorn_binance_websocket_api.manager import BinanceWebSocketApiManager
binance_us_websocket_api_manager = BinanceWebSocketApiManager(exchange="binance.us")
def process_stream_data(binance_websocket_api_manager):
while True:
if binance_websocket_api_manager.is_manager_stopping():
exit(0)
oldest_data = binance_websocket_api_manager.pop_stream_data_from_stream_buffer()
is_empty = is_empty_message(oldest_data)
if is_empty:
time.sleep(0.02)
else:
oldest_data_dict = json.loads(oldest_data)
data = oldest_data_dict['data']
# Handle price change
handle_price_change(symbol=data['s'], timestamp=data['T'], price=float(data['p']))
def start_websocket_listener():
global stop_running
channels = {'trade', }
try:
binance_us_websocket_api_manager.create_stream(channels, markets=lc_symbols, api_key=TEST_API_KEY, api_secret=TEST_API_SECRET)
# Start a worker process to move the received stream_data from the stream_buffer to a print function
worker_thread = threading.Thread(target=process_stream_data, args=(binance_us_websocket_api_manager,))
worker_thread.start()
except Exception as e:
logger.error('Failure in worker thread for websocket: ', e)
I dont see the reason sorry.
I need to test this, but it will need a while, i am very busy with other stuff at the moment.
You mix api test key with binance us? I think that is in general not supported?