clickhouse-sqlalchemy
clickhouse-sqlalchemy copied to clipboard
session hangout without any error output
Describe the bug https://github.com/xzkostyan/clickhouse-sqlalchemy/blob/master/clickhouse_sqlalchemy/drivers/http/transport.py#L102
self.timeout = float(timeout) if timeout is not None else None
with this line code, is there any issue here when timeout is None, then self.timeout is None? when self.timeout is None, means code below the post method with self.timout this inarg will not timeout forever?
r = self.http.post(
self.db_url, auth=self.auth, params=params, data=data,
stream=stream, timeout=self.timeout, headers=self.headers,
verify=self.verify, cert=self.cert
)
I met a problem below, with a batch of files need to parse and parsed data to insert into clickhouse. in the python script loop the files, every file build a database connection session for inserting data; after running the script a long time, then the run will hangout and without raising any error info. manual stop the run, and with output: site-packages\clickhouse_sqlalchemy\drivers\http\transport.py, line 169, in _send r = self.http.post( site-packages\requests\sessions.py, line 577, in post return self.request("POST", url, data=data, json=json, **kwargs) site-packages\requests\sessions.py, line 529, in request resp = self.send(prep, **send_kwargs) site-packages\requests\sessions.py, line 645, in send r = adapter.send(request, **kwargs) site-packages\requests\adapters.py, line 501, in send raise ConnectionError(err, request=request) requests.exceptions.ConnectionError: (‘Connection aborted.’, RmeoteDisconnected('Remote end closed connection without response'))
The clickhosue server maybe sometime miss handle some request, so without response. But with this situation, i think the corresponding request will timeout and raise error. but now with this hangout issue, is there any relation with the timeout parameter? coz is None, so the session will not timeout, and always wait the response from clickhouse server side no matter how long time?
To Reproduce as long as the script running of file loop operation doesn't finish, the hangout issue always occurs, just a time issue.
Expected behavior session with a default timeout config, if not provide timeout with the inarg, then use the default value as timeout setting, once a response is not reveived after timeout time, then raise error instead of hangout.
Versions
python 3.8.5 clickhouse-sqlalchemy 0.2.0 requests 2.27.1
engine = create_engine(dbsession.bind.url, echo=False, connect_args={'timeout': 600}, poolclass=NullPool)
session = sessionmaker(bind=engine)()
the seesion settings as above