PyHive
PyHive copied to clipboard
Need Updates for python 3.10, using `collections.Iterable` as an example
System Info:
Python 3.10.0 | packaged by conda-forge | (default, Nov 20 2021, 02:25:18) [GCC 9.4.0]
pyhive 0.6.4 | pyhd8ed1ab_0 | conda-forge
Logs:
~/anaconda3/envs/****/lib/python3.10/site-packages/pyhive/hive.py in execute(self, operation, parameters, **kwargs)
442 sql = operation
443 else:
--> 444 sql = operation % _escaper.escape_args(parameters)
445
446 self._reset_state()
~/anaconda3/envs/****/lib/python3.10/site-packages/pyhive/common.py in escape_args(self, parameters)
211 return {k: self.escape_item(v) for k, v in parameters.items()}
212 elif isinstance(parameters, (list, tuple)):
--> 213 return tuple(self.escape_item(x) for x in parameters)
214 else:
215 raise exc.ProgrammingError("Unsupported param format: {}".format(parameters))
~/anaconda3/envs/****/lib/python3.10/site-packages/pyhive/common.py in <genexpr>(.0)
211 return {k: self.escape_item(v) for k, v in parameters.items()}
212 elif isinstance(parameters, (list, tuple)):
--> 213 return tuple(self.escape_item(x) for x in parameters)
214 else:
215 raise exc.ProgrammingError("Unsupported param format: {}".format(parameters))
~/anaconda3/envs/****/lib/python3.10/site-packages/pyhive/common.py in escape_item(self, item)
246 elif isinstance(item, basestring):
247 return self.escape_string(item)
--> 248 elif isinstance(item, collections.Iterable):
249 return self.escape_sequence(item)
250 elif isinstance(item, datetime.datetime):
AttributeError: module 'collections' has no attribute 'Iterable'
This is because collections.Iterable
is deprecated and we should use collections.abc.Iterable
instead.
In fact, there has always been an DeprecationWarning
in place:
DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3, and in 3.10 it will stop working
Are there any plans for a code review for such updates?
Can confirm that an environment with Python 3.10 breaks the code, which is working in Python 3.7.11
As mentioned, the issue with collections.Iterable
usage is that that has been moved to collections.abc.Iterable
since python 3.3.
collections.Iterable
is used in the default ParamEscaper
here:
https://github.com/dropbox/PyHive/blob/master/pyhive/common.py#L248
I should really just PR this (not sure they are being accepted since unsupported?), but for those in need of a quick fix, you can just monkey patch this particular problem with this:
collections.Iterable = collections.abc.Iterable
I know PyHive is unsupported. I'm just using the escaping functions to supplement some pyathena
usage, where the escaping is currently extremely thin.
[Off topic] I saw some recommended https://github.com/cloudera/impyla as an alternative. It seems that they provided a similar interface (DB API). I have not tried using it though.
Why is this not fixed for so long?
Addressed in https://github.com/dropbox/PyHive/pull/451