depfinder
depfinder copied to clipboard
Published
20 hours ago •
ericdill
ericdill
SyntaxError: invalid character in identifier
What in the world is going on here. @ocefpaf @CJ-Wright @tonyfast anyone have cycles to help me figure out what's going on?
- clone jupyter-incubator/sparkmagic (
hub clone jupyter-incubator/sparkmagic) - run depfinder over the
hdijupyterutilssubdir (depfinder ./hdijupyterutils)
/home/centos/miniconda/envs/depfinder/lib/python3.7/site-packages/depfinder/main.py:67: YAMLLoadWarning: calling yaml.lo
ad() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full
details.
pkgutil.get_data(__name__, 'pkg_data/pkg_data.yml').decode()
Traceback (most recent call last):
File "/home/centos/miniconda/envs/depfinder/bin/depfinder", line 10, in <module> sys.exit(cli()) File "/home/centos/miniconda/envs/depfinder/lib/python3.7/site-packages/depfinder/cli.py", line 191, in cli
deps = simple_import_search(file_or_dir, remap=not args.no_remap)
File "/home/centos/miniconda/envs/depfinder/lib/python3.7/site-packages/depfinder/main.py", line 332, in simple_import_search
for mod, path, catcher in catchers:
File "/home/centos/miniconda/envs/depfinder/lib/python3.7/site-packages/depfinder/main.py", line 293, in iterate_over_
library
yield parse_file(full_file_path)
File "/home/centos/miniconda/envs/depfinder/lib/python3.7/site-packages/depfinder/main.py", line 262, in parse_file
catcher = get_imported_libs(code)
File "/home/centos/miniconda/envs/depfinder/lib/python3.7/site-packages/depfinder/main.py", line 236, in get_imported_
libs
tree = ast.parse(code) File "/home/centos/miniconda/envs/depfinder/lib/python3.7/ast.py", line 35, in parse
return compile(source, filename, mode, PyCF_ONLY_AST)
File "<unknown>", line 1 # Distributed under the terms of the Modified BSD License.
^
SyntaxError: invalid character in identifier
I edited my local install of depfinder to print out the files it was parsing:
depfinder hdijupyterutils -v master :: 3d :: ⬢
2019-07-22 15:58:52,115 - depfinder - DEBUG - Treating hdijupyterutils as a directory and recursively searching it for p
ython files
DEBUG:depfinder:Treating hdijupyterutils as a directory and recursively searching it for python files
2019-07-22 15:58:52,115 - depfinder - DEBUG - Setting PACKAGE_NAME global variable to hdijupyterutils
DEBUG:depfinder:Setting PACKAGE_NAME global variable to hdijupyterutils
2019-07-22 15:58:52,115 - depfinder - DEBUG - Reading file: hdijupyterutils/setup.py
DEBUG:depfinder:Reading file: hdijupyterutils/setup.py
2019-07-22 15:58:52,116 - depfinder - DEBUG - Reading file: hdijupyterutils/hdijupyterutils/events.py
DEBUG:depfinder:Reading file: hdijupyterutils/hdijupyterutils/events.py
2019-07-22 15:58:52,117 - depfinder - DEBUG - Reading file: hdijupyterutils/hdijupyterutils/eventshandler.py
DEBUG:depfinder:Reading file: hdijupyterutils/hdijupyterutils/eventshandler.py
2019-07-22 15:58:52,117 - depfinder - DEBUG - Reading file: hdijupyterutils/hdijupyterutils/utils.py
DEBUG:depfinder:Reading file: hdijupyterutils/hdijupyterutils/utils.py
2019-07-22 15:58:52,117 - depfinder - ERROR - Error encountered reading file: hdijupyterutils/hdijupyterutils/utils.py
Traceback (most recent call last):
File "/home/ericdill/dev/depfinder/depfinder/main.py", line 297, in iterate_over_library
yield parse_file(full_file_path)
File "/home/ericdill/dev/depfinder/depfinder/main.py", line 265, in parse_file
catcher = get_imported_libs(code)
File "/home/ericdill/dev/depfinder/depfinder/main.py", line 238, in get_imported_libs
tree = ast.parse(code)
File "/home/ericdill/miniconda/envs/depfinder/lib/python3.7/ast.py", line 35, in parse
return compile(source, filename, mode, PyCF_ONLY_AST)
File "<unknown>", line 1
# Distributed under the terms of the Modified BSD License.
^
SyntaxError: invalid character in identifier
So it seems like these files have some invalid characters in them, but i've got no idea how to find out which characters are "invalid". Also this seems insane because the library clearly works... help!
After a bit more digging, it seems that some of the sparkmagic files have \ufeff characters at the beginning of them:
bad_file = '/home/ericdill/dev/sparkmagic/hdijupyterutils/hdijupyterutils/log.py'
with open(bad_file, 'r', encoding='utf-8') as f:
content = f.read()
print(content)
Shows:
'\ufeff# Distributed under the terms of the Modified BSD License.\n\nfrom __future__ import print_function\nimport logging\nimport logging.config\n\nfrom .constants import LOGGING_CONFIG_CLASS_NAME\n\n\nclass Log(object):\n """Logger for magics. A small wrapper class around the configured logger described in the configuration file"""\n def __init__(self, logger_name, logging_config, caller_name):\n logging.config.dictConfig(logging_config)\n \n assert caller_name is not None\n self._caller_name = caller_name\n self.logger_name = logger_name\n self._getLogger()\n\n def debug(self, message):\n self.logger.debug(self._transform_log_message(message))\n\n def error(self, message):\n self.logger.error(self._transform_log_message(message))\n\n def info(self, message):\n self.logger.info(self._transform_log_message(message))\n\n def _getLogger(self):\n self.logger = logging.getLogger(self.logger_name)\n\n def _transform_log_message(self, message):\n return u\'{}\\t{}\'.format(self._caller_name, message)\n\n\ndef logging_config():\n return {\n u"version": 1,\n u"formatters": {\n u"magicsFormatter": {\n u"format": u"%(asctime)s\\t%(levelname)s\\t%(message)s",\n u"datefmt": u""\n }\n },\n u"handlers": {\n u"magicsHandler": {\n u"class": LOGGING_CONFIG_CLASS_NAME,\n u"formatter": u"magicsFormatter",\n u"home_path": "~/.hdijupyterutils"\n }\n },\n u"loggers": {\n u"magicsLogger": {\n u"handlers": [u"magicsHandler"],\n u"level": u"DEBUG",\n u"propagate": 0\n }\n }\n } \n'
Not quite sure yet how to fix this issue
I think so, I think this was the same weird starting character that I was running into before. I did not run the test though, so reopening.