PyCG
PyCG copied to clipboard
vitsalis
Error running pycg.
Hi there,
writing the Fasten-PyPI-plugin, we would need to run pycg locally to analyze missing call graphs on the FASTEN server.
Trying out the tool, I got with several packages this error.
michelescarlato@splinter:~/call-graphs-creation/scipy-1.8.1$ pycg --fasten --package scipy $(find scipy -type f -name "*.py") --product "scipy" --version "1.8.1" --forge "PyPI" --max-iter -1 -o cg.json
Traceback (most recent call last):
File "/home/michelescarlato/.local/bin/pycg", line 8, in <module>
sys.exit(main())
File "/home/michelescarlato/.local/lib/python3.10/site-packages/pycg/__main__.py", line 79, in main
cg.analyze()
File "/home/michelescarlato/.local/lib/python3.10/site-packages/pycg/pycg.py", line 155, in analyze
self.do_pass(PreProcessor, True,
File "/home/michelescarlato/.local/lib/python3.10/site-packages/pycg/pycg.py", line 144, in do_pass
self.import_manager.install_hooks()
File "/home/michelescarlato/.local/lib/python3.10/site-packages/pycg/machinery/imports.py", line 203, in install_hooks
loader = get_custom_loader(self)
File "/home/michelescarlato/.local/lib/python3.10/site-packages/pycg/machinery/imports.py", line 34, in get_custom_loader
class CustomLoader(importlib.abc.SourceLoader):
AttributeError: module 'importlib' has no attribute 'abc'. Did you mean: '_abc'?
Am I doing something wrong?
I see. It might be due to some updates to the class on version 3.10. Can you try running it with version 3.9?
Hi there,
@gdrosos tried it with the Python version 3.9.13.
He used this command:
pycg --fasten --package scipy $(find scipy -type f -name "*.py") --product "scipy" --version "1.8.1" --forge "PyPI" --max-iter -1 -o cg.json
And this is the output:
File "/Users/gdrosos/test/PyCG/pycg/processing/preprocessor.py", line 65, in analyze_submodule
super().analyze_submodule(PreProcessor, modname,
File "/Users/gdrosos/test/PyCG/pycg/processing/base.py", line 484, in analyze_submodule
visitor = cls(fname, imp, *args, **kwargs)
File "/Users/gdrosos/test/PyCG/pycg/processing/preprocessor.py", line 33, in __init__
super().__init__(filename, modname, modules_analyzed)
File "/Users/gdrosos/test/PyCG/pycg/processing/base.py", line 37, in __init__
self.contents = f.read()
File "/opt/homebrew/Cellar/[email protected]/3.9.13/Frameworks/Python.framework/Versions/3.9/lib/python3.9/codecs.py", line 322, in decode
(result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xcf in position 0: invalid continuation byte
He got the same output using numpy version 1.22.4.
Hey @gdrosos and @vitsalis ,
I am trying to help Michele with some problems running pycg but I am facing problems by myself:
Downloading https://pypi.org/project/setuptools/#files (the tar.gz file) untar it and running
pycg --package setuptools --fasten --product "setuptools" --forge "PyPI" --version "0.1" --timestamp 42 $(find setuptools -type f -name "*.py") -o setuptools.json inside the folder setuptools-62.3.2
results in
Traceback (most recent call last):
File "/mnt/stuff/projects/work/pypi-Plugin-venv/bin/pycg", line 11, in <module>
load_entry_point('pycg==0.0.5', 'console_scripts', 'pycg')()
File "/mnt/stuff/projects/work/pypi-Plugin-venv/lib/python3.9/site-packages/pycg-0.0.5-py3.9.egg/pycg/__main__.py", line 79, in main
cg.analyze()
File "/mnt/stuff/projects/work/pypi-Plugin-venv/lib/python3.9/site-packages/pycg-0.0.5-py3.9.egg/pycg/pycg.py", line 155, in analyze
self.do_pass(PreProcessor, True,
File "/mnt/stuff/projects/work/pypi-Plugin-venv/lib/python3.9/site-packages/pycg-0.0.5-py3.9.egg/pycg/pycg.py", line 148, in do_pass
processor.analyze()
File "/mnt/stuff/projects/work/pypi-Plugin-venv/lib/python3.9/site-packages/pycg-0.0.5-py3.9.egg/pycg/processing/preprocessor.py", line 375, in analyze
self.visit(ast.parse(self.contents, self.filename))
File "/usr/lib/python3.9/ast.py", line 50, in parse
return compile(source, filename, mode, flags,
File "/mnt/stuff/projects/work/packagesToBeAnalyzed/setuptools-62.3.2/setuptools/tests/script-with-bom.py", line 1
# -*- coding: utf-8 -*-
^
SyntaxError: invalid non-printable character U+FEFF
Running it one folder above results in an empty call Graph:
{"product": "setuptools", "forge": "PyPI", "generator": "PyCG", "depset": [], "version": "0.1", "timestamp": "42", "modules": {"internal": {}, "external": {}}, "graph": {"internalCalls": [], "externalCalls": [], "resolvedCalls": []}, "nodes": 0}?
What is the Problem?
Hi there,
I report different errors, running different python versions:
michele.scarlato@pycg-tester:~/setuptools-62.3.2$ pycg --package setuptools --fasten --product "setuptools" --forge "PyPI" --version "0.1" --timestamp 42 $(find setuptools -type f -name "*.py") -o setuptools.json
Traceback (most recent call last):
File "/home/michele.scarlato/.local/bin/pycg", line 8, in <module>
sys.exit(main())
File "/home/michele.scarlato/.local/lib/python3.8/site-packages/pycg/__main__.py", line 79, in main
cg.analyze()
File "/home/michele.scarlato/.local/lib/python3.8/site-packages/pycg/pycg.py", line 155, in analyze
self.do_pass(PreProcessor, True,
File "/home/michele.scarlato/.local/lib/python3.8/site-packages/pycg/pycg.py", line 148, in do_pass
processor.analyze()
File "/home/michele.scarlato/.local/lib/python3.8/site-packages/pycg/processing/preprocessor.py", line 375, in analyze
self.visit(ast.parse(self.contents, self.filename))
File "/usr/lib/python3.8/ast.py", line 47, in parse
return compile(source, filename, mode, flags,
File "/home/michele.scarlato/setuptools-62.3.2/setuptools/tests/script-with-bom.py", line 1
# -*- coding: utf-8 -*-
^
SyntaxError: invalid character in identifier
Error in sys.excepthook:
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/apport_python_hook.py", line 57, in apport_excepthook
apt_pkg.DATE
AttributeError: module 'apt_pkg' has no attribute 'DATE'
Original exception was:
Traceback (most recent call last):
File "/home/michele.scarlato/.local/bin/pycg", line 8, in <module>
sys.exit(main())
File "/home/michele.scarlato/.local/lib/python3.8/site-packages/pycg/__main__.py", line 79, in main
cg.analyze()
File "/home/michele.scarlato/.local/lib/python3.8/site-packages/pycg/pycg.py", line 155, in analyze
self.do_pass(PreProcessor, True,
File "/home/michele.scarlato/.local/lib/python3.8/site-packages/pycg/pycg.py", line 148, in do_pass
processor.analyze()
File "/home/michele.scarlato/.local/lib/python3.8/site-packages/pycg/processing/preprocessor.py", line 375, in analyze
self.visit(ast.parse(self.contents, self.filename))
File "/usr/lib/python3.8/ast.py", line 47, in parse
return compile(source, filename, mode, flags,
File "/home/michele.scarlato/setuptools-62.3.2/setuptools/tests/script-with-bom.py", line 1
# -*- coding: utf-8 -*-
^
SyntaxError: invalid character in identifier
michele.scarlato@pycg-tester:~/setuptools-62.3.2$ python3
python3 python3-config python3.8 python3.8-config python3.9
michele.scarlato@pycg-tester:~/setuptools-62.3.2$ python3 --version
Python 3.8.10
michele.scarlato@pycg-tester-instance-3:~/setuptools$ pycg --package setuptools --fasten --product "setuptools" --forge "PyPI" --version "0.1" --timestamp 42 $(find setuptools -type f -name "*.py") -o setuptools.json
Traceback (most recent call last):
File "/home/michele.scarlato/.local/bin/pycg", line 8, in <module>
sys.exit(main())
File "/home/michele.scarlato/.local/lib/python3.9/site-packages/pycg/__main__.py", line 79, in main
cg.analyze()
File "/home/michele.scarlato/.local/lib/python3.9/site-packages/pycg/pycg.py", line 155, in analyze
self.do_pass(PreProcessor, True,
File "/home/michele.scarlato/.local/lib/python3.9/site-packages/pycg/pycg.py", line 146, in do_pass
processor = cls(input_file, input_mod,
File "/home/michele.scarlato/.local/lib/python3.9/site-packages/pycg/processing/preprocessor.py", line 33, in __init__
super().__init__(filename, modname, modules_analyzed)
File "/home/michele.scarlato/.local/lib/python3.9/site-packages/pycg/processing/base.py", line 36, in __init__
with open(filename, "rt") as f:
File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
File "<frozen importlib._bootstrap>", line 982, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 925, in _find_spec
File "<frozen importlib._bootstrap_external>", line 1349, in find_spec
File "<frozen importlib._bootstrap_external>", line 1321, in _get_spec
File "<frozen importlib._bootstrap_external>", line 1477, in find_spec
File "<frozen importlib._bootstrap_external>", line 1434, in _get_spec
File "/home/michele.scarlato/.local/lib/python3.9/site-packages/pycg/machinery/imports.py", line 39, in __init__
ig_obj.create_edge(self.fullname)
File "/home/michele.scarlato/.local/lib/python3.9/site-packages/pycg/machinery/imports.py", line 87, in create_edge
raise ImportManagerError("Can't add edge to a non existing node")
pycg.machinery.imports.ImportManagerError: Can't add edge to a non existing node
michele.scarlato@pycg-tester-instance-3:~/setuptools$ python3 --version
Python 3.9.2
michele.scarlato@pycg-tester-instance-2:~/setuptools-62.3.2$ pycg --package setuptools --fasten --product "setuptools" --forge "PyPI" --version "0.1" --timestamp 42 $(find setuptools -type f -name "*.py") -o setuptools.json
Traceback (most recent call last):
File "/home/michele.scarlato/.local/bin/pycg", line 8, in <module>
sys.exit(main())
File "/home/michele.scarlato/.local/lib/python3.9/site-packages/pycg/__main__.py", line 79, in main
cg.analyze()
File "/home/michele.scarlato/.local/lib/python3.9/site-packages/pycg/pycg.py", line 155, in analyze
self.do_pass(PreProcessor, True,
File "/home/michele.scarlato/.local/lib/python3.9/site-packages/pycg/pycg.py", line 148, in do_pass
processor.analyze()
File "/home/michele.scarlato/.local/lib/python3.9/site-packages/pycg/processing/preprocessor.py", line 375, in analyze
self.visit(ast.parse(self.contents, self.filename))
File "/usr/lib/python3.9/ast.py", line 50, in parse
return compile(source, filename, mode, flags,
File "/home/michele.scarlato/setuptools-62.3.2/setuptools/tests/script-with-bom.py", line 1
# -*- coding: utf-8 -*-
^
SyntaxError: invalid non-printable character U+FEFF
Error in sys.excepthook:
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/apport_python_hook.py", line 57, in apport_excepthook
apt_pkg.DATE
AttributeError: module 'apt_pkg' has no attribute 'DATE'
Original exception was:
Traceback (most recent call last):
File "/home/michele.scarlato/.local/bin/pycg", line 8, in <module>
sys.exit(main())
File "/home/michele.scarlato/.local/lib/python3.9/site-packages/pycg/__main__.py", line 79, in main
cg.analyze()
File "/home/michele.scarlato/.local/lib/python3.9/site-packages/pycg/pycg.py", line 155, in analyze
self.do_pass(PreProcessor, True,
File "/home/michele.scarlato/.local/lib/python3.9/site-packages/pycg/pycg.py", line 148, in do_pass
processor.analyze()
File "/home/michele.scarlato/.local/lib/python3.9/site-packages/pycg/processing/preprocessor.py", line 375, in analyze
self.visit(ast.parse(self.contents, self.filename))
File "/usr/lib/python3.9/ast.py", line 50, in parse
return compile(source, filename, mode, flags,
File "/home/michele.scarlato/setuptools-62.3.2/setuptools/tests/script-with-bom.py", line 1
# -*- coding: utf-8 -*-
^
SyntaxError: invalid non-printable character U+FEFF
michele.scarlato@pycg-tester-instance-2:~/setuptools-62.3.2$ python3 --version
Python 3.9.7
On each machine I run these commands:
pip install pycg
wget https://files.pythonhosted.org/packages/4a/25/ec29a23ef38b9456f9965c57a9e1221e6c246d87abbf2a31158799bca201/setuptools-62.3.2.tar.gz
tar -xvzf setuptools-62.3.2.tar.gz
cd setuptools-62.3.2/
pycg --package setuptools --fasten --product "setuptools" --forge "PyPI" --version "0.1" --timestamp 42 $(find setuptools -type f -name "*.py") -o setuptools.json
Downloading https://pypi.org/project/setuptools/#files (the tar.gz file) untar it and running pycg --fasten --package SETUPTOOLS $(find SETUPTOOLS -type f -name "exerxises.py") --product "SETUPTOOLS" --version "1.8.1" --forge "PyPI" --max-iter -1 -o cg.json **give me the error
format parameter not correct**
how I can get the call graph for exercises.py?
