[Ubuntu 18 LTS + Anaconda3] Pigar seems to not find all imports in a project.

[JPGlaser]

I am currently refining a project and need to generate a requirements.txt. For some reason, when I run pigar on the folder containing the main scripts and module src files, it only returns the following:

# Automatically generated by https://github.com/damnever/pigar.

# test/cut_encounters.py: 19,20,21,22,23,24,25,29,30
# test/gen_RandEncounters.py: 19,20,21,22,23,24,25,29,30
# test/gen_scatterDB.py: 12,13,14,15,16,17,18,19
# test/get_init-final_database.py: 13,14,15,16,17,18,19,20
# test/sim_cluster.py: 34,35,36,37,38,39,40,43,44,47,48,49,50,51,52
# test/sim_encounters.py: 27,28,29,30,31,32,33,37,38,39
# test/src/tycho/analysis_old.py: 19,20,21,22,23,24,25,28,29,32,33,34,35
# test/src/tycho/create.py: 14,15,16,17,18,19,20,23,24,25
# test/src/tycho/encounter_db.py: 17
# test/src/tycho/multiples3.py: 7,8,9,10,11,12,14,15,16,17,18,19,2077
# test/src/tycho/multiples4.py: 7,8,9,10,11,12,14,15,16,17,18,19,2068
# test/src/tycho/read.py: 18,19,20,21,22,23,24,28,29
# test/src/tycho/stellar_systems.py: 26,27,28,29,30,31,32,35
# test/src/tycho/util.py: 26,27,28,29,30,31,32,34
# test/src/tycho/write.py: 18,19,20,21,22,23,24,27,28
amuse_twobody == 13.1.0

# test/sim_cluster.py: 10
# test/sim_encounters.py: 8,45
# test/src/tycho/analysis_old.py: 42,44,45,46
# test/src/tycho/create.py: 10
# test/src/tycho/read.py: 14
# test/src/tycho/stellar_systems.py: 11,224,225,226,227,235
# test/src/tycho/util.py: 11
# test/src/tycho/write.py: 13
matplotlib == 3.1.3

# test/cut_encounters.py: 7
# test/gen_RandEncounters.py: 7
# test/gen_scatterDB.py: 1
# test/get_init-final_database.py: 1
# test/sim_cluster.py: 9
# test/sim_encounters.py: 7
# test/src/tycho/analysis_old.py: 9
# test/src/tycho/create.py: 9
# test/src/tycho/multiples3.py: 2
# test/src/tycho/multiples4.py: 2
# test/src/tycho/read.py: 13
# test/src/tycho/stellar_systems.py: 10
# test/src/tycho/util.py: 10
# test/src/tycho/write.py: 12,15
numpy == 1.18.1

# test/gen_scatterDB.py: 4,5,6
# test/get_init-final_database.py: 4,6,7
# test/src/tycho/stellar_systems.py: 15,16,17
scipy == 1.4.1

This is no-where close to the correct number of imported modules. Am I executing this incorrectly?

~ Joe G.

[jlhervy]

I have the same problem, I use pigar on a big torch project, and it doesnt even get close to finding all the imports. torch is not even added, though it says :

"torch referenced from: blablabla Some of them may be not installed in local environment. Try to search PyPI for the missing modules and filter some unnecessary modules? (y/[N]) "

And if I answer yes, it outputs a requirements file without torch and many modules...

[jlhervy]

I found that the output of pigar varies in function of which virtualenv you run it from. If all the requirements are already installed, it seems that pigar won't output them in the requirements file. Doesn't it defeat the purpose ? The goal is to output all the necessary requirements regardless of whether they are already installed, right ?

[damnever]

@JPGlaser FYI, I am guessing you are facing the same issue with @jlhervy

This is no-where close to the correct number of imported modules

The goal is to output all the necessary requirements regardless of whether they are already installed, right ?

This is the limitation of pigar, it can not find every package correctly only by the import name, for example:

import a

def myfunc(whatever):
    print(a.take(whatever))

Maybe both pkg_a and pkg_a_fork offer the same import name a, even worse, they may offer the same function name take with different logic internally.

You can read the FAQ, I will add more explanations for that.

[jlhervy]

Alright, and what about the different output when you use pigar from different virtual environments ?

For instance, if pigar outputs a requirements.txt with numpy, and you run pip install numpy, then re-run pigar, it won't output numpy in the requirements.txt. Is this to be expected ?

[damnever]

@jlhervy

and what about the different output when you use pigar from different virtual environments ?

Please read the FAQ, it is the same issue.

For instance, if pigar outputs a requirements.txt with numpy, and you run pip install numpy, then re-run pigar, it won't output numpy in the requirements.txt. Is this to be expected ?

If this bug is true, please provide a minimal environment to reproduce it.