AaronC81
Make pip integration optional and replace by history search
Personally, if I get a NameError with foo and foo is not a module, it is way more likely that this was because I forgot to do from foobar import foo than because I forgot to install a module. (In fact, I can't ever think of a time where I'd have wanted to pip install something from inside IPython.) So I'd appreciate if the pip integration was made optional (say, by a module-level flag).
An idea may be to retrieve the history (ip = get_ipython(); ip.history_manager.get_tail(ip.history_load_length, raw=False)) and look for from bar import foo (via regexes or better via the ast module), and then offer them as suggestions.
So I think there's at least another reason why auto-pip'ing is a bad idea: there's a bunch of modules where the module names does not correspond to the PyPI name; e.g. sklearn/skimage. Just to make things worse, there's actually an "empty" sklearn project on PyPI (https://pypi.python.org/pypi/sklearn) which can be pip-installed, probably to even greater confusion.
Meanwhile, I have an implementation for the history search:
import ast
def get_imports():
imports = set()
class Visitor(ast.NodeVisitor):
def visit_Import(self, node):
imports.update((alias.name, alias.asname) for alias in node.names)
self.generic_visit(node)
def visit_ImportFrom(self, node):
if not node.level: # Skip relative imports.
imports.update((node.module, alias.name, alias.asname)
for alias in node.names)
self.generic_visit(node)
ip = get_ipython()
for _, _, entry in (
ip.history_manager.get_tail(ip.history_load_length, raw=False)):
try:
parsed = ast.parse(entry)
except SyntaxError:
continue
Visitor().visit(parsed)
return imports
(py3, but py2 should be similar) You get a set of pairs (modulename, alias) (for "import foo as bar") and triplets (for "from foo import bar as baz") (in either case the alias may be None).
I guess you should call this when the module is loaded and cache the result for the session?
A nice side effect is that you can also remove the "special-cased" numpy/scipy/etc. aliases: they can be directly picked up from the history!
That's cool - I've added a pip flag on my local copy, so I'll integrate this function and push.
Okay, check out the fromimport branch. You can disable pip functionality using the PIP_ENABLED constant at the top of import_wrapper.py, and it also uses the function you wrote. Certainly needs more testing, and caching the result of the function is a good idea.
This doesn't match against "import foo as bar". A few more ideas:
- I'd probably get rid of the common_others mapping (because the user's commonly used abbreviations are going to get picked from the history anyways -- for example I never import scipy as sp).
- If the import is successful, I would insert it in the IPython history db so that it doesn't get forgotten for later sessions.
- I would hide the traceback if the import is successful (because the short message from AutoImport itself contains just as much information in a much shorter form); likewise I would hide the AutoImport message if it cannot find any match.
- I would try to parse the input string and only try to autoimport names that appear as
<name>.<attr>in it (this can easily be found using another AST walker). This should avoid a ton of line noise when I try, say,plot(x, y)and forgot to definex.
The idea is to make this as transparent as possible: just type your commands; if you forgot an import and it can be autoimported you get a short message saying that this was done and that's it.
Thanks for the feedback. I'm planning to do a large code refactor soon (it's a bit of a mess), so I'll accommodate these changes.
Okay, many of these changes are now in the v2 branch, along with much neater code. Needs some testing before merging to master though.
I assume it's not in a finished state yet? There's still a bunch of things apparently left to complete.
Feature-wise, I think it's done, although I may have missed something. It's not been extensively tested yet, but in my experience it's usable.