byterun
byterun copied to clipboard
nedbat
Abstract Virtual Machine Support in byterun
I used byterun as the basis for an abstract VM for some work here at Google and I wanted to offer the changes back to you. This also fixes some bug, notably classes work correctly after the first commit. However the bulk of the changes are to allow abstract interpretation to be done.
-Arthur Peters (working from inside Google)
Oh I should mention something here. I have only tested this on PY2.7. Since that's my target here. I did make an attempt to stay portable, but I'm sure I messed it up somewhere. So a PY3 fix commit will probably be in order. I should be able to produce that in a few weeks if needed (once I'm done with my google internship, so I have more time).
I don't have tox setup yet. However I will when I have time to devote to this. I am aware of it. Thanks for the reminder.
Spectator note: If you guys set up with travis, I think they process your tox.ini to run the tests on all platforms.
@arthurp This is amazing! Thanks for all this work. Reviewing this would be easier if we could have separate PRs for the various aspects, though I understand that's difficult to accomplish sometimes. BTW: setting up tox is dead simple (well, once you have Python 3.3 installed also)
$ pip install -r requirements.txt
$ tox
@arthurp Could you tell us more about what you used this for in Google? It will help us understand the motivation for the refactoring.
A apologize for the giant PR and big commits. It's partly just a bad habit of mine (doing everything and then pushing), but also an artifact of trying to make sure that every internal google commit is testable. I don't think I have a good way to take this apart at this point. Though if github supports it I might well be able to push each of my 3 main commits as a separate PR which would help somewhat.
Would you be willing to glance at the commits and tell me if that would be useful? It would really get weird in terms of the commit history it would produce I think, so I don't want to do it unless it's useful.
About the project: This is related to the previous google project here pytypedecl (https://github.com/google/pytypedecl). The idea is to produce a type language for python. This work with byterun is to create a system that can do type inference on python program. An abstract interpreter is a good way to do static analysis, hence all the abstraction, so I can replace each operation with abstract versions that track the information we are interested in.
@arthurp I think I'd like to do this: generalize byterun so that it can be subclassed as you want, but let's keep the specific subclasses out of the byterun repo. So for example, your Control Flow Graph could be a separate repo that uses and subclasses byterun's classes. What should we expect from you in terms of working more on this pull request to get it to a common understanding?
This makes a lot of sense. Just so I understand: You want to integrate my changes to pyvm2, but I should lift abstractvm and other things into a dependent project. I can do that but probably not for another few weeks. I'm in my last weeks at good and I'm quite busy.
But once I get started: Do you want to minimize the changes to pyvm2? Or directly abstract it? I think it would be possible to push a lot of the changes I made in pyvm2 into abstractvm. However it might increase code duplication. What would be your preference?
@arthurp yes, you have the right idea. I'm not sure what changes from pyvm2 could go into AbstractVm, but I definitely don't like "increased code duplication". Take a stab at the best way you think, and we can go from there.