Add properties to access V8 values

[qistoph]

These commits add some properties to AST classes to allow access to those values from python.

[buffer]

The PR is ok for me but just a question before merging. As you probably already realized the V8 code included in the tree is quite old and your commit patches it. Are the feature you are including already in some more recent version of the V8 code? Just asking because I would love to align the PyV8 code base with more recent versions of V8 and this could probably be a good occasion for starting such effort.

[qistoph]

I am using the V8 version that was, until recently, available from some SVN URL on https://code.google.com/archive/p/v8/source The top line in the Changelog says 2014-02-10: Version 3.24.35

This is still an old version and I have been trying to update to a more recent one.

The 3.24.35 version contains all the attributes referenced in this PR.

[qistoph]

Yesterday I've managed to compile PyV8 against the current V8 branch. Currently I'm retracing my steps and I'll try to post the details today.

Unfortunately, I just discovered that somehow PyV8 was still building against old v8-binaries, stuck in the PyV8 directory... This was noticed when compiling it on a clean system. These are the steps taken and noted so far though.

https://developers.google.com/v8/build

I'm running Debian testing with a pretty clean, base install. On Debian stable the build failed, probably because it doesn't have gcc-5. Identified requirements: sudo apt-get install curl make binutils libc6-dev libboost-thread-dev

V8 instructions: https://github.com/v8/v8/wiki/Using%20Git

Install depot-tools: https://www.chromium.org/developers/how-tos/install-depot-tools

git clone https://chromium.googlesource.com/chromium/tools/depot_tools.git
export PATH=`pwd`/depot_tools:"$PATH"
gclient
fetch v8
cd v8
make x64.release

After verifying V8 was build by runing out/x64.release/d8, I downloaded and compiled PyV8. Requirements identified: apt-get install gcc g++ python-dev systemtap-sdt-dev

sudo apt-get update
sudo apt-get install gcc-5

Updated the path to standalone.gypi using sed.

Fixed paths in src/V8Internal.h, by searching for the files in the v8 folder.

git clone https://github.com/buffer/pyv8
cd pyv8
sed -i 's#build/standalone\.gypi#gypfiles/standalone.gypi#' setup.py

cat <<EOF >src/V8Internal.h
#pragma once

#undef COMPILER
#undef TRUE
#undef FALSE

#include "src/v8.h"

#include "src/bootstrapper.h"
#include "src/snapshot/natives.h"
#include "src/base/platform/platform.h"
#include "src/ast/scopes.h"

#include "src/debug/debug.h"

//#include "src/serialize.h"
#include "src/ic/stub-cache.h"
#include "src/heap/heap.h"

#include "src/parsing/parser.h"
#include "src/compiler.h"
#include "src/parsing/scanner.h"

#include "src/api.h"

namespace v8i = v8::internal;
EOF

V8_HOME=/path/to/src/v8/ python setup.py build

[buffer]

Thanks for your efforts! I confirm the setup script was modified in order to build the V8 code embedded in the tree. I did it because such release was no more available to download. By the way the trick of setting the env variable V8_HOME allows you to change it but seems like you are already aware of it. Can you share the repo you are working on? I would like to do some tests and even contribute if possible. Thanks again!

[qistoph]

For you convenience I have updated and pushed my repo here: https://github.com/qistoph/pyv8/tree/updateV8

[buffer]

Can you confirm gcc 5 is strictly required? Testing the compilation both on Gentoo and Devuan platforms but both failed (gcc-4.9 in both cases).

[qistoph]

Actually, I can't. This was a difference in the vm that it did compile on from the vm that it didn't. I wrote that down before I discovered that it wasn't the issue though.

According to the v8 compile instructions it needs at least something like 4.6 or 4.8. With just gcc 4 on my vm V8 compiled when compiled alone. It just didn't when compiling it from PyV8.

Op 3 aug. 2016 01:07 schreef "buffer" [email protected]:

Can you confirm gcc 5 is strictly required? Testing the compilation both on Gentoo and Devuan platforms but both failed (gcc-4.9 in both cases).

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/buffer/pyv8/pull/1#issuecomment-237167842, or mute the thread https://github.com/notifications/unsubscribe-auth/AApaZL0TFMNGOao1ls1VhLQoMvXnpOhzks5qcEhIgaJpZM4JXFv9 .

[buffer]

I am experiencing exactly the same issue on all platforms (V8 compiles if compiled alone but not if compiled from PyV8). I suppose it is due to some of the compile options used by PyV8 setup. I will investigate and let you know.

[qistoph]

Any updates in the mean time?

[buffer]

No updates in the mean time. Unfortunately this is an extra busy period for me and I was not able to perform any further test. I would like to be able to compile PyV8 vs. the latest version of V8 and hope to be able start working on it by the end of August. Please if you have some updates about that feel free to share. Thanks for your patience.

[buffer]

Noting here for everyone's convenience...

Yesterday I took a look at the code and performed a few tests on the same machine (so same environment). I noticed a few interesting things:

1. compiling my own PyV8 tree works
2. compiling the latest V8 (standalone) works
3. compiling your PyV8 tree does not work

Items 1 and 3 prove that the environment is not the reason behind the failure. I took a look at the differences wrt items 1 and 3 and realized that the only different things are the includes (because part of different versions of V8). This is an excerpt from PyV8 setup.py

149 include_dirs = [ 150 os.path.join(V8_HOME, 'include'), 151 V8_HOME, 152 os.path.join(V8_HOME, 'src'), 153 ]

I just tried commenting out line 152 and the issues about incomplete types disappeared. I had no time to deeply investigate but it seems like line 151 and 152 can't live together because they end up redefining some symbols.

[77cc33]

https://github.com/qistoph/pyv8/tree/updateV8

Just tried to build this tree at Ubuntu 16.04 with GCC 5.4.0 with no luck

So as described above, I can build V8 directly, but not from PyV8

Maybe some one already have solution to fix this issue?

[buffer]

Good news to share. I have talked with the author of PyV8 and he is currently porting it to the new V8 API. This means that once the effort will be completed (and maybe it will require some weeks of work) there will be no need for this fork anymore. I will take care of helping him with testing in the next weeks. Thanks for your patience.

[hecoding]

Hi! I'm currently interested in PyV8. It's been a while since your last update, how is the port going so far @buffer? Thanks for all your work.

[buffer]

Unfortunately still no news. The porting effort is taking much more time than expected so no relevant updates...

[rstocktox]

Hi,

first, thanks for your amazing effort. 👍

...so it's your tree the preferred choice over the @qistoph one for installing PyV8? Which one has the more recent V8 revision?

Regards

[desertkun]

Just came here to say thank you for your effort!

Don't really have any issues now, but yet it would be nice to have more recent version of V8 just in case!

[buffer]

IMHO we should focus our efforts on this branch

https://github.com/flier/pyv8/tree/v5_api

which attempts to port PyV8 to the latest V8 API version and try to help @flier in finalizing his efforts.

[desertkun]

Would be awesome to hear status from flier himself, because there wasn't much updates lately.

[flier]

Thanks to use and contribute to PyV8 project.

I have evaluated the new V8 API, it has a lot of API and mechanism changes since latest PyV8 version, we need to discard some PyV8 API and features for migration, that may cause compatibility issues.

So, my suggestion is to fix bug and add minor features base on the old version, because it is more stable and good enough for most scenes. If you need my contribution, please let's me known.

On the other hand, I will continue put efforts to the v5_api branch, but I hard to promise the schedule, because my focus has moved to other area. I prefer to split PyV8 to a tiny core and more extensions, to reduce the maintain efforts later.

Thanks :)

[buffer]

@flier I totally agree. I think the best thing we can do is to converge our efforts to your repo and avoid wasting time in maintaining our own trees. During the last months I committed some patches and embedded V8 r19632 in the tree (because V8 SVN repo is not available anymore). Please take a look at these changes at

https://github.com/buffer/pyv8/commits/master

If you agree I would be glad to send a PR.

[flier]

I think the first priority for me is to finish the migration to V5 API, it has been a long, drawn-out affair.

The good news is I have refactor PyV8 to use new GN building system with dynamic library, but there are still a few of crash because PyV8 base on so many V8 internal mechanism.

After that, we could back to discuss how to maintain the old branch, I'm ok to merge PR or use your branch :)

[buffer]

@flier great to hear that. Please let me know if I can help and I will be glad to do it!