trufflesqueak icon indicating copy to clipboard operation
trufflesqueak copied to clipboard

Network unavailable

Open psvensson opened this issue 3 years ago • 57 comments

I've donlnoaded and started the latest version (TruffleSqueak-22.1.0.image) running under GraalVm for OpenJDk 17 on Linux Mint 20.2.

It starts fine and I can run som eexamples, but as soon as I try to do anything involving the network (like starting a monitcello installer), the image hangs and eventually produces a debugger where it's obvious that the result was nil.

psvensson avatar Jul 07 '22 17:07 psvensson

Hi @psvensson, thanks for raising this. Unfortunately, TruffleSqueak's implementation of the SocketPlugin is currently incomplete and still has a few issues, so it is not surprising that some networking tasks fail. The problem is that we're emulating what the SocketPlugin does using Java NIO and that's not always working.

Ultimately, it would be nice to implement the SocketPlugin based on a POSIX backend, similar to GraalPython (https://github.com/hpi-swa/trufflesqueak/issues/39#issuecomment-1134566709).

fniephaus avatar Jul 08 '22 07:07 fniephaus

Hi Fabio, thanks for the quick reply. OK, but then it is a known issue and you also know where it can be fixed. I know a bit of Smalltalk and a bit of Java, but not much about GraalVM or indeed truffleSqueak. I will probably fail but will see what I can do :)

I do not know Python so would start to see if I could find a good enough solution fixing existing bugs / implementing what needs to implemented.

I assume that I start looking here; https://github.com/hpi-swa/trufflesqueak/tree/main/src/de.hpi.swa.trufflesqueak/src/de/hpi/swa/trufflesqueak/nodes/plugins/network

Do you have any more hints or ideas on how to go about this, obvious shortlist of what is not working or something like that?

psvensson avatar Jul 08 '22 07:07 psvensson

Yes, it is a known issue. It just never was pressing enough to fix because, well, you can do networking via interop with Ruby or other programming languages. :wink:

I don't think you need to know much about Python. GraalPython is, just like TruffleSqueak, implemented in Java using Truffle. What you could do is copy over the PosixSupportLibrary and its implementations (e.g., NFIPosixSupport and/or EmulatedPosixSupport) and re-implement the SocketPlugin primitives.

An example: primitiveResolverGetAddressInfo is currently not implemented, I think, and uses getaddrinfo under the hood in the OpenSmalltalkVM (here's primitiveResolverGetAddressInfo and here's the getaddrinfo() call in the Unix impl.). So in TruffleSqueak, the primitive could be implemented using getaddrinfo from the POSIX backend. If you have troubles doing that, you could copy over just the getaddrinfo impl from the EmulatedPosixBackend and use that to implement primitiveResolverGetAddressInfo.

One more tip: If you select the TruffleSqueak test image the first time you start TruffleSqueak, you can also look at the original Slang sources of the SocketPlugin, which may be helpful as well.

I don't want to discourage you in any way but I think it's fair to warn you that this is not really a beginner project. Please feel free to give it a shot and asked questions!

fniephaus avatar Jul 08 '22 08:07 fniephaus

Yes, of course. But it would be more easy for first-time users to come to what otherwise works as a normal Squeak environment, and then learn a bit at a time from there.

Thank you! This is much more info that I had hoped for. I am not a beginner, really. I rage-quite Java in 2005 and reacquainted myself with Smalltalk since 2015 or so, and have done quite a bit of network programming (although not often this low-level). I did teach Networking Technologies in most of the 90s as well. I might survive. We'll see :) .

I will definitely ask. By chance my vacation start tomorrow also, so good winds ahead..

psvensson avatar Jul 08 '22 08:07 psvensson

Just thought I could post my proceedings here, for reference.

I have been bale to compile from command-line both GraalVM and truffleSqueak according to instructions, using the proper JVM downloaded by mx.

It then did prove possible to connect eclipse as a remote debugger (using an already defined target for this - thanks) to connect to a truffleSqueak instance using according to the development instructions. I then set a lot of breakpoints in SqueakSocket.java and SqueakTCPSocket.java.

It turns out that name resolution works well, I'm able to set breakpoints in SqueackTCPSocket::receiveDataFrom where an address is present.

But there things are stuck and I'm trying to figure out why. It feels like receive never returns, and then eventually it times out and truffleSqueak gets back nil and throws an exception. The odd hting is that I can see that data is received, and follow it a couple of times. I'm still trying to get to grips with Eclipse deugging again, but I'm fairly sure that something comes across.

trufflesqueak_net_debug_2_ 2022-07-09 09-46-51

psvensson avatar Jul 09 '22 08:07 psvensson

Hmm, I might juts as well have started properly inside truffleSqueak, though, as when I look more closely, it does actually receive data, the main problem seems to be that the connectiontimes out before data is fully read, or that it thinks so.

trufflesqueak_debug_net_3 2022-07-09 11-31-50

psvensson avatar Jul 09 '22 09:07 psvensson

Great that you're able to dive into networking in TruffleSqueak!

I remember that there are some issues with certain websites. I just had a quick look and the non-SSL version of the website seems to work fine (WebClient httpGet: 'http://dn.se'). Maybe this has also something to do with the SqueakSSL plugin?

In case you only need to send some simple http requests, you can also implement a subclass of WebClient using Java, Python, Ruby, or some other GraalVM language.

fniephaus avatar Jul 09 '22 10:07 fniephaus

Ahaa, good catch! yes, this seems to be just about https indeed. It might be SqueakSSL, but I also saw some comments in the SecureSocketStream, which says:

"Note: The loop here is necessary to catch cases where a TLS packet is split among TCP packets. In this case we would pull the first portion of the TLS packet here but receiveAvailableData would return nothing since the contents of the packet can't be decoded until the rest has come in."

So I'll check that out first. It feels like that for SSL, something thinks there is data to be read when it really isn't any, so I hpoe to narrow down where soon.

And yes, I could call out to node.js for example, but my goal is to see if it would be possible to lift truffleSqueak to a level where it's more or less as usable as the standard distribution, but with ultra-mega-superpowers. So I'll continue to weld here in the cellar for a while longer :)

psvensson avatar Jul 09 '22 10:07 psvensson

Silly question, but what is the easiest way to get some logging output from the Java/vm code? I tried something like this;

protected final boolean isDataAvailable() throws IOException {
        selector.selectNow();
        final Set<SelectionKey> keys = selector.selectedKeys();
        System.out.println("isDataAvailable keys length = " + keys.size());
        for (final SelectionKey key : keys) {
            if (key.isReadable()) {
                LogUtils.SOCKET.finer(() -> this + " data available");
                return true;
            }
        }

        LogUtils.SOCKET.finer(() -> this + " no data available");
        return false;
    }

And checked bout stdout and the console tab in Eclipse so of course nothing showed (but it does get called, I can set a breakpoint there, for example).

Since data transport function get called many times, debugging doesn't give the whole picture I need to see where things go awry, so I do need a way to print things over time. In Squeak land Transcript works well as usual, of course.

There are also logging options to set for the vm when starting it, but I don't understand what to write here; From the help:

--log.[logger].level=<String> Set language log level to OFF, SEVERE, WARNING, INFO, CONFIG, FINE, FINER, FINEST or ALL

What are possible values of [logger] here?

psvensson avatar Jul 10 '22 12:07 psvensson

Not a silly question at all, looks like we haven't documented this yet. You've seen the list of loggers, and the way you use enable is like this:

# enable only the socket logger on the FINER level
trufflesqueak --log.smalltalk.socket.level=FINER
# enable all loggers at the same time on the ALL level (this is very verbose)
trufflesqueak --log.smalltalk.level=ALL

fniephaus avatar Jul 10 '22 12:07 fniephaus

Thanks! OK, hmm. I do not see any output anywhere when I do that (even with the ALL option).

Also, I tried to do --log.java.level=ALL but nothing happened with that either. It sounded like it should work and I am 100% for this, but more magic is needed, apparently

psvensson avatar Jul 10 '22 12:07 psvensson

For the ongoing research I'm now fairly convinced that the culprit is the SqueakSocket::isDataAvailable method in teh plugin, which I think does not flag that data is available when it is.

psvensson avatar Jul 10 '22 12:07 psvensson

Can you check that --log.smalltalk.primitives.level=ALL or some other logger works for you? Are you sure you're running the code that you are seeing? It's sometimes easy to forget to rebuild with mx.

BTW, I recommend something like this:

cd trufflesqueak
mx --dy /compiler build # build TruffleSqueak and Graal compiler
../graal/sdk/latest_graalvm_home/bin/trufflesqueak # always points to the latest TruffleSqueak build
# change TruffleSqueak code...
mx --dy /compiler build # should rebuild only TruffleSqueak
../graal/sdk/latest_graalvm_home/bin/trufflesqueak --log.smalltalk.socket.level=FINER

Bonus points for setting $GRAALVM_HOME to your /path/to/graal/sdk/latest_graalvm_home, which allows you to use this instead:

$GRAALVM_HOME/bin/trufflesqueak --log.smalltalk.socket.level=FINER

fniephaus avatar Jul 10 '22 13:07 fniephaus

and this would let you run with the Java debugger:

$GRAALVM_HOME/bin/trufflesqueak --vm.Xrunjdwp:transport=dt_socket,server=y,address=8000,suspend=y

fniephaus avatar Jul 10 '22 13:07 fniephaus

Aha! Wow, of course. I had all the time happily just started trufflesqueak without specifying which one, so of course I ran the earlier installation of trufflesqueak in my classpath from when I downloaded it while trying things out.

Now, when I correctly start using the following line;

$GRAALVM_HOME/bin/trufflesqueak --vm.Xrunjdwp:transport=dt_socket,server=y,address=8000,suspend=y ../../../Installs/graalvm-ce-java17-22.1.0/languages/smalltalk/resources/TruffleSqueakTestImage-6.0alpha-20288b-64bit.image --log.smalltalk.socket.level=FINER

I do see a lot of logging, including my System.out.println. Thank you, the car is moving again :)

psvensson avatar Jul 10 '22 13:07 psvensson

However, now that am using the correct and locally built truffleSqueak, I am starting to see a lot of DNUs for getLoopAndCount . image

They appear after I've get the usual exception on not being able to go to the https site, and after that appear every time I try to do anything with a calltarget browser (like looking at any code, so I then need to restart again. Any ideas what it might be?

psvensson avatar Jul 10 '22 15:07 psvensson

It seems like you're using the CallTargetBrowser without opening the right modules for allow for introspection. Note that this also only works when running with the Graal compiler.

If you don't need to use the CallTargetBrowser, I suggest you switch back to the standard Browser via, for example, SystemBrowser askForDefault and then pick "Browser".

fniephaus avatar Jul 10 '22 15:07 fniephaus

Thanks a lot for the help

psvensson avatar Jul 10 '22 15:07 psvensson

I have now run into another problem which is very confusing. I merely made the suggested changes (and changed browser), saved the image and then restarted. Now, after I start truffleSqueak and then tries to attach the Eclipse debugger, I get this;

$ $GRAALVM_HOME/bin/trufflesqueak --vm.Xrunjdwp:transport=dt_socket,server=y,address=8000,suspend=y ../../../Installs/graalvm-ce-java17-22.1.0/languages/smalltalk/resources/TruffleSqueakTestImage-6.0alpha-20288b-64bit.image --log.smalltalk.level=FINER
WARNING: Unknown module: jdk.internal.vm.compiler specified to --add-opens
Listening for transport dt_socket at address: 8000
[To redirect Truffle log output to a file use one of the following options:
* '--log.file=<path>' if the option is passed using a guest language launcher.
* '-Dpolyglot.log.file=<path>' if the option is passed using the host Java launcher.
* Configure logging using the polyglot embedding API.]
[engine] WARNING: The polyglot context is using an implementation that does not support runtime compilation.
The guest application code will therefore be executed in interpreted mode only.
Execution only in interpreted mode will strongly impact the guest application performance.
For more information on using GraalVM see https://www.graalvm.org/java/quickstart/.
To disable this warning the '--engine.WarnInterpreterOnly=false' option or use the '-Dpolyglot.engine.WarnInterpreterOnly=false' system property.
[trufflesqueak] Running TruffleSqueakTestImage-6.0alpha-20288b-64bit.image on Interpreted...
ERROR: java.lang.NullPointerException: Cannot invoke "de.hpi.swa.trufflesqueak.image.SqueakImageChunk.setObject(Object)" because "specialChunk" is null
org.graalvm.polyglot.PolyglotException: java.lang.NullPointerException: Cannot invoke "de.hpi.swa.trufflesqueak.image.SqueakImageChunk.setObject(Object)" because "specialChunk" is null
	at de.hpi.swa.trufflesqueak.image.SqueakImageReader.initPrebuiltConstant(SqueakImageReader.java:303)
	at de.hpi.swa.trufflesqueak.image.SqueakImageReader.initObjects(SqueakImageReader.java:354)
	at de.hpi.swa.trufflesqueak.image.SqueakImageReader.run(SqueakImageReader.java:90)
	at de.hpi.swa.trufflesqueak.image.SqueakImageReader.load(SqueakImageReader.java:75)
	at de.hpi.swa.trufflesqueak.SqueakImage$SqueakImageNode.execute(SqueakImage.java:40)
	at <smalltalk> null(Unknown)
	at org.graalvm.sdk/org.graalvm.polyglot.Context.eval(Context.java:399)
	at de.hpi.swa.trufflesqueak.launcher.TruffleSqueakLauncher.execute(TruffleSqueakLauncher.java:126)
	at de.hpi.swa.trufflesqueak.launcher.TruffleSqueakLauncher.launch(TruffleSqueakLauncher.java:82)
	at org.graalvm.launcher.AbstractLanguageLauncher.launch(AbstractLanguageLauncher.java:296)
	at org.graalvm.launcher.AbstractLanguageLauncher.launch(AbstractLanguageLauncher.java:121)
	at org.graalvm.launcher.AbstractLanguageLauncher.runLauncher(AbstractLanguageLauncher.java:168)
Original Internal Error: 
java.lang.NullPointerException: Cannot invoke "de.hpi.swa.trufflesqueak.image.SqueakImageChunk.setObject(Object)" because "specialChunk" is null
	at de.hpi.swa.trufflesqueak.image.SqueakImageReader.initPrebuiltConstant(SqueakImageReader.java:303)
	at de.hpi.swa.trufflesqueak.image.SqueakImageReader.initObjects(SqueakImageReader.java:354)
	at de.hpi.swa.trufflesqueak.image.SqueakImageReader.run(SqueakImageReader.java:90)
	at de.hpi.swa.trufflesqueak.image.SqueakImageReader.load(SqueakImageReader.java:75)
	at de.hpi.swa.trufflesqueak.SqueakImage$SqueakImageNode.execute(SqueakImage.java:40)
	at org.graalvm.truffle/com.oracle.truffle.api.impl.DefaultCallTarget.callDirectOrIndirect(DefaultCallTarget.java:85)
	at org.graalvm.truffle/com.oracle.truffle.api.impl.DefaultCallTarget.call(DefaultCallTarget.java:102)
	at org.graalvm.truffle/com.oracle.truffle.polyglot.PolyglotContextImpl.eval(PolyglotContextImpl.java:1326)
	at org.graalvm.truffle/com.oracle.truffle.polyglot.PolyglotContextDispatch.eval(PolyglotContextDispatch.java:63)
	at org.graalvm.sdk/org.graalvm.polyglot.Context.eval(Context.java:399)
	at de.hpi.swa.trufflesqueak.launcher.TruffleSqueakLauncher.execute(TruffleSqueakLauncher.java:126)
	at de.hpi.swa.trufflesqueak.launcher.TruffleSqueakLauncher.launch(TruffleSqueakLauncher.java:82)
	at org.graalvm.launcher.AbstractLanguageLauncher.launch(AbstractLanguageLauncher.java:296)
	at org.graalvm.launcher.AbstractLanguageLauncher.launch(AbstractLanguageLauncher.java:121)
	at org.graalvm.launcher.AbstractLanguageLauncher.runLauncher(AbstractLanguageLauncher.java:168)
Caused by: Attached Guest Language Frames (1)

I have tried to rebuild the truffleSqueak vm and then all of it, but to no avail. I then thought that the changes file of the image might have been damaged (maybe I terminated it too soon?) so I removed it and tried again, but got the same thing again.

It is very strange, I have not had time to do anything in particular (what I can remember). Is this something you have come across before?

As a last resort I'll download everything again and begin from scratch.

psvensson avatar Jul 10 '22 17:07 psvensson

It seems the image has been damanged somehow. I believe the first positional argument is the image and all following are considered image arguments. So to avoid confusion, I'd either move the log option before the image file or drop the image argument. If no image is provided, TruffleSqueak will automatically use the one in the resources directory.

To get a fresh test image, simply empty "Installs/graalvm-ce-java17-22.1.0/languages/smalltalk/resources/" and run TruffleSqueak again. It should asked you again to select an image.

fniephaus avatar Jul 10 '22 18:07 fniephaus

Thank you, that did the trick.

psvensson avatar Jul 10 '22 19:07 psvensson

Some observations while spelunking the code;

  1. When rebuilding the truffleSqueak VM, the downloaded image gets (understandable) deleted and must be downloaded anew.
  2. It is only the 'test' image (option 2) which has logging enabled, so trying to set logging to anything at all while using (for example) the recommended image does not work.
  3. Copying a downloaded squeak image to a safe location and using the when starting truffleSqueak for debugging is not quite enough, since it then complains that it cannot find its sources

image

This just in case anyone else stumbles on the same things in the future :)

The Graal socket python code is not like the smalltalk one, so they don't compare directly. But it's very little code in both cases. The problem arises when in the SecureSocketStream tries to read further from the socket but is blocked until timeout. It reads initial data but then in a subsequent step just sits there.

It has to be an assumption on what the underlying layer should do, or not, but it is not very obvious (yet) what it is. I'm reading up on java.nio and getting my bearings.

Will be travelling for a couple of days now, so getting back to this later in the week.

psvensson avatar Jul 11 '22 10:07 psvensson

So, while waiting for packing to be done, I noticed something interesting. When using the WebClient to go directly to 'https://dn.se' it reads a bit then times out, but when going to the basic vanilla http url 'http://dn.se', I notice that it first reads from port 80 but then apparently gets redirected to https --- and that works.


Squeak SecureSocketsStream socket vm plugin log:
---------------------------------------------------

isDataAvailable key = channel=java.nio.channels.SocketChannel[connected local=/192.168.86.207:51928 remote=/52.29.209.1:443], selector=sun.nio.ch.EPollSelectorImpl@57b75756, interestOps=13, readyOps=5 isReadable = true
PrimSocketReceiveDataBufCountNode::receiveData vanilla called
receiveData key = channel=java.nio.channels.SocketChannel[connected local=/192.168.86.207:51928 remote=/52.29.209.1:443], selector=sun.nio.ch.EPollSelectorImpl@57b75756, interestOps=13, readyOps=5 isReadable = true
receiveData received 4096 bytes
PrimSocketReceiveDataBufCountNode::doCount finalCount = 4096
isDataAvailable key = channel=java.nio.channels.SocketChannel[connected local=/192.168.86.207:51928 remote=/52.29.209.1:443], selector=sun.nio.ch.EPollSelectorImpl@57b75756, interestOps=13, readyOps=5 isReadable = true
isDataAvailable key = channel=java.nio.channels.SocketChannel[connected local=/192.168.86.207:51928 remote=/52.29.209.1:443], selector=sun.nio.ch.EPollSelectorImpl@57b75756, interestOps=13, readyOps=5 isReadable = true
PrimSocketReceiveDataBufCountNode::receiveData vanilla called
receiveData key = channel=java.nio.channels.SocketChannel[connected local=/192.168.86.207:51928 remote=/52.29.209.1:443], selector=sun.nio.ch.EPollSelectorImpl@57b75756, interestOps=13, readyOps=5 isReadable = true
receiveData received 300 bytes
PrimSocketReceiveDataBufCountNode::doCount finalCount = 300
isDataAvailable key = channel=java.nio.channels.SocketChannel[connected local=/192.168.86.207:51928 remote=/52.29.209.1:443], selector=sun.nio.ch.EPollSelectorImpl@57b75756, interestOps=13, readyOps=4 isReadable = false
isDataAvailable key = channel=java.nio.channels.SocketChannel[connected local=/192.168.86.207:51928 remote=/52.29.209.1:443], selector=sun.nio.ch.EPollSelectorImpl@57b75756, interestOps=13, readyOps=4 isReadable = false
isDataAvailable key = channel=java.nio.channels.SocketChannel[connected local=/192.168.86.207:51928 remote=/52.29.209.1:443], selector=sun.nio.ch.EPollSelectorImpl@57b75756, interestOps=13, readyOps=4 isReadable = false
isDataAvailable key = channel=java.nio.channels.SocketChannel[connected local=/192.168.86.207:51928 remote=/52.29.209.1:443], selector=sun.nio.ch.EPollSelectorImpl@57b75756, interestOps=13, readyOps=4 isReadable = false
isDataAvailable key = channel=java.nio.channels.SocketChannel[connected local=/192.168.86.207:51928 remote=/52.29.209.1:443], selector=sun.nio.ch.EPollSelectorImpl@57b75756, interestOps=13, readyOps=4 isReadable = false


SocketStream sock vm plugin log:
--------------------------------
isDataAvailable key = channel=java.nio.channels.SocketChannel[connected local=/192.168.86.207:60030 remote=/52.29.209.1:80], selector=sun.nio.ch.EPollSelectorImpl@58a63629, interestOps=13, readyOps=4 isReadable = false
isDataAvailable key = channel=java.nio.channels.SocketChannel[connected local=/192.168.86.207:60030 remote=/52.29.209.1:80], selector=sun.nio.ch.EPollSelectorImpl@58a63629, interestOps=13, readyOps=5 isReadable = true
isDataAvailable key = channel=java.nio.channels.SocketChannel[connected local=/192.168.86.207:60030 remote=/52.29.209.1:80], selector=sun.nio.ch.EPollSelectorImpl@58a63629, interestOps=13, readyOps=5 isReadable = true
PrimSocketReceiveDataBufCountNode::receiveData vanilla called
receiveData key = channel=java.nio.channels.SocketChannel[connected local=/192.168.86.207:60030 remote=/52.29.209.1:80], selector=sun.nio.ch.EPollSelectorImpl@58a63629, interestOps=13, readyOps=5 isReadable = true
receiveData received 112 bytes
PrimSocketReceiveDataBufCountNode::doCount finalCount = 112
isDataAvailable key = channel=java.nio.channels.SocketChannel[connected local=/192.168.86.207:60030 remote=/52.29.209.1:80], selector=sun.nio.ch.EPollSelectorImpl@58a63629, interestOps=13, readyOps=4 isReadable = false
isDataAvailable key = channel=java.nio.channels.SocketChannel[connected local=/192.168.86.207:40666 remote=/2.19.113.133:443], selector=sun.nio.ch.EPollSelectorImpl@680d4a6a, interestOps=13, readyOps=5 isReadable = true
PrimSocketReceiveDataBufCountNode::receiveData vanilla called
receiveData key = channel=java.nio.channels.SocketChannel[connected local=/192.168.86.207:40666 remote=/2.19.113.133:443], selector=sun.nio.ch.EPollSelectorImpl@680d4a6a, interestOps=13, readyOps=5 isReadable = true
receiveData received 3608 bytes
PrimSocketReceiveDataBufCountNode::doCount finalCount = 3608
isDataAvailable key = channel=java.nio.channels.SocketChannel[connected local=/192.168.86.207:40666 remote=/2.19.113.133:443], selector=sun.nio.ch.EPollSelectorImpl@680d4a6a, interestOps=13, readyOps=4 isReadable = false
isDataAvailable key = channel=java.nio.channels.SocketChannel[connected local=/192.168.86.207:40666 remote=/2.19.113.133:443], selector=sun.nio.ch.EPollSelectorImpl@680d4a6a, interestOps=13, readyOps=5 isReadable = true
PrimSocketReceiveDataBufCountNode::receiveData vanilla called
receiveData key = channel=java.nio.channels.SocketChannel[connected local=/192.168.86.207:40666 remote=/2.19.113.133:443], selector=sun.nio.ch.EPollSelectorImpl@680d4a6a, interestOps=13, readyOps=5 isReadable = true
receiveData received 242 bytes
PrimSocketReceiveDataBufCountNode::doCount finalCount = 242
isDataAvailable key = channel=java.nio.channels.SocketChannel[connected local=/192.168.86.207:40666 remote=/2.19.113.133:443], selector=sun.nio.ch.EPollSelectorImpl@680d4a6a, interestOps=13, readyOps=4 isReadable = false
isDataAvailable key = channel=java.nio.channels.SocketChannel[connected local=/192.168.86.207:40666 remote=/2.19.113.133:443], selector=sun.nio.ch.EPollSelectorImpl@680d4a6a, interestOps=13, readyOps=4 isReadable = false
isDataAvailable key = channel=java.nio.channels.SocketChannel[connected local=/192.168.86.207:40666 remote=/2.19.113.133:443], selector=sun.nio.ch.EPollSelectorImpl@680d4a6a, interestOps=13, readyOps=5 isReadable = true
isDataAvailable key = channel=java.nio.channels.SocketChannel[connected local=/192.168.86.207:40666 remote=/2.19.113.133:443], selector=sun.nio.ch.EPollSelectorImpl@680d4a6a, interestOps=13, readyOps=5 isReadable = true
PrimSocketReceiveDataBufCountNode::receiveData vanilla called
receiveData key = channel=java.nio.channels.SocketChannel[connected local=/192.168.86.207:40666 remote=/2.19.113.133:443], selector=sun.nio.ch.EPollSelectorImpl@680d4a6a, interestOps=13, readyOps=5 isReadable = true
receiveData received 4096 bytes
PrimSocketReceiveDataBufCountNode::doCount finalCount = 4096
isDataAvailable key = channel=java.nio.channels.SocketChannel[connected local=/192.168.86.207:40666 remote=/2.19.113.133:443], selector=sun.nio.ch.EPollSelectorImpl@680d4a6a, interestOps=13, readyOps=5 isReadable = true
isDataAvailable key = channel=java.nio.channels.SocketChannel[connected local=/192.168.86.207:40666 remote=/2.19.113.133:443], selector=sun.nio.ch.EPollSelectorImpl@680d4a6a, interestOps=13, readyOps=5 isReadable = true
PrimSocketReceiveDataBufCountNode::receiveData vanilla called
receiveData key = channel=java.nio.channels.SocketChannel[connected local=/192.168.86.207:40666 remote=/2.19.113.133:443], selector=sun.nio.ch.EPollSelectorImpl@680d4a6a, interestOps=13, readyOps=5 isReadable = true
receiveData received 4096 bytes
PrimSocketReceiveDataBufCountNode::doCount finalCount = 4096
isDataAvailable key = channel=java.nio.channels.SocketChannel[connected local=/192.168.86.207:40666 remote=/2.19.113.133:443], selector=sun.nio.ch.EPollSelectorImpl@680d4a6a, interestOps=13, readyOps=5 isReadable = true
isDataAvailable key = channel=java.nio.channels.SocketChannel[connected local=/192.168.86.207:40666 remote=/2.19.113.133:443], selector=sun.nio.ch.EPollSelectorImpl@680d4a6a, interestOps=13, readyOps=5 isReadable = true
.... etc

That is super-weird though, so it means that something special happens when connecting initially using TLS. Oh well, vacation it is, then.

psvensson avatar Jul 11 '22 13:07 psvensson

Good find, that is indeed interesting! And enjoy your vacation! I'm currently occupied with some other things anyway until the end of next week, but it'd be great to fix this at some point and happy to help as much as my time allows.

fniephaus avatar Jul 11 '22 13:07 fniephaus

Since the truffleSqueak SockePlugin obviously works in some cases, I have been trying to find the difference in handling from the SqueakSSL side between going directly to https and being redirected to https. I had several great theories, but none stuck unfortunately.

I now tried other sites than the one I've been testing, and interesting things happens.

vanilla http request WebClient httpGet: 'http://google.com' fails silently with a Transcript complaint of 'Unknown cookie field: SameSite' after redirecting from google.com to www.google.com.

On the other had, going directly to https to github; WebClient httpGet: 'https://github.com' works!

The only reason I can think of is that it is the WebClient which is wonky.

Trying to update Squeak fails happily as usual, trying to access http://source.squeak.org/trunk which is just http:// and accessible through a browser.

it might be that it is the WebClient which is not correctly hotted-up to dance the latest dances? However, that can't be true since it does actually work in a vanilla VM squeak 6.0.

Very X-files right now :)

psvensson avatar Jul 14 '22 09:07 psvensson

TruffleSqueak's image and test image are based on a fixed Squeak trunk image that we haven't updated in a while. So maybe worth doing these experiments in a Squeak 6.0 release image since WebClient might have seen some updates?

fniephaus avatar Jul 14 '22 09:07 fniephaus

Yes, that is a good idea. Actually, I was wondering how to create new truffleSqueak images and a bit about how one would go about to write the Smalltalk parts (if needed) to get truffleSqueak to work with another Smalltalk (I was thinking Pharo, for example). If you could add some notes on that to the docs that would be great. And then I could also help out more

psvensson avatar Jul 14 '22 09:07 psvensson

Hmm, I have also been trying to find the 'gu' utility in the graalvm directory in the trufflesqueak-build folder I created, but I can't seem to find it.

In the pre-built graalvm installation it can be found in bin/gu which is a link to lib/installer/bin/gu but nothing can be found in the mx built graalvm folder in the same places.

I have then tried to understand if there are any specific mx build command that can be used to build and 'install' the gu command, but I don't see ny there either. To be fair, I've never used mx before so I'm mostly plodding along semi-randomly.

If there is an easy way to to this, it would be great, because then I can start installing node.js and see if I can install packages (which was my next step anyway)

psvensson avatar Jul 14 '22 10:07 psvensson

Yes, that is a good idea. Actually, I was wondering how to create new truffleSqueak images and a bit about how one would go about to write the Smalltalk parts (if needed) to get truffleSqueak to work with another Smalltalk (I was thinking Pharo, for example). If you could add some notes on that to the docs that would be great. And then I could also help out more

Here's how the TruffleSqueak images are created. Maybe it works without any problems with the Squeak 6.0 release, I don't know but it is something we need to look into now that there's a new release.

Regarding Pharo support, it's blocked by at least two problems:

  • They have changed object layouts, so loading the image doesn't work atm. TruffleSqueak's object model needs to be extended to accommodate that.
  • Pharo does drawing and windowing through FFI. TruffleSqueak's FFI needs to be finished, only very few things are working. I'm afraid that alone may be more work than fixing the SocketPlugin.

fniephaus avatar Jul 14 '22 13:07 fniephaus

Thank you, of course i was already documented, sry :) I'll give it a try and see if anything have changed in WebClient to the better. Also, I will of course ask in the GraalVM channels on how to install gu - still mixing projects, it seems

And I can't say I'm surprised that Pharo is a tougher nut to crack. Sometimes it feels like the whole point of something is to be incompatible, when possible. But it is much easier to work with out of the box (code completion, searching for anything, et.c.) Maybe I can find something done for Squeak too, I'll look around.

psvensson avatar Jul 14 '22 14:07 psvensson