scala-native-wasm

scala-native-wasm copied to clipboard

Hi Shadaj... I've been experimenting with your fork most of the day and so far it's looking pretty stable. I've even been able to create custom externs and pass strings via pointers (I know println works).. for example.

@native.extern
object libc {
  def cool_console_log(length: Int, value:  CString): Unit = native.extern
}

object Main {
  def main(args: Array[String]): Unit = Zone { implicit z =>
    val s = "Hello World: Scala Native!!!----"
    val converted = native.toCString(s)

    libc.cool_console_log(s.getBytes(StandardCharsets.UTF_8).length, converted)
  }
}

and in JS land ....

cool_console_log: function(size, data) {
      var chars = HEAPU8.subarray(data, data+size);
      var text = "";

      var i;
      for (i = 0; i < chars.length; i++) {
          text += String.fromCharCode(chars[i]);
      }

      console.log(text);
  },

There's really very little difference between using emscripten from c++ or Scala... and I prefer Scala. So my question is this.. what's left to be done to land your PR? I suspect once your PR is merged and people find out a number of developers might jump on board to assist....

Kind Regards

/W

Hi @shadaj ... I am still working with this and finally found an issue. It looks like the string handling in Scala native is handled by a C library CRE2 which is not compiling correctly to wasm. An easy test is to simply use the string.split method.. You'll see a number of underfined symbol warnings on the console and when you try to run the wasm it will abort with a fatal exception... I'd be happy to look into this if it's not something you already know about but I thought I would ask before spending any amount of time on it.... Kind Regards. /W

Hi! So to answer the first question, right now the first step is to merge in 32 bit platform support. This example is based on my first strategy to recompile sources for each word size, but this puts a lot of the burden on library authors. To fix this, I'm now working on a different approach that switches between platforms at link time.

For the regex problems, it might be possible to statically link cre2 to make it available to WASM.

@shadaj thank you for your response.. I will look into statically linking cre2. How else can I be of assistance? I would very much like to see you land this. Kind Regards.

@shadaj ... I also noticed that a very basic example was using about 100MB of ram on initial load in the browser.. I tracked the issue down to this...

// Dummy GC that maps chunks of 4GB and allocates but never frees.

// Map 64MB
//#define CHUNK (64 * 1024 * 1024L)
#define CHUNK (4 * 1024 * 1024L)

I reverted this line to the original and now it's only using 34MB initially.... is there a reason you increased it? Thanks /W

@shadaj ... ok so static linking works with a little tweak to the re2 library header. There is an instance of an int64_t which, obviously, doesn't exist here so I just switched it to an int and everything compiled fine.. I'll have to investigate that a bit further to make sure that's a reasonable thing to do.

However, I'm re-thinking whether or not this is a good approach in general. Adding this library significantly increased the compiled output; it added about 400K (without optimisation) so I'm wondering if perhaps we might want to take a page from the Scala.js project and see about implementing this in js since there's already a solid regex implementation in the browser. I'll take a look to see how difficult that will be marshalling back and forth across the wasm / js boundary.

Kind Regards /W

Hey Shadaj. How do you see the future of this project? Is it realistic to expect - even an experimental branch of - SN to compile to wasm in the near future? I would love to code a small sound server in Scala, but with the negative signals from Sébastien about commitment of SN to wasm support, I wonder if this is feasible.

#3 seems like an outstanding step, if you want to take a stab, @WolfieWerewolf 😇