OPFS Hangs when using firefox

[rickg-hcl]

Please include the following in your bug report:

Version of emscripten/emsdk: emcc -v emcc (Emscripten gcc/clang-like replacement + linker emulating GNU ld) 4.0.9 (e93c648a75ee38bb6e021c4c16fd18cf8f9896e9) clang version 21.0.0git (https:/github.com/llvm/llvm-project 2f05451198e2f222ec66cec4892ada0509519290) Target: wasm32-unknown-emscripten Thread model: posix

Failing command line in full: emcc -s WASMFS -s PROXY_TO_PTHREAD=1 -s PTHREAD_POOL_SIZE=16 -pthread -o test.html test/wasmfs/wasmfs_opfs.c emrun --browser=firefox test.html

The test runs, but if you reload the page a few times, it hangs.

[kripken]

As this is a Firefox bug, the best place to file it is on their tracker,

https://bugzilla.mozilla.org/enter_bug.cgi?product=Firefox&_gl=1i27ttf_gaNzUxODQ5NzA4LjE3NDQxNDQ4MjY._ga_2VC139B3XV*czE3NTIwOTk4NzYkbzEkZzAkdDE3NTIwOTk4NzYkajYwJGwwJGgw

[brendandahl]

We recently added some deadlock detection to debug builds. You could try using the latest emscripten ./emsdk install tot ./emsdk activate tot and see if gives a backtrace.

[rickg-hcl]

I didn't get a backtrace automatically, but I was able to use the firefox performance tab to get one

and in my product, the main browser thread is hung with this stack:

[rickg-hcl]

After some investigation, I've determined that the issue only occurs when I attempt to use filesystem operations like FS.readFile from the main browser thread while also doing c++ file operations on worker threads. The main browser thread gets stuck waiting. I can open an issue with Firefox, but this really seems to be a synchronization issue in the WASMFS part of emscripten.

[rickg-hcl]

Minimum reproducible testcase. here:

emcc -g -s WASMFS -s PROXY_TO_PTHREAD=1 -s PTHREAD_POOL_SIZE=16  -pthread  -s FORCE_FILESYSTEM -o test.html firefox_test.c

firefox_test.c:

#include <assert.h>
#include <dirent.h>
#include <fcntl.h>
#include <pthread.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <sys/stat.h>
#include <sys/types.h>
#include <unistd.h>
#include <emscripten.h>

#include <emscripten/console.h>
#include <emscripten/wasmfs.h>

// Define WASMFS_SETUP then WASMFS_RESUME to run the test as two separate
// programs to test persistence. Alternatively, define neither and run the full
// test as a single program.

void cleanup(void);

int main_thread_read(const char *filename, const char *contents) {
  return MAIN_THREAD_EM_ASM_INT({
    const contents = FS.readFile(UTF8ToString($0), { encoding: 'utf8' });
    if (contents && contents !== UTF8ToString($1)) {
      return 1;
    } else {
      return 0;
    }
  }, filename, contents);
}

void *thread_run_test(void *arg) {
    int fd, nwritten;
    const char *msg = "Hello from thread, OPFS!\n";
    fd = open("/opfs/working/foo_thread.txt", O_RDWR | O_CREAT | O_EXCL, 0777);
    nwritten = write(fd, msg, strlen(msg));
    assert(nwritten == strlen(msg));

    close(fd);

    for (int i = 0; i < 100; ++i) {
        int fd = open("/opfs/working/foo_thread.txt", O_RDWR | O_EXCL, 0777);
        lseek(fd, 0, SEEK_SET);
        int nwritten = write(fd, msg, strlen(msg));
        assert(nwritten == strlen(msg));
        close(fd);
        usleep(10000); // Sleep for 10ms to allow other threads to run.
    }
    return NULL;
}

int main(int argc, char* argv[]) {
  int err, fd, nwritten;
  const char* msg = "Hello, OPFS!\n";
  pthread_t thread = { 0 };

  emscripten_console_log("starting main");

  backend_t opfs = wasmfs_create_opfs_backend();
  emscripten_console_log("created OPFS backend");

  err = wasmfs_create_directory("/opfs", 0777, opfs);
  assert(err == 0);
  emscripten_console_log("mounted OPFS root directory");

  // Remove old files if they exist.
  cleanup();

  err = mkdir("/opfs/working", 0777);
  assert(err == 0);
  emscripten_console_log("created OPFS directory");

  pthread_create(&thread, NULL, thread_run_test, NULL);


  fd = open("/opfs/working/foo.txt", O_RDWR | O_CREAT | O_EXCL, 0777);
  assert(fd > 0);
  emscripten_console_log("created OPFS file");

  nwritten = write(fd, msg, strlen(msg));
  assert(nwritten == strlen(msg));
  emscripten_console_logf("wrote message: %s (%d)", msg, nwritten);

  err = main_thread_read("/opfs/working/foo.txt", msg);
  assert(err == 0);

  close(fd);
  for (int i = 0; i < 100; ++i) {
    err = main_thread_read("/opfs/working/foo.txt", msg);
    fd = open("/opfs/working/foo.txt", O_RDWR | O_EXCL, 0777);
    lseek(fd, 0, SEEK_SET);
    nwritten = write(fd, msg, strlen(msg));
    assert(nwritten == strlen(msg));
    close(fd);
    usleep(10000); // Sleep for 10ms to allow the thread to run.
  }

  pthread_join(thread, NULL);
  printf("done!\n");
}

void cleanup(void) {
  printf("cleaning up\n");

  unlink("/opfs/working/foo.txt");
  unlink("/opfs/working/foo_thread.txt");
  rmdir("/opfs/working");
  unlink("/opfs/foo.txt");

  assert(access("/opfs/working/foo.txt", F_OK) != 0);
  assert(access("/opfs/working", F_OK) != 0);
  assert(access("/opfs/foo.txt", F_OK) != 0);
}

to test:

emrun --browser=firefox test.html

[kripken]

This could be a WasmFS issue, but I confirmed it hangs in Firefox and not Chrome, so there is a relevant browser difference. But whether it's a bug or a minor timing difference that avoids the bug, would take more investigation.

[rickg-hcl]

This still fails on 4.0.15