Reading files from code

[psychemedia]

trafficstars

If I mount a file system directory, open a notebook in that context, and then try to read a file in that context, e.g. using pandas I get a file not found error.

Is there a way of setting a path either to a file in the file system directory, or a path to a file in the default file system (the browser file directory context, not the shared file-system-access context) when working on a notebook in the shared file-system context?

[martinRenou]

Thanks for opening an issue.

Are you talking about a JupyterLab context or a JupyterLite context?

• In the case of jupyterlite, you will be able to read and write into the "mounted" drive, because we also mount the drive in the Emscripten context (at least concerning jupyterlite-xeus's python kernel or jupyterlite-pyodide).
• In the case of jupyterlab, it will not work. The Python kernel executes its code on the server or an another external machine and it does not know anything about what happens in the front-end. jupyterlab-filesystem-access being a pure front-end package it does not communicate anything to the Python kernel about the mounted directory. I don't know if we can easily workaround this, but it's surely a difficult problem: jupyterlab is multi-client, what do you do if two users mount a local directory? Should we communicate their files to the server for the Python kernel to read and write? I don't think there is a good (and safe) solution to this problem.

[psychemedia]

JupyterLab ("full" server). Ok - so jupyterlab-filesystem-access is purely a UI tool in the browser, rather than a mount, although changes made to files in the UI will be saved back to the desktop.

FWIW, the use case is an educational environment, where students are accessing a remote server, The remote server does have file persistence, but it is also useful for students to have local copies of their files. Being able to access and run local files using the remotely served UI is really useful.

I suppose a workaround is for students to mirror data files by uploading them to the remote server, the kernel presumably seeing that file system as code is executed.

I guess another solution might be to upload eg data files that are intended to be read from executed code to a temporary directory on the server? Or maybe changing the background to a file listed in the jupyterlab-file-system-access file listing if a file of the same name exists the current directory path as seen on the server and as such was readable from code?

[MRYingLEE]

In Jupyterlite with Pyodide kernel, I tried to access the mounted files.

def show_file_contents(file_path):
    try:
        with open(file_path, 'r', encoding='utf-8') as file:
            contents = file.read()
            print(contents)
    except FileNotFoundError:
        print(f"File not found: {file_path}")
    except Exception as e:
        print(f"An error occurred: {e}")

file_path = "FileSystemAccess:client-patch.py"

But an issue happened that "File not found: FileSystemAccess:client-patch.py". Actually I copied the path from the menu item "Copy Path" in the Jupyterlite file browser.

Maybe there are some conventions to refer the local files.

Please clarify. Thanks,

[martinRenou]

Have you tried simply:

def show_file_contents(file_path):
    try:
        with open(file_path, 'r', encoding='utf-8') as file:
            contents = file.read()
            print(contents)
    except FileNotFoundError:
        print(f"File not found: {file_path}")
    except Exception as e:
        print(f"An error occurred: {e}")

file_path = "client-patch.py"

[MRYingLEE]

I tried before and I failed also.

Because there is another built-in filesystem, which is based on IndexedDB, so there must be some conventions for the code to know it is a local file instead of an IndexedDB file.

[martinRenou]

Because there is another built-in filesystem, which is based on IndexedDB, so there must be some conventions for the code to know it is a local file instead of an IndexedDB file.

Similar to classic usage, it depends on where your Notebook lives. If your notebook lives in the directory exposed by jupyterlab-filesystem-access, it has access to local files there. If it lives in the IndexedDB file system, it has access to files on the indexedDB

[MRYingLEE]

I expected a local file system can be accessed from any code after an approval after I read https://pyodide.org/en/stable/usage/file-system.html#experimental-using-the-native-file-system-in-the-browser.

Now I understand the underlying file system is the same as that for the notebook and I tested successfully.

Thanks,

[juntyr]

I tried the JupyerLite demo of this repo in Chrome and allowed access to a local folder. While I was able to see all files in the sidebar, opening them from a pyodide kernel notebook didn’t work. Where is the local file system mounted?

[martinRenou]

opening them from a pyodide kernel notebook didn’t work

Where was the notebook? If it's not in the local folder it does not have access to the local folder

[juntyr]

I tried it with a notebook from the local folder and that didn't work either, the current working dir was still the general JupyterLite drive. When I tried to open a notebook in a subdirectory of the local folder, launching the kernel failed since that folder was not available in the filesystem.

Interestingly, the behaviour you describe does work with the jupyterlab-github extension, where opening a notebook within the readonly repo allows that notebook access to the repo folder (though seemingly in a sub-filesystem in the middle of nowhere since its current path is just "." and listing the contents of ".." lists nothing and listing the "/drive" also lists nothing even though the main JupyterLite drive does contain files).

I think I had expected that the JupyterLite drive would be mounted at "/drive" and work regardless and that other drives like the file system access one would also be mounted at public paths within the kernel FS, e.g. "/drive-local"