nix-gl-host icon indicating copy to clipboard operation
nix-gl-host copied to clipboard

Experimental jetson support

Open SomeoneSerge opened this issue 1 year ago • 6 comments

PoC for https://github.com/numtide/nix-gl-host/issues/11 (accidentally, also https://github.com/numtide/nix-gl-host/issues/9, but I don't like my code)

$ nix build -f '<nixpkgs>' --arg 'config' '{ allowUnfree = true; cudaCapabilities = [ "7.2" ]; }' cudaPackages.cuda_compat -o cuda-compat
$ nix shell \
  github:SomeoneSerge/pkgs#pkgsXavier.some-pkgs-py.edm.pythonWith \
  github:SomeoneSerge/nix-gl-host/feat/jetson \
  --command nixglhost -d "$(readlink -f ./cuda_compat/compat)" -d /usr/lib/aarch64-linux-gnu/tegra \
  -- python -m edm.example

SomeoneSerge avatar Dec 19 '23 18:12 SomeoneSerge

Actually, I think a better approach would be to just hide the system's libcuda from LD_LIBRARY_PATH. We already link cuda_compat directly in Nixpkgs, we only need to provide its impure dependencies

SomeoneSerge avatar Dec 22 '23 14:12 SomeoneSerge

Hey! Sorry for the latency.

I don't have any nvidia hardware to test this against currently :( But I think the approach is sound.

On the long run, I envisioned to extract those DSO list from the Python script and move it to a JSON (or any other format) file that users could override on runtime. It doesn't have to happen here, but at least it's written down somewhere :)

As I said on another issue, I sadly don't use this project myself and can't maintain it on the long run. That being said, I know your nickname and I know I can trust you. Would you be interested in getting the commit bit and maintain this project? I'm up to help you during the transition phase.

picnoir avatar Jan 17 '24 08:01 picnoir

On the long run, I envisioned to extract those DSO list from the Python script and move it to a JSON (or any other format) file that users could override on runtime. It doesn't have to happen here, but at least it's written down somewhere :)

One more direction to ponder is that libnvidia-container maintains a similar list

Would you be interested in getting the commit bit and maintain this project? I'm up to help you during the transition phase.

Thanks! Yes, absolutely. I have some time limitations too, but I'm interested in having this project maintained

SomeoneSerge avatar Jan 17 '24 09:01 SomeoneSerge

One more direction to ponder is that libnvidia-container maintains a similar list

I did not consider that. Good idea.

Thanks! Yes, absolutely. I have some time limitations too, but I'm interested in having this project maintained

Seems like I'm not admin on this repo. @zimbatm, could you give the commit bit to @SomeoneSerge? (up until we decide to move this project to nix-community).

As I said in the other post, I can't test the Nvidia code, but feel free to ping me on Matrix/IRC if you need a rubberduck, chat about the problem space a bit. I have more context in mind than what I've written down in the docs :)

picnoir avatar Jan 17 '24 10:01 picnoir

Welcome to the project @SomeoneSerge :)

zimbatm avatar Jan 17 '24 11:01 zimbatm

Thanks @zimbatm! I'm planning to look over these changes again soon-ish and then have a more careful look at the repo. So far my thoughts are that we should do something about the glibc we link in Nixpkgs (LD_XXXXXXX_PATH=/usr/lib shouldn't be breaking things), but even then a tool capable of separating the flies from the meat generating a fine-grained LD_LIBRARY_PATH or exporting a /run/opengl-driver/lib in a FHS environment would be valuable

SomeoneSerge avatar Jan 17 '24 14:01 SomeoneSerge