[Bug] Linux CLI python not found: should the env be activated before checking python is setup?

[gee-gendo]

The run-cmd script fails early when trying to train over ssh.

It looks like it can't validate the python executable under ${OT_PYTHON_CMD}.

After managing to bypass this specific error by hacking in the OT_PYTHON_CMD in the remote cmd call, I noticed this in the sh lib.

https://github.com/Nerogar/OneTrainer/blob/6276c512c2ff6ad50e74eb4617274ed2f44bdcee/lib.include.sh#L421-L425

Could it be that exit_if_no_runtime runnnig before before activate_chosen_env makes this script miss the python executable from ./venv (or ${OT_PYTHON_VENV}?

When running locally the easy fix is to set the python cmd env var... but remotely it might be worth activating the env before running the runtime check.

You might have ordered it this way for another reason though... is this a chicken and egg issue?

[dxqb]

changed subject line because not cloud-related, or were you using the cloud tab?

After managing to bypass this specific error by hacking in the OT_PYTHON_CMD in the remote cmd call,

I don't think that's hacking, but intended behaviour. python must be available, or in OT_PYTHON_CMD but others know more about this than me

[O-J1]

I didnt write the script nor do I help maintain linux install scripts but as Dxq said, if you dont have a runtime available (which is python) then you cant use python to activate the venv.

This means you either

• you dont have python installed
• python is installed in such a way that its not availble system wide
• the script is failing to detect your installed python.

As for the exact reason I wouldnt know (yet) as I dont maintain that script and havent taken the time to dig in.

Please attach your config.json, debug_report.log and then describe the exact steps you took to trigger this error.

[gee-gendo]

Here's my reading on the sh function calls stack in run-cmd.sh. Note that this is for the python venv use case

1. prepare_runtime_environment
2. exit_if_no_runtime
3. has_python

4. • can_exec "${OT_PYTHON_CMD}"

So I would have expected has_python to run on the venv python if it exists? Maybe not since the run_venv wouldn't be able to run...

The reason it's failing is because my ec2 instance python executable maps to python3 and not python.

How would you set OT_PYTHON_CMD to force it to use my env? Other than hardcoding in the cmd_env

[dxqb]

This is the current behaviour. You either have python in $PATH, or you set $OT_PYTHON_CMD. It does not automatically activate ./venv just because their might be a python in there.

I let others judge whether this is a bug / feature request / non-issue. Argument for non-issue: You already had to have a python in $PATH or in $OT_PYTHON_CMD when you run ./install.sh, so why don't you have now?

[O-J1]

I think there’s a misunderstanding, he is saying his instance doesn’t have python-is-python3 installed meaning our calls to python fail because modern python actually expects python3 (on Ubuntu).

I would regard this as a bug as Arcitec tried to make the Linux stuff as compatible as possible and aliasing python3 to python in the current session is simple, non permanent and doesn’t require sudo.

The question is whether this is appropriate. I really want @Arcitec to weigh in but he has been MIA

[Arcitec]

@O-J1 Hey, yeah I've had an extremely busy year (working for two AI companies). I'll add this ticket to my TODO tracker to look at it when I have free time.

[O-J1]

@O-J1 Hey, yeah I've had an extremely busy year (working for two AI companies). I'll add this ticket to my TODO tracker to look at it when I have free time.

Thanks dude, I am extremely hesitant to touch your scripts for fear of breaking it on some edge case

[O-J1]

Okay havent heard from Acitec, so I am going to have to take a crack at this myself.