WARN dora_daemon::pending: node `webcam` exited before initializing dora connection

[meua]

Describe the bug sh -c 'conda activate python_virtual_env' can not work normally, but all kinds of conda operations in ubuntu terminal are completely normal.

Customize the node according to the following configuration algorithm.

jarvis@jia:~/coding/dora_home/dora/binaries/cli/src$ dora-cli logs 6382d842-7471-44d7-8b76-2e5ae8b30d45 midas_op
     │ Logs from midas_op.
   1 │ sh: 1: conda: not found
jarvis@jia:~/coding/dora_home/dora/binaries/cli/src$ 

To Reproduce Steps to reproduce the behavior:

1. Dora start daemon: dora up
2. Start a new dataflow: RUST_LOG=info RUST_BACKTRACE=full cargo run start /path/to/your_rust_home/github.com/meua/dora-drives/graphs/tutorials/webcam_midas_frame_custom_node.yaml --attach --hot-reload

Expected behavior The python environments between different algorithms are isolated from each other.

Environments (please complete the following information):

• System info: ubuntu 22.04
• Dora version: v0.2.3-rc6

Additional context

  - id: midas_op
    custom:
      source: shell
      args: conda run -n midas-py310 python ../../operators/midas_op.py
#      args: source /home/jarvis/anaconda3/etc/profile.d/conda.sh && conda activate midas-py310 && python ../../operators/midas_op.py
#      args: conda env create -f ../../dependencies/MiDaS/environment.yaml && conda activate midas-py310 && python ../../operators/midas_op.py
#      args: conda activate midas-py310 && python ../../operators/midas_op.py
#      args: conda info -e && python ../../operators/midas_op.py
      outputs:
        - depth_frame
      inputs:
        image: webcam/image
···

[haixuanTao]

We're not looking to support conda and especially multi-env conda yet.

Were you able to solve your issue with conda run -n midas-py310 python ../../operators/midas_op.py ?

[meua]

We're not looking to support conda and especially multi-env conda yet.

Were you able to solve your issue with conda run -n midas-py310 python ../../operators/midas_op.py ?

No, so algorithm environment isolation is impossible.

[haixuanTao]

You can try to use:

  - id: midas_op
    custom:
      source: bash
      args: -c 'conda run -n midas-py310 python ../../operators/midas_op.py'
...
···

[meua]

You can try to use:

  - id: midas_op
    custom:
      source: bash
      args: -c 'conda run -n midas-py310 python ../../operators/midas_op.py'
...
···

You can run the test by yourself. Dora does not support your way of writing.

(dora3.7) jarvis@jia:~/coding/dora_home/dora/binaries/cli/src$ RUST_LOG=info RUST_BACKTRACE=full cargo run start /home/jarvis/coding/rust_home/github.com/meua/dora-drives/graphs/tutorials/webcam_midas_frame_custom_node.yaml --attach --hot-reload
   Compiling dora-operator-api-c v0.2.3-rc6 (/home/jarvis/coding/dora_home/dora/apis/c/operator)
   Compiling dora-cli v0.2.3-rc6 (/home/jarvis/coding/dora_home/dora/binaries/cli)
    Finished dev [unoptimized + debuginfo] target(s) in 4.40s
     Running `/home/jarvis/coding/dora_home/dora/target/debug/dora-cli start /home/jarvis/coding/rust_home/github.com/meua/dora-drives/graphs/tutorials/webcam_midas_frame_custom_node.yaml --attach --hot-reload`
Could not validate yaml: Dataflow could not be validated.: no node exists at `bash`: No such file or directory (os error 2)
(dora3.7) jarvis@jia:~/coding/dora_home/dora/binaries/cli/src$ 

Please refer to the following issues, thank you. https://github.com/dora-rs/dora-drives/issues/51

[meua]

Run the following command:

(dora3.7) jarvis@jia:~/coding/dora_home/dora/binaries/cli/src$ RUST_LOG=info RUST_BACKTRACE=full cargo run start /home/jarvis/coding/rust_home/github.com/meua/dora-drives/graphs/tutorials/webcam.yaml --attach --hot-reload
   Compiling dora-operator-api-c v0.2.3-rc6 (/home/jarvis/coding/dora_home/dora/apis/c/operator)
   Compiling dora-cli v0.2.3-rc6 (/home/jarvis/coding/dora_home/dora/binaries/cli)
    Finished dev [unoptimized + debuginfo] target(s) in 4.13s
     Running `/home/jarvis/coding/dora_home/dora/target/debug/dora-cli start /home/jarvis/coding/rust_home/github.com/meua/dora-drives/graphs/tutorials/webcam.yaml --attach --hot-reload`
c500ae11-72b4-4ce2-93b2-e833ca2f942a
  2023-05-30T01:48:46.997112Z  INFO dora_cli::attach: dataflow c500ae11-72b4-4ce2-93b2-e833ca2f942a stopped
    at binaries/cli/src/attach.rs:132

(dora3.7) jarvis@jia:~/coding/dora_home/dora/binaries/cli/src$ 

dora-daemon outputs the following logs:

  2023-05-30T01:48:46.620593Z  WARN dora_daemon::pending: node `webcam` exited before initializing dora connection
    at binaries/daemon/src/pending.rs:76

  2023-05-30T01:48:46.675948Z ERROR dora_daemon: 
    c500ae11-72b4-4ce2-93b2-e833ca2f942a/plot failed with exit code 1.

    Check logs using: dora logs c500ae11-72b4-4ce2-93b2-e833ca2f942a plot
                            
    at binaries/daemon/src/lib.rs:906

The webcam.yaml content of the file is as follows:

nodes:
  - id: webcam
    custom:
      source: shell
      args: /home/jarvis/anaconda3/condabin/conda run -n py3.7 python ../../operators/webcam_op.py
      inputs:
        tick: dora/timer/millis/50
      outputs:
        - image
  - id: plot
    operator:
      python: ../../operators/plot.py
      inputs:
        image: webcam/image

[haixuanTao]

Again, this is not a use case, we're looking to support, so I have limited time to spend on it.

You can try:

nodes:
  - id: webcam
    custom:
      source: conda
      args:  run -n py3.7 python ../../operators/webcam_op.py
      inputs:
        tick: dora/timer/millis/50
      outputs:
        - image

[meua]

Again, this is not a use case, we're looking to support, so I have limited time to spend on it.

You can try:

nodes:
  - id: webcam
    custom:
      source: conda
      args:  run -n py3.7 python ../../operators/webcam_op.py
      inputs:
        tick: dora/timer/millis/50
      outputs:
        - image

Sorry dora has limited support for this way.

[haixuanTao]

Can you show the logs please? as well as reporting errors as plain text instead of screenshot for better tracking in the future?

[haixuanTao]

On this one, I think one of the issue is that conda run sometimes get stuck for unknown reason.

I would recommend not using conda run