Mount the relevant directories so they are accessible

[GACHAPO]

Hi Graham,

I've struck an issue, where I need to use the exec node in node red to run a bash script on my pi and it can't because once in the docker container it can't get access to the external directories. Is there a way to build the docker containers to allow this please?

[theDiverDK]

In the docker-compose file you can add it as a volume:

volumes:
  - /home/pi/homeassistant/organizr:/config

This maps it so that seen from inside the docker container, if /config is accessed, it will use ' /home/pi/homeassistant/organizr' on the system running the docker container.

So you could have: Let's say you want the docker container to be able to access

/home/pi/FromOutside

on the path /FromInside seen from inside the docker- container.

Add this volume, to the docker-compose file: volumes: - /home/pi/FromOutside:/FromInside

Then you can place a 'bird.png' fil in '/home/pi/FromOutside'

And any program running inside the docker container can access it from '/FromInside/bird.png'

Hope it makes sense?

[GACHAPO]

Thanks for this;

so, in my case I want node red to access /home/pi/bin, what would I set my volume to be?

[GACHAPO]

Also, can this be done by creating a volume in portainer?

[theDiverDK]

What you want the volume to be, depends on how the NodeRed system should access is, do you know what path it is trying to access on the 'inside' ? If you can set NodeRed to access the file as '/outside/bin' then make the volume in docker-compose.yaml as:

• /home/pi/bin:/outside/bin

I havent tried through portainer, but you can also make a docker volume, but still being a docker newbie, i can't help you with that, sorry.

[gcgarner]

You can map a folder in portainer however the issue is the next time you run docker-compose down and up then you will lose the changes. The docker-compose.yml file is the master file that decides how everything is mapped and started.

In Portainer under stack you will see that portainer doesnt have control over the services because the compose file wasn't created inside it

[gcgarner]

Remember anything run in the exec node in node red will run within the container not on the host

edit: I haven't tried this myself but you could do something described here https://www.cyberciti.biz/faq/how-to-execute-a-remote-script-unix/

you would need to use a ssh key so that it doesn't require a password. However I'm not sure that the ssh key would be persistent and you may lose it on recreation

[877dev]

Couldn’t this be fixed by running node red in host mode? I’m doing this so that my Alexa nodes work properly and I’ve had no issues.

[gcgarner]

network mode refers to how the network interface maps to the container. Some of the side effects are that some picky devices work better. the exec will still only run inside the container.

If you run a reboot from inside the container then its the container that restarts not the host indicating that the script is note executing on the host

[theDiverDK]

Gcgarner i Can hear my docket knowledge is less than I thought

Any tips for a good book for a newbie/intermediate user?

[GACHAPO]

Remember anything run in the exec node in node red will run within the container not on the host

edit: I haven't tried this myself but you could do something described here https://www.cyberciti.biz/faq/how-to-execute-a-remote-script-unix/

you would need to use a ssh key so that it doesn't require a password. However I'm not sure that the ssh key would be persistent and you may lose it on recreation

Graham, I think this approach is probably the best, but having looked at your link I think SSH keys will need to be generated inside the node red docker and the public key then copied outside the container or to a seperate machine. My problem is that there's plenty online and an excellent youtube video https://youtu.be/vpk_1gldOAE to explain all of this from one machine to another, but none specific to docker. Can anybody point me in the right direction please?

[gcgarner]

@GACHAPO

ssh communication can be thought of as "terminal to terminal" communication. Therefore you would need to access the terminal of the container to perform the sshkegen etc.

I have a helper script for terminal access ./services/nodered/termnial.sh (when executed from the IOTstack directory. Or if you use Portainer you can access the terminal from there by clicking on the contatner and selecting terminal. What is happening in the background is that those scripts execute docker exec -it nodered /bin/bash - Docker Execute -InteractiveTerminal Container Command

Once in the terminal it should be able to follow the write-up

I do have one point of concern for the ssh method and that is the ssh key you generate is not in a persistent. If you had to run the command docker-compose down that key would disappear. I'm going to look into a method to map that generated key outside into a volume. I should be able to map /root/.ssh/ into the volumes and that 'should' make the key persistent.

I'll do a few tests and report back

[gcgarner]

Okay, I have some good news.

I can successfully execute a script on the Pi without entering a password. And it survives when the container is removed I will update the project version in the week to reflect these changes

If you want to do it now this is how i did it:

modify docker-compose.yml to show (and update your services/nodered/service.yml as well)

  nodered:
    container_name: nodered
    build: ./services/nodered/.
    restart: unless-stopped
    user: "0"
    privileged: true
    env_file: ./services/nodered/nodered.env
    ports:
      - 1880:1880
    volumes:
      - ./volumes/nodered/data:/data
      - ./volumes/nodered/ssh:/root/.ssh

run docker-compose up -d

I created a test.sh file in my IOTstack folder

#!/bin/bash
echo "hello"

make it executable with chmod +x test.sh

now enter the terminal for nodered with docker exec -it nodered /bin/bash

generate a ssh key with ssh-keygen follow the default prompts and don't enter a passphrase

copy that key to the Pi ssh-copy-id [email protected] replace with your address

select yes to add and enter the password for your Pi.

test with ssh [email protected] /home/pi/IOTstack/test.sh you should see "hello"

exit to leave the nodered terminal and get back to the pi

the flow:

I'm going to look into removing the user 0 option in the yml file. That may have an impact on the volume mapping no longer being /root/.shh

[GACHAPO]

copy that key to the Pi ssh-copy-id [email protected] replace with your address

select yes to add and enter the password for your Pi.

test with ssh [email protected] /home/pi/IOTstack/test.sh you should see "hello"

Thanks Graham,

I'm nearly there but not quite.

I generate the keys and have included -i and the name I gave the file holding the keys (in my case "keysnew") into the copy command. However, although it works it still asks for the password to enter my pi at 192.168.1.140.

Please advise?

Thanks

Glenn

[gcgarner]

Hello Glenn

I tried to use a named key and I was having the same issue. The ssh agent doesnt know about keynew after creation

how i created keynew: ssh-keygen -f ~/.ssh/keynew

how i added the key: ssh-copy-id -i ~/.ssh/keynew [email protected] (got the same error)

and testing it ssh [email protected] i had to add the password

however if you tell ssh which key to use then it will work ssh -i ~/.ssh/keynew [email protected] it works without the password

this is the link that helped me: https://askubuntu.com/questions/30788/does-ssh-key-need-to-be-named-id-rsa

note this extract regarding the config file for multiple keys:

Using multiple keys

It's not uncommon to use multiple keys. Instead of running ssh user@host -i /path/to/identity_file, you can use a configuration file, ~/.ssh/config.

Common settings are the IdentityFile (the keys) and port. The next configuration will check "id_dsa" and "bender" only when connecting with ssh youruser@yourhost:

Host yourhost
   IdentityFile ~/.ssh/id_dsa
   IdentityFile ~/.ssh/bender

If you omit Host yourhost, the settings will apply to all SSH connections. Other options can also be specified for this host match, like User youruser, Port 2222, etc. This would allow you to connect with the shorthand ssh yourhost instead of ssh -p2222 youruser@yourhost -i ~/.ssh/id_dsa -i ~/.ssh/bender

Side note: https://adamdehaven.com/blog/how-to-generate-an-ssh-key-and-add-your-public-key-to-the-server-for-authentication/

here they refer to using the ssh-add command which requires the ssh-agent to be running. I did the ssh-add and i could log in with ssh [email protected] without specifying the key path. I however dont know if this will survive a recreations where i am confident ssh -i ~/.ssh/keynew [email protected] will

[gcgarner]

I've been looking at ssh-keygen -f ~/.ssh/keynew ... and seeing that you need to specify the key to uses anyway my thoughts are it would be better to rather use /data/ssh in stead. /data is already mapped to the volume. This will require no addition changes to the current setup of nodered and no new volume to map

this would be the procedure:

from terminal:

docker exec -it nodered /bin/bash

create the ssh folder, no need to have a .ssh just call it ssh

mkdir -p /data/ssh

create key, this will require naming the output file

ssh-keygen -f /data/ssh/nodered put in any additional config you want key type strength copy the key to the Pi (no password)

ssh-copy-id -i /data/ssh/nodered [email protected] ( still get an error in the format but it does work )

exit the terminal with exit (or test first with ssh -i /data/ssh/nodered [email protected] exit)

then when you want to run the ssh in the exec node

ssh -i /data/ssh/nodered [email protected] "script path"

$ tree volumes/nodered/data/ssh
volumes/nodered/data/ssh
├── nodered
└── nodered.pub

0 directories, 2 files

updated the wiki https://github.com/gcgarner/IOTstack/wiki/Node-RED#running-the-exec-node-against-the-host-pi

[GACHAPO]

Hi Graham,

I didn't realise that it was because I'd named the file - ahhh!

I have tried your latest step through and it works great, thank you. Having tried it on Node Red, I get an debug output of "code: 0" though. I'll try another script and see what happens.

Big ask - can it be done for home assistant as well?

Thanks again

Glenn

[gcgarner]

I checked my script and if i add exit 0 at the end then i don't get the red error

I'll update that on the wiki as well

Regarding doing this on HomeAssistant.. it should be.

The long answer: I don't don't use Home Assistant and i don't have it installed now, however what we just went though is a fundamental part of Linux so i don't see why it wouldn't work. If in HA you have access to a shell then this is theoretically possible. I'll do some reading up and see what i can find

[GACHAPO]

Thanks

Still got a code:0, but my other script to run ffmpeg works beautifully.

I think you may struggle with Homeassistant, because your menu.sh loads Hassio which is regarded as an "appliance", sealed to linux tinkering. The route may have to be Hassio to Node red as described. To be honest Hassio/Node Red is the perfect marriage with Node Red being a lot more intuitive and visual for automations