cmc-csci143 icon indicating copy to clipboard operation
cmc-csci143 copied to clipboard

topic_01_mapreduce lab

Open abizermamnoon opened this issue 1 year ago • 1 comments

Hi, I was doing the shell scripting and parallel programming tutorial. Even though I do not have nohup, my script.sh is running and output2 is updating when I close one window.

lambda-server:~/bigdata $ ps -ef | grep Abizer
Abizer.+  4282     1  0 19:43 ?        00:00:00 /lib/systemd/systemd --user
Abizer.+  4283  4282  0 19:43 ?        00:00:00 (sd-pam)
root      7057  1767  0 19:51 ?        00:00:00 sshd: Abizer.Mamnoon.25 [priv]
Abizer.+  7133  7057  0 19:51 ?        00:00:00 sshd: Abizer.Mamnoon.25@pts/1
Abizer.+  7134  7133  0 19:51 pts/1    00:00:00 -bash
root      8021  1767  0 19:53 ?        00:00:00 sshd: Abizer.Mamnoon.25 [priv]
Abizer.+  8086  8021  0 19:53 ?        00:00:00 sshd: Abizer.Mamnoon.25@pts/7
Abizer.+  8087  8086  0 19:53 pts/7    00:00:00 -bash
Abizer.+ 10343  7134  0 20:01 pts/1    00:00:00 ps -ef
Abizer.+ 10344  7134  0 20:01 pts/1    00:00:00 grep Abizer
lambda-server:~/bigdata $ rm output2
lambda-server:~/bigdata $ ls
books-notw-full.json  main.log                                   script.sh
lab-goodreads         mistral-7b-instruct-v0.2.Q5_K_M.llamafile
llama.log             reviews-notw-full.json
lambda-server:~/bigdata $ ./script.sh > output2
^Z
[1]+  Stopped                 ./script.sh > output2
lambda-server:~/bigdata $ bg
[1]+ ./script.sh > output2 &
lambda-server:~/bigdata $ wc -l output2
32 output2
lambda-server:~/bigdata $ wc -l output2
33 output2
lambda-server:~/bigdata $ wc -l output2
35 output2

I then exited this session and opened a new session:

lambda-server:~/bigdata $ wc -l output2
73 output2
lambda-server:~/bigdata $ wc -l output2
74 output2
lambda-server:~/bigdata $ wc -l output2
76 output2
lambda-server:~/bigdata $ ps -ef | grep Abizer
Abizer.+  4282     1  0 19:43 ?        00:00:00 /lib/systemd/systemd --user
Abizer.+  4283  4282  0 19:43 ?        00:00:00 (sd-pam)
root      8021  1767  0 19:53 ?        00:00:00 sshd: Abizer.Mamnoon.25 [priv]
Abizer.+  8086  8021  0 19:53 ?        00:00:00 sshd: Abizer.Mamnoon.25@pts/7
Abizer.+  8087  8086  0 19:53 pts/7    00:00:00 -bash
Abizer.+ 13843     1  0 20:10 ?        00:00:00 /bin/sh ./script.sh
Abizer.+ 14490 13843  0 20:12 ?        00:00:00 sleep 1
Abizer.+ 14493  8087  2 20:12 pts/7    00:00:00 ps -ef
Abizer.+ 14494  8087  0 20:12 pts/7    00:00:00 grep Abizer

abizermamnoon avatar Jan 23 '24 04:01 abizermamnoon

In general, I don't recommend working through the contents of the future folder. Much of that content is out of date.

In particular, the file you were looking at is an old version that is missing the following note, which I believe answers your question.

Be default on most Linux machines, the ssh program sends the HUP signal automatically when it disconnects. This is polite to do: Since we know we won't be using those processes any more, we should kill them and free their resources. But on some machines (especially Macs), the ssh program is configured to not send the HUP signal automatically. Therefore, the programs will not be immediately terminated. The lambda server waits about ten minutes before sending the HUP signal and terminating the processes itself. (This delay is necessary to handle temporary disconnects due to things like bad wifi.) If you are using a ssh program configured in this way, then you won't immediately see your programs be killed by the HUP signal when you logout. These programs will eventually be killed by the lambda server, but you don't need to wait for the server to kill them before progressing with the lab.

mikeizbicki avatar Jan 23 '24 06:01 mikeizbicki