lux
lux copied to clipboard
hawk
Large send commands (!) not returning prompt in lux
I've been poking at this for a little while and I'm at my wits' end. I would appreciate any direction/pointers. I'll include the relevant tidbits from files.
I have the following setup for a test suite:
- package.luxinc [file w/ package-specific macros]
- macro-27.luxinc [file w/ test-case specific macros, INCLUDE package.luxinc]
- function-27.lux [main test file, INCLUDE macro-27.luxinc]
function-27:
[shell function-27-REST]
[invoke setup]
[invoke function27_rest $var1 $var2 $var3 $var4 $var5 $SOME_URL $SOME_FILE]
!bash compare.sh ../expectedOutput $SOME_FILE
??True
[invoke function27_rest $var1 $var2 $var3 $var4 $var5 $OTHER_URL $OTHER_FILE]
!grep '201 Created' $OTHER_FILE
??HTTP/1.1 201 Created
macro-27.luxinc:
[macro function27_rest var1 var2 var3 var4 var5 URL FILE]
"""!echo '
<GIANT XML BODY w/ access to above variables>
</GIANT XML BODY>
' | curl -is -X POST -d @- -u admin:admin -H "Content-type: application/vnd.yang.data+xml" $URL >> $FILE
"""
?SH-PROMPT:
!echo ==$?==
?==0==
[endmacro]
package.luxinc:
[macro setup]
[progress 'Setup']
!mkdir -p lux_logs/latest_run/output_files
?SH-PROMPT:
!echo ==$?==
?==0==
[endmacro]
So, the main test script function-27 first calls setup from the package.luxinc, which just creates a directory. This works fine. The place where I encounter trouble is when the main test script calls function27rest from the macro-27.luxinc file, which echoes a large XML body to a cURL command then pipes the resulting return body into a file.
For whatever reason, when I expect the SH-PROMPT, it never returns. The part that really confuses me is that I have the same setup for multiple tests (i.e., function-28, function-29, etc...) and they all work except this one. The biggest difference I can think of between this one and the rest of the tests is that this one has the largest XML body. But I'm not sure exactly what problem that causes.
The following comments on this issue contain the output from the .stdin and .stdout of the test suite. Any help would be appreciated!!
function27 stdin:
export PS1=SH-PROMPT:
mkdir -p lux_logs/latest_run/output_files
echo '
<GIANT XML BODY>
' | curl -is -X POST -d @- -u admin:admin -H "Content-type: application/vnd.yang.data+xml" [REDACTED_URL] >> lux_logs/latest_run/output_files/FUNC-027-DRYRUN-CREATE.tmp
function27 stdout:
[?1034hsh-3.2$ export PS1=SH-PROMPT:
SH-PROMPT:mkdir -p lux_logs/latest_run/output_files
SH-PROMPT:
function28 stdin:
export PS1=SH-PROMPT:
mkdir -p lux_logs/latest_run/output_files
echo '
<GIANT XML BODY>
' | curl -is -X POST -d @- -u admin:admin -H "Content-type: application/vnd.yang.data+xml" [REDACTED_URL] >> lux_logs/latest_run/output_files/FUNC-028-DRYRUN-CREATE.tmp
echo ==$?==
bash compare.sh ../expected/native/FUNC-028 lux_logs/latest_run/output_files/FUNC-028-DRYRUN-CREATE.tmp
echo '
<GIANT XML BODY>
' | curl -is -X POST -d @- -u admin:admin -H "Content-type: application/vnd.yang.data+xml" [REDACTED_URL] >> lux_logs/latest_run/output_files/FUNC-028-COMMIT-CREATE.tmp
echo ==$?==
function28 stdout:
[?1034hsh-3.2$ export PS1=SH-PROMPT:
SH-PROMPT:mkdir -p lux_logs/latest_run/output_files
SH-PROMPT: echo '
> <GIANT XML BODY>
> ' | curl -is -X POST -d @- -u admin:admin -H "Content-type: application/vnd.ya
ng.data+xml" [REDACTED_URL] >> lux_logs/l
atest_run/output_files/FUNC-028-DRYRUN-CREATE.tmp
SH-PROMPT:echo ==$?==
==0==
SH-PROMPT:bash compare.sh ../expected/native/FUNC-028 lux_logs/latest_run/output
_files/FUNC-028-DRYRUN-CREATE.tmp
#### File1: ../expected/native/FUNC-028
#### File2: lux_logs/latest_run/output_files/FUNC-028-DRYRUN-CREATE.tmp
Partial True: ../expected/native/FUNC-028 is a subset of lux_logs/latest_run/output_files/FUNC-028-DRYRUN-CREATE.tmp
SH-PROMPT: echo '
> <GIANT XML BODY>
> ' | curl -is -X POST -d @- -u admin:admin -H "Content-type: application/vnd.ya
ng.data+xml" [REDACTED_URL] >> lux_logs/latest_run/outp
ut_files/FUNC-028-COMMIT-CREATE.tmp
SH-PROMPT:echo ==$?==
==0==
grep '201 Created' lux_logs/latest_run/output_files/FUNC-028-COMMIT-CREATE.tmp
SH-PROMPT:grep '201 Created' lux_logs/latest_run/output_files/FUNC-028-COMMIT-CR
EATE.tmp
HTTP/1.1 201 Created
As you can clearly see, function28 has the exact same makeup as function27, but is working while function27 is not. Once again, the only difference I can think of is that function27 has a much larger XML body.
Thanks again in advance for your time/help.
Does it work sometimes? If not, perhaps it may be some issue with how long string echo can manage (on your host)?
If it works sometimes I would compare the (textual) event.log file from a successful run with a failing run. More specifically I would try to figure out what happened just before the "echo giant | curl" command. Such as has the previous command REALLY finished before the giant command gets executed? And what differs in the stdout stream when the giant command gets executed??
Yes, it works sometimes, so that's how I came to the conclusion it was some kind of weird race condition, although I could be wrong.
Unfortunately, looking at the .event.log files doesn't help much.... between the runs that worked and the runs that didn't work, everything is the same up until one of them fails...
Successful run:
func-027-rest(19): progress "'Dry-Run'"
func-027-rest(20): invoke_function27_rest "var1=VAR1 var2=VAR2 var3=VAR3 URL=[REDACTED_URL] FILE=lux_logs/latest_run/output_files/FUNC-027-DRYRUN-CREATE.tmp "
file_enter 20 41 68 "~/test/rest/FUNC-027.lux"
func-027-rest(42): fail pattern "(\\\\[error\\\\])|(.*database is locked)|(Aborted)|([F|f]alse)"
func-027-rest(63): send " echo ' <LARGE XML BODY> ' | curl -is -X POST -d @- -u admin:admin -H "Content-type: application/vnd.yang.data+xml" [REDACTED_URL] >> lux_logs/latest_run/output_files/FUNC-027-DRYRUN-CREATE.tmp\n"
func-027-rest(64): expect "SH-PROMPT:"
func-027-rest(64): timer start (10 seconds * 1.000)
func-027-rest(64): recv " echo ' <LARGE XML BODY> ... "
func-027-rest(64): recv "SH-PROMPT:"
func-027-rest(64): timer cancel (0 seconds)
Unsuccessful run:
func-027-rest(19): progress "'Dry-Run'"
func-027-rest(20): invoke_function27_rest "var1=VAR1 var2=VAR2 var3=VAR3 URL=[REDACTED_URL] FILE=lux_logs/latest_run/output_files/FUNC-027-DRYRUN-CREATE.tmp "
file_enter 20 41 68 "~/test/rest/FUNC-027.lux"
func-027-rest(42): fail pattern "(\\\\[error\\\\])|(.*database is locked)|(Aborted)|([F|f]alse)"
func-027-rest(63): send "echo ' <LARGE XML BODY> ' | curl -is -X POST -d @- -u admin:admin -H "Content-type: application/vnd.yang.data+xml" [REDACTED_URL] >> lux_logs/latest_run/output_files/FUNC-029-DRYRUN-CREATE.tmp\n"
func-027-rest(64): expect "SH-PROMPT:"
func-027-rest(64): timer start (10 seconds * 1.000)
func-027-rest(64): recv "timeout"
func-027-rest(64): timer fail (10 seconds)
func-027-rest(64): skip ""
func-027-rest(64): stop fail
func-027-rest(64): skip ""
Looking at the above output and comparing it, it looks like the failing test doesn't see the 'echo' output. This probably means the echo command doesn't complete, as it's expecting SH-PROMPT afterwards and never gets it.
I'm a little stumped about what could possibly block an echo command, however.
Check the previous "send" item in the event log and verify that it has completed. With this I mean that you have matched on something reliable, such as the SH-PROMPT produced AFTER the command has finished. Do ensure it was not matched on some earlier prompt.
Yes, that's correct. It looks like the last successful match was on SH-PROMPT in a previous macro, so it doesn't look like we have an expect/output mismatch. I'll paste in the event.log items.
func-027-rest(11): send "mkdir -p lux_logs/latest_run/output_files\n"
func-027-rest(11): recv "mkdir"
func-027-rest(12): expect "SH-PROMPT:"
func-027-rest(12): timer start (10 seconds * 1.000)
func-027-rest(12): recv " -p lux_logs/latest_run/output_files\r\n"
func-027-rest(12): recv "SH-PROMPT:"
func-027-rest(12): timer cancel (0 seconds)
func-027-rest(12): skip "mkdir -p lux_logs/latest_run/output_files\r\n"
func-027-rest(12): match "SH-PROMPT:"
file_exit 15 68 73 "/Users/mvxt_dd/Documents/Data_Ductus/sprintvpn/packages/sprint-l2vpn/test/rest/FUNC-027.lux"
func-027-rest(16): progress "'Function-027"
func-027-rest(19): progress "'Dry-Run'"
func-027-rest(20): invoke_function27_rest "REDACTED VARIABLES"
file_enter 20 55 97 "/Users/mvxt_dd/Documents/Data_Ductus/sprintvpn/packages/sprint-l2vpn/test/rest/FUNC-029.lux"
func-027-rest(42): fail pattern "(\\\\[error\\\\])|(.*database is locked)|(Aborted)|([F|f]alse)"
func-027-rest(63): send "echo '<LARGE XML BODY>\n' | curl -is -X POST -d @- -u admin:admin -H "Content-type: application/vnd.yang.data+xml" [REDACTED_URL] >> lux_logs/latest_run/output_files/FUNC-027-DRYRUN-CREATE.tmp\n"
func-027-rest(64): expect "SH-PROMPT:"
func-027-rest(64): timer start (10 seconds * 1.000)
func-027-rest(64): recv "timeout"
func-027-rest(64): timer fail (10 seconds)
func-027-rest(64): skip ""
func-027-rest(64): stop fail
func-027-rest(64): skip ""
Hey Hakan, how does invoking a macro work? Does it just spawn a separate shell to run the macro or does it simply include the commands and run it within the sequence? test:
!some command
?some expect
[invoke some macro]
[macro some macro]
!some other command
?some other expect
[endmacro]
Are the above two separate files essentially the equivalent of:
!some command
?some expect
!some other command
?some other expect
? Or does it actually spawn a separate shell or some separate thread to run the macro?
They are essentially equivalent. Shells are started on demand with [shell NAME]. You can this in the event.log. It is very transparent.
Btw, avoid to build in assumptions about where the script will be executed or where its logs will be located. Use "mkdir -p $LUX_EXTRA_LOGS/output_files" instead of using "mkdir -p lux_logs/latest_run/output_files".
Have you tried to copy the "echo giant |curl" command from the stdin log and paste it repeatedly in a (Bourne) shell? It may cause the same problems as you encounter in the lux script.
Yes, I ran the command manually on the terminal and it works as expected, repeatedly.
I managed to narrow down the problem to the specific line with the echo function. It looks like macro was not the issue, so I've renamed this thread to reflect that.
Here is what I did:
- Copied all of the contents inside the macro to run directly inside the test file
- Removed the extra
| curlcommand following the multi-line echo. - Then I ran the debugger to step through line-by-line
Results:
- Even without piping into
curland then redirecting output to a file, it looks like the multi-lineechocommand doesn't return the prompt. What could stop anechocommand from returning? - It freezes both when I run the test as well as when I step through each line manually in the Lux debugger
- As stated above, I can paste the multi-line XML body directly into the Terminal and
echoit without problems. Always consistently returns within 0.35 seconds, so I don't think it's theechocommand itself.
Conclusions:
- I can only conclude at this point that maybe it's the way
echoplays with the multi-line command in Lux. I'll paste in my test script so you can see its contents.
[shell func-027-rest]
[invoke setup_rest]
[progress 'FUNC-027 - TEST']
[progress 'Dry-Run']
"""!echo '
<GIANT XML BODY>
...
...
'
"""
?SH-PROMPT:
!echo ==$?==
?==0==
[progress 'Commit']
"""!echo '
<GIANT XML BODY>
...
...
'
"""
?SH-PROMPT:
!echo ==$?==
?==0==
Thanks again for your time and help. Much appreciated
UPDATE 1: I experimented with the XML body I was sending and shortened it to test, and it looks like the shorter body works (consistently, with repeated running), which explains why the rest of my tests work (shorter XML bodies).
UPDATE 2: Replaced echo with printf and it looks like it still encounters the same issues. Sometimes works, sometimes doesn't... :/
Based on UPDATE 1, is there some kind of buffer limit or buffer overflow or some kind of array or variable size limit being placed on multi-line regex and other items for Lux?
I do not think the problem really is lux. The argument string given to 'echo' is seems quite extreme. So I would avoid it as it may indicate that you have run into some system limit related to the shell (/bin/sh), the terminal or what not. Try using 'cat' or something else which avoids the very long argument string. Something like this may work:
"""!
cat > xml.tmp << END
line1
.
.
line10000
END
[invoke eval "cat xml.tmp | curl"]
"""
[cleanup]
[invoke eval "rm -f xml.tmp"]
[macro eval cmd] !$cmd ?SH-PROMPT: !echo ==$$?== ?==0== ?SH-PROMPT: [endmacro]
[macro eval_any cmd] !$cmd ?SH-PROMPT: !echo ==$$?== ?==[0-9]+== ?SH-PROMPT: [endmacro]
Gave your suggestions a try and it runs into the same problem. The multi-line command with cat > xml.tmp << HEREDOC <CONTENT> HEREDOC runs into the same issue: I ran it a multitude of times, and it succeeded once, and failed the other times.
Every single multi-line command tested (i.e., echo, printf, cat w/ heredoc, even directly passing the entire body as an argument to curl) all worked fine when run directly on the shell, but all commands w/ the multi-line XML hang inside Lux.
I did notice that when I shorten the XML body, it then does work in Lux, and I'm not convinced that that is a system limitation, as you say, because once again, it works when run directly on the terminal.
Thanks for your help thus far, I'll keep digging through and experimenting and I'll report back whether I find anything.
Are you using the same shell as lux when you run it outside lux? See the event.log about how lux starts its shell. (It is probably "/bin/sh -i".)
/Håkan
On Jan 12, 2017 00:10, "Michael Thanh" [email protected] wrote:
Gave your suggestions a try and it runs into the same problem. The multi-line command with cat > xml.tmp << HEREDOC HEREDOC runs into the same issue: I ran it a multitude of times, and it succeeded once, and failed the other times.
Every single multi-line command tested (i.e., echo, printf, cat w/ heredoc, even directly passing the entire body as an argument to curl) all worked fine when run directly on the shell, but the test hangs inside Lux.
I did notice that when I shorten the XML body, it then does work in Lux, and I'm not convinced that that is a system limitation, as you say, because once again, it works when run directly on the terminal.
Thanks for your help thus far, I'll keep digging through and experimenting and I'll report back whether I find anything.
— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/hawk/lux/issues/17#issuecomment-272025530, or mute the thread https://github.com/notifications/unsubscribe-auth/AAJpsO5PfUur6DIRBpsku8pX4I7suuAFks5rRWF6gaJpZM4Lf6yC .
Hmmm. No, my terminal runs zsh. So you think it might be a limitation of sh vs. zsh?
I'll try running the lux tests in a different shell tomorrow.
Maybe. It is one possible difference in the run environment. We have had issues with /bin/sh compliance on some platforms.
Another issue may be the terminal settings. If you cannot reproduce the problem with plain "/bin/sh -i", you may try with /bin/sh in conjunction with lux/priv/bin/runpty like lux does. See the event.log for the shell start details with runpty.
/Håkan
On Thu, Jan 12, 2017 at 5:35 AM, Michael Thanh [email protected] wrote:
Hmmm. No, my terminal runs zsh. So you think it might be a limitation of sh vs. zsh?
I'll try running the lux tests in a different shell tomorrow.
— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/hawk/lux/issues/17#issuecomment-272074030, or mute the thread https://github.com/notifications/unsubscribe-auth/AAJpsJ8H8oy1TDJuw_3vZWcFKCsdnqBAks5rRa2dgaJpZM4Lf6yC .
Hello Håkan,
I'm working with Michael here. Seems like there is a lux limitation on how many characters can be in the send command. Please see my lux script it seems to fail when the number of characters are around 2048 on a send line. I've gotten the same results using printf and echo instead of cat as shown.
/Lloyd
[shell test]
Passes
!cat '0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000' ?SH-PROMPT
Passes
!cat '00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000' ?SH-PROMPT
Will timeout
!cat '00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000' ?SH-PROMPT
Hey @hawk,
I tried your suggestions to reproduce the error on the shell and was unable to. I first logged into the standard Bourne shell using the command as follows:
/usr/local/lux/lib/lux-1.15.1/priv/bin/runpty /bin/sh -i, which is what I pulled out of the Lux .event.log file.
Afterwards, I ran the same large command from inside the Lux script manually in the Bourne shell, and it succeeds without any problems repeatedly.
echo '
<LARGE XML BODY>
...
...
...
</LARGE XML BODY>
' | curl -is -X POST -d @- -u admin:admin -H "Content-type: application/vnd.yang.data+xml" -H "Expect:" "[REDACTED_URL]"
and the output I get is as expected:
HTTP/1.1 201 Created
Server:
Location: [REDACTED_URL]
Date: Thu, 12 Jan 2017 18:40:01 GMT
Last-Modified: Thu, 12 Jan 2017 16:59:20 GMT
Cache-Control: private, no-cache, must-revalidate, proxy-revalidate
Etag: 1484-240360-717269
Content-Length: 1047
Content-Type: text/xml
Vary: Accept-Encoding
Pragma: no-cache
<XML RETURN BODY>
I also tried a few other shells like zsh and bash, and those are also working as expected. At this point my tests and experiments don't reinforce that the issue is with the system or with the specific shell environment the command is being run in.
I can confirm that it is a bug in Lux. It was possible to reproduce with:
/usr/local/lux/lib/lux-1.15.1/priv/bin/runpty /bin/sh -i
echo <HERE I PASTED A BIG CHUNK DATA>
then the shell hanged.
Unfortunately I don't know a good temporary workaround. The bug needs to be fixed.
Hey @hawk just curious when we might see the fix come through? Being able to send these sends with > 2048 bytes is a bit critical for our project.
If you are brave you may test the latest on the non-blocking-pty branch. It is not fully tested though.
/Håkan
On Mon, Jan 16, 2017 at 5:09 PM, Lloyd Rochester [email protected] wrote:
Hey @hawk https://github.com/hawk just curious when we might see the fix come through? Being able to send these sends with > 2048 bytes is a bit critical for our project.
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/hawk/lux/issues/17#issuecomment-272902390, or mute the thread https://github.com/notifications/unsubscribe-auth/AAJpsGWOOMXi_MoKFbR8iQAlP_o-olshks5rS5ZPgaJpZM4Lf6yC .
Working on testing it. I looked through the code and the 4*BUF was interesting. The runpty is quite complex and cool as well, was looking at how you did that. Currently get 2 issues always when installing on OS X in that my erlc path is wrong so I edit include.mk, but then get an issue in make saying
lux_html_parse.erl:14: can't find include lib "xmerl/include/xmerl.hrl"
Once I fix that I should be able to get back to you.
@lloydroc
When I fix my include.mk files to reflect the correct erl and erlc locations, my error for xmerl goes away. Maybe try checking that again?
@hawk
Pulled the non-blocking-pty branch and tested it with the tests I was previously having problems with. In the stdin.log, I'm seeing the expected SEND command. However, in the stdout.log, I'm seeing an oddly mangled version of what I sent, and it's also multiple times what the original buffer had. Example:
ORIGINAL COMMAND:
!echo '
<LARGE XML BODY>
'
STDOUT OUTPUT:
SH-PROMPT: echo '
<LARGE XML BODY w/ LAST 2 LINES MISSING>
...
<DUPLICATE LARGE XML BODY w/ FIRST 2 LINES MISSING>
...
<DUPLICATE LARGE XML BODY w/ FIRST 6 LINES MISSING>
...etc
If I had to make a guess, it looks like the 4*BUF didn't actually expand the size of the buffer to 4 times, but instead duplicated the limited-size buffer 4 times, resulting in the output you see above.
I tested also with a simple lux script where I echo a large body of 0s like so:
[shell test]
!echo '000000 ... ' # ... replaces many more 0s for brevity
?SH-PROMPT:
and I am getting an error as follows:
result : FAIL at 3
expected
SH-PROMPT:
actual error
The command must be executed in context of a shell
successful : 0
failed : 1
temp.lux:3 - The command must be executed in context of a shell
summary : FAIL
It also looks like the SH-PROMPT: is still not returning
Can you provide a minimal test case which reproduces the (new) problem?
The error message saying something about shell context, probably implies that the shell has died prematurely. Take a look at the event log. Or run lux with the -v flag.
/Håkan
On Jan 17, 2017 6:58 PM, "Michael Thanh" [email protected] wrote:
@lloydroc https://github.com/lloydroc
When I fix my include.mk files to reflect the correct erl and erlc locations, my error for xmerl goes away. Maybe try checking that again?
Pulled the non-blocking-pty branch and tested it with the tests I was previously having problems with. In the stdin.log, I'm seeing the expected SEND command. However, in the stdout.log, I'm seeing an oddly mangled version of what I sent, and it's also multiple times what the original buffer had. Example:
SH-PROMPT: echo ' <LARGE XML BODY w/ LAST 2 LINES MISSING> ... <DUPLICATE LARGE XML BODY w/ FIRST 2 LINES MISSING> ... <DUPLICATE LARGE XML BODY w/ FIRST 6 LINES MISSING>
...etc.
I tested also with a simple lux script where I echo a large body of 0s like so:
[shell test] !echo '000000 .... ' ?SH-PROMPT:
and I am getting an error as follows:
result : FAIL at 3 expected SH-PROMPT: actual error The command must be executed in context of a shell
successful : 0 failed : 1 temp.lux:3 - The command must be executed in context of a shell summary : FAIL
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/hawk/lux/issues/17#issuecomment-273247067, or mute the thread https://github.com/notifications/unsubscribe-auth/AAJpsPPpLfWN3LkpirG0vwzcr6GSs2lBks5rTQFCgaJpZM4Lf6yC .
1) Regarding the early-terminated shell
Yes, you're right. I looked through the event log and it looks like the shell is terminated.
test(3): recv "0"
test(3): recv "Child terminated by signal 1\n"
test(3): recv "shell_exit"
For reference, in the test, I echo a string of 6222 zeroes. It looks like the recv is a bit odd though...
test(3): expect "SH-PROMPT:"
test(3): timer started (10 seconds * 1.000 multiplier)
test(3): recv "echo '"
test(3): recv "00000000000000000000"
test(3): recv "000000000000000000"
test(3): recv "00000000000"
test(3): recv "0000000000000"
test(3): recv "00 \r0000000000000000"
test(3): recv "000000000000000000"
test(3): recv "0000000000000"
test(3): recv "00000000000"
test(3): recv "00000000000"
test(3): recv "000000000"
test(3): recv "00 \r000000000"
test(3): recv "000000000000000"
test(3): recv "000000000000"
test(3): recv "00000000"
test(3): recv "00000000000"
test(3): recv "000000"
This continues for about a thousand more lines because the recv eventually dwindles down to receiving a single 0 at a time.... line 974: test(3): recv "0"
The total number of 0s it receives is only around 3000 or so however before the shell is terminated.
2) Regarding Simple Test Case to Replicate New Problem
[shell test]
"""!echo '00000000000000000000000000000000000000000000000000000000
000000000000000000000000000000000000000000000000000000000000000000
000000000000000000000000000000000000000000000000000000000000000000
000000000000000000000000000000000000000000000000000000000000000000
000000000000000000000000000000000000000000000000000000000000000000
000000000000000000000000000000000000000000000000000000000000000000
000000000000000000000000000000000000000000000000000000000000000000
000000000000000000000000000000000000000000000000000000000000000000
000000000000000000000000000000000000000000000000000000000000000000
000000000000000000000000000000000000000000000000000000000000000000
000000000000000000000000000000000000000000000000000000000000000000
000000000000000000000000000000000000000000000000000000000000000000
000000000000000000000000000000000000000000000000000000000000000000
000000000000000000000000000000000000000000000000000000000000000000
000000000000000000000000000000000000000000000000000000000000000000
000000000000000000000000000000000000000000000000000000000000000000
000000000000000000000000000000000000000000000000000000000000000000
000000000000000000000000000000000000000000000000000000000000000000
000000000000000000000000000000000000000000000000000000000000000000
000000000000000000000000000000000000000000000000000000000000000000
000000000000000000000000000000000000000000000000000000000000000000
000000000000000000000000000000000000000000000000000000000000000000
0000000000000000000000000000000000000000
'
"""
?SH-PROMPT:
That's the whole test. It seems to die because of the new line characters - if I remove the new lines and have it pasted as a single contiguous line, then it works, but I somehow feel like that defeats the purpose of the multi-line send...
This test works / passes.
[shell test]
"""!echo '000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
'
"""
?SH-PROMPT:
@mvxt I modified my include.mk and it has this.
# Erlang
ERL="/opt/local/bin/erl"
ERLC="/opt/local/bin/erlc"
Erlang I installed from brew.
From shell I have:
╭─[email protected] ~/Documents/workspace/sprintvpn ‹remotes/origin/Refactor/REST*›
╰─➤ echo $ERL_LIBS 1 ↵
/usr/local/lux/lib
╭─[email protected] ~/Documents/workspace/sprintvpn ‹remotes/origin/Refactor/REST*›
╰─➤ ls /usr/local/lux/lib
compiler-5.0.3.ez et-1.5.ez lux-1.12.2 lux-1.15.1 sasl-2.4.1.ez xmerl-1.3.7.ez
erts-6.3.ez kernel-3.1.ez lux-1.12.2.ez lux-1.15.1.ez stdlib-2.3.ez
╭─[email protected] ~/Documents/workspace/sprintvpn ‹remotes/origin/Refactor/REST*›
╰─➤
Not sure why it can't find xmerl from /usr/local/lux/lib tried other things but not working.