Commands other than 'tail' do not seem to work

[BenBaril83]

I'm attempting to run an AWK against the tail output, and no matter what I do, I never see any output. Happy to provide more details if necessary.

I've taken the two 'commands' from the log and run them like commandA | commandB and they work perfectly.

[gvalkov]

Hi. You most likely have to add a flush() command to your awk script: {print $1; fflush()} (sorry, I know it's not intuitive at all).

[BenBaril83]

I wish that worked, but no dice. Here are things I've tried:

{print $0; fflush()} '{print $0; fflush()}'

My actual script: 'BEGIN {OFS = "*"} {printf("Alert|%s|%s|%s|%8d|%8d|%8d|%s|%s|%s\n",$1,$2,$3,$4,$5,$4-$5,$10,$9,substr($22,1,40)); fflush() }'

[BenBaril83]

Log output:

2018/08/30 11:05:04 Stopping pid 28
2018/08/30 11:05:04 Stopping pid 29
2018/08/30 11:05:04 Running command: [tail -n 50 -F mylog.log]
2018/08/30 11:05:04 Running command: [awk --sandbox 'BEGIN {FS = "*"} {printf("Alert|%s|%s|%s|%8d|%8d|%8d|%s|%s|%s\n",$1,$2,$3,$4,$5,$4-$5,$10,$9,substr($22,1,40)); fflush() }']

If I take the last one and pipe it, it works:

tail -n 50 -F mylog.log | awk --sandbox 'BEGIN {FS = "*"} {printf("Alert|%s|%s|%s|%8d|%8d|%8d|%s|%s|%s\n",$1,$2,$3,$4,$5,$4-$5,$10,$9,substr($22,1,40)); fflush()}'
Alert|2018-08-30 11:01:31,230 GMT|8|PEGA0005|47939242|     250|47938992|System|B2W382WFGS1C6VI30M5XBREHPBFWPS7PY|Code-.SystemPulse
Alert|2018-08-30 11:01:31,308 GMT|8|PEGA0005|    4280|     250|    4030|System|B43JVRY925ELLU6CT59DSI6WU8TG2U75O|System-Queue-ExecutionRequest-Service-De
Alert|2018-08-30 11:01:31,241 GMT|8|PEGA0005|47939237|     250|47938987|none|B191O6QHUHWGAG5CKNGLGOTTJCC0H5I4D|@baseclass.pxCallDataTransform
Alert|2018-08-30 11:01:31,562 GMT|8|PEGA0005|    4937|     250|    4687|none|BYCNP0HPMNGFUCUNKB23XHC4TU3VIEE1J|NA
Alert|2018-08-30 11:01:31,562 GMT|8|PEGA0005|    4931|     250|    4681|System|BSFNCO7IUKJTZ64MSVN4IVJONOZ5TLXOZ|Work-ProcessEmail.pyProcessEmailRequest
Alert|2018-08-30 11:01:32,700 GMT|8|PEGA0005|     311|     250|      61|System|BPBSIAFM5G5TCOYEL29OIPTX4IXOKEP4T|Log-System-State-Cluster.pzPersistCluste
Alert|2018-08-30 11:02:05,333 GMT|8|PEGA0005|    1778|     250|    1528|System|BRV2OOSBZMSRH4AA7S9NJQ21K2OW48VM7|System-Queue-Notification.pzProcessNotif
Alert|2018-08-30 11:02:05,334 GMT|8|PEGA0005|    3377|     250|    3127|System|BA69ER5GMGBT7B4375S5B2BWNSPYU7OYQ|System-Queue-FlowDependency.pzProcessFlo

[gvalkov]

Sorry for the delay. If the log file is not sensitive, could you please attach it so that I can debug locally?

[mbigras]

Aha! It's because the grep, sed, and awk running inside the gvalkov/tailon image are running the busybox versions.

When I'm using grepping for foobar, this is a log message from a tailon container:

tailon[25660]: 2020/01/13 07:24:39 Running command: [tail -n 10 -F /var/log/syslog]
tailon[25660]: 2020/01/13 07:24:39 Running command: [grep --text --line-buffered --color=never -e foobar]

but look what happens when I try to run the same pipeline inside the container:

root@log01-dev:~# docker exec -it tailon /bin/sh
/tailon # tail -n 10 -F /var/log/syslog | grep --text --line-buffered --color=never -e foobar
grep: unrecognized option: text
BusyBox v1.27.2 (2018-06-06 09:08:44 UTC) multi-call binary.

Usage: grep [-HhnlLoqvsriwFE] [-m N] [-A/B/C N] PATTERN/-e PATTERN.../-f FILE [FILE]...

...

The error message is: grep: unrecognized option: text, which is coming from the busybox grep inside the alpine image. PR on the way! https://github.com/gvalkov/tailon/pull/26