Jim Garlick

Oh! Even though the remote subprocess is line buffered (the default), we are using `flux_subprocess_read()` in the server to consume data from the buffer and put it into messages. We...

This seems to fix that problem. ```diff diff --git a/src/common/libsubprocess/server.c b/src/common/libsubprocess/server.c index fca01444e..139923f23 100644 --- a/src/common/libsubprocess/server.c +++ b/src/common/libsubprocess/server.c @@ -246,7 +246,10 @@ static void proc_output_cb (flux_subprocess_t *p, const char *stream)...

I guess it's a trade off. The client has turned out to be pretty heavy with the buffering on that end. OTOH if we buffer on the server end then...

We tried with `-o verbose=2` and verified that we got a Flux pmi server trace, so MPI is definitely using Flux PMI. We also tried setting `FI_PROVIDER=tcp,shm` and it still...

Hmm, it seems like the output ~~prep~~ check watcher is always active, so while the output channel is open, every event loop will check the buffer for data. The prep...

> remember the cbuf_is_valid() from awhile back eating up a nice chunk of CPU Looks like cbuf complies that out if NDBEBUG is set, along with other checks. Maybe we...

another thought: since the remote subprocess already buffers data before sending it, what are we gaining by doing it again on the local side? Could we just have the message...

Ah this seems to have gotten `remote_out_check_cb` out of the picture which is progress. I guess `remote_out_prep_cb()` is continually making the same checks even when nothing has been put into...

> It seems that we don't pass LINE_BUFFERED to the "server side" subprocess, It's part of the `flux_cmd_t` object (an option) so it _should_ be getting passed through to the...

Compiling cbuf with `-DNDEBUG` may be the way to go.