typestream icon indicating copy to clipboard operation
typestream copied to clipboard

tee command

Open dezren39 opened this issue 1 year ago • 3 comments

outputs to a local file or another stream. could be used to output file data from repl while viewing might be nice to have a utcNow() function or tee -rolling 5m and then could accept other units of time. then one could output a new file every execution. dunno, maybe it's out of scope.

dezren39 avatar Dec 17 '23 11:12 dezren39

dunno, maybe it's out of scope.

I don't think it is. In fact, the core suggestion that there should be a way to make TypeStream "mount" a local FileSystem into its operations has come up quite consistently in the conversations I had had.

I'm very curious to understand better what you're proposing with utcNow() and tee --rolling 5m. Especially the function, I think I understand the latter:

cat books | tee --rolling 5m books.ndjson

would roll books.ndjson.<datetime> files every 5 min? That seems pretty interesting also in the context of "mounting" object storage into TypeStream (s3 comes up all the time)

Is that what you have in mind?

lucapette avatar Dec 18 '23 09:12 lucapette

yes that is exactly what i had in mind! certain kinds of log or data streaming, s3 mounting would be perfect here too. I had only thought through as far as local disk. At my work, we looked at mountpoint-s3 for managing app log outputs that had a similar rolling feature.

dezren39 avatar Dec 18 '23 15:12 dezren39

Exciting!

I have to think a little about how to implement tee (I'm sure my brain will do that even if I try not to prio this, it's just too interesting!) before moving forward with it so I say we keep the issue open to reflect the current state: want to do, not sure yet how/when.

lucapette avatar Dec 18 '23 15:12 lucapette