process icon indicating copy to clipboard operation
process copied to clipboard

Document issue with large string output and check / deref

Open borkdude opened this issue 5 years ago • 0 comments

This is a deadlock:

(-> (process ["cat"] {#_#_:out (io/file "/tmp/foo.csv") :in (io/file "/Users/borkdude/Downloads/1mb-test_csv.csv")}) check)

since the stream can't write anywhere, it's waiting before it can exit.

This works:

user=> (def csv (with-out-str (-> (process ["cat"] {:out *out* :in (io/file "/Users/borkdude/Downloads/1mb-test_csv.csv")}) check)))
#'user/csv
user=> (count csv)
1000448

This also works:

(def sw (java.io.StringWriter.))
(-> (process ["cat"] {:in (slurp "https://datahub.io/datahq/1mb-test/r/1mb-test.csv") :out sw}) check)
(count (str sw)) ;; 1043005

We implemented :out :string to make this easier:

(count (-> (process ["cat"] {:in (slurp "https://datahub.io/datahq/1mb-test/r/1mb-test.csv") :out :string}) check :out))
;; 1043005

borkdude avatar Oct 20 '20 22:10 borkdude