tranduce to file abstraction?

[metasoarous]

@mahinshaw I just found this note I left myself... Not sure if it still makes sense or not but here it is:

There was this chunk of code in the transducer work:

   (transduce (comp (batch batch-size)
                    (map #(impl/apply-kwargs csv/write-csv % writer-opts)))
              (fn [w rowstr]
                (.write w rowstr)
                w)
              file
              vect-rows)))))

Should this be abstracted into something more general? Does this make any sense?

[mahinshaw]

Looks like you are writing back into a file. I will take a look at what we have for writing to files and see where it fits.

[metasoarous]

Awesome; thanks @mahinshaw.

Looking at this again, I think the general gist of my idea was that this might be some sort of helper which would let us pass in a transducer (would batch be there by default?), and throw it some data. Could maybe be nice for implementing some of what we have for writing already, but also serve as something generally useful? If it doesn't make sense to you to do this, let's not worry about it. Mostly just didn't want to let this note I found disappear into the ether without consideration.

[mahinshaw]

@metasoarous Looking at spit-csv in transducers, we basically have the above, plus more. The main difference is there is a lazy sequence, which is built from a transducer, that is passed to reduce. It may be that your note was about refactoring the reduce and using transduce instead. I'm not sure though

[metasoarous]

Ah... Good catch.