Sterling Paramore

My vote is that it is a useful operation. Since we already have "pivot" operation, can we call this one "unpivot"?

Makes sense to me...

Thanks for the quick response. I tried some of your suggestions, but they didn't seem to help. The best I could do was convert my CSV into a hash of...

I can't supply the CSV I used in that test. But here's some publicly available data from medicare (too big to attach directly, but still only a few 10k records):...

I got similar relative benchmarks using both some of the wide and long datasets. Basically, Daru seems to take about 3-4x as long as just parsing the CSV. My suspicion...

Could you clarify what is meant by "broadcasting data structures"? I've recently been playing around a bit with PyArrow and so far it seems like a performant internal data structure....

I understand how automatic conversion could useful for quick ad hoc data analysis, but as an ETL developer using Daru, I need to maintain tight control of my data types...

I guess if you're using automatic conversion, then yes, they should convert to booleans.

By default Daru uses automatic conversion from the core Ruby CSV reader. Sounds like it might not be doing so for booleans.

I process a lot of CSV files and have gotten into the habbit of reading all fields as strings and doing conversions after it's built into a dataframe.