lune icon indicating copy to clipboard operation
lune copied to clipboard

Add csv library support

Open Kampfkarren opened this issue 1 year ago • 4 comments

I would like to be able to use Lune for manipulating my LocalizationTables. I don't really want to write my own Luau CSV parser, it's somewhat annoying.

Kampfkarren avatar Sep 26 '24 07:09 Kampfkarren

We do actually have a pure-luau CSV parser of sorts as a lune script. See .lune/csv_printer.luau.

Ideally this sort of thing is intended to be done as a library, I do not see much reason to include this in the runtime.

CompeyDev avatar Sep 26 '24 12:09 CompeyDev

As someone who's recently worked with 150k+ row csvs in Lune, I'd also appreciate dedicated runtime support for (fast) Rust-based serde.encode/decode("csv", text) functions in Lune's serde standard library.

My Lune-based impl. w/ string.splits took an average of 6+ seconds to parse and split the file into a 2d array-like table, whereas my string.split() + generalized character iteration approach took upwards of 15 seconds (before i ctrl Ced it) w/ the same file.

Although csvs are kinda old fashioned, having a fast way to read or write to csv is a pretty important feature to have in a runtime.

deviaze avatar Sep 28 '24 20:09 deviaze

I still personally think this kind of thing should be implemented in the language itself, or if performance is really imperative, using FFI.

CompeyDev avatar Sep 29 '24 06:09 CompeyDev

We do actually have a pure-luau CSV parser of sorts as a lune script

This is really just an example file and is meant to be easy to read - not parse all CSV files correctly. I wouldn't mind adding CSV support to @lune/serde, iff we can:

  • Agree on what CSV should look like in Lua - is it a plain 2D array? Or do we assume CSVs always have headers and use a map of header key -> columns?
  • Get a more robust implementation (handles quoting and so on, maybe even tab separated values)
  • Get noticeably improved performance

filiptibell avatar Oct 17 '24 09:10 filiptibell