Feature Request: Recommend and demonstrate a library for csv parsing.

[jgarte]

Just making a TODO here for adding a tutorial on how to parse CSV.

I realize that you can parse csv with split-sequence but might be nice to show to do it with a proper csv parser.

This might be a good candidate/starting point:

https://github.com/AccelerationNet/cl-csv#examples

;; read a file into a list of lists
(cl-csv:read-csv #P"file.csv")
=> (("1" "2" "3") ("4" "5" "6"))

;; read a file that's tab delimited
(cl-csv:read-csv #P"file.tab" :separator #\Tab)

;; read a file and return a list of objects created from each row
(cl-csv:read-csv #P"file.csv"
                 :map-fn #'(lambda (row)
                             (make-instance 'object
                                            :foo (nth 0 row)
                                            :baz (nth 2 row))))
;; read csv from a string (streams also supported)
(cl-csv:read-csv "1,2,3
4,5,6")
=> (("1" "2" "3") ("4" "5" "6"))

;; loop over a CSV for effect
(let ((sum 0))
  (cl-csv:do-csv (row #P"file.csv")
    (incf sum (parse-integer (nth 0 row))))
  sum)
  
  
;; loop over a CSV using iterate
(iter (for (foo bar baz) in-csv #P"file.csv")
  (collect (make-instance 'object :foo foo :baz baz)))

[vindarel]

just found that data-table is a good companion to cl-csv as it allows to access the cells by column name.

• data-table.

cl-csv "only" reads and parses the CSV. data-table allows to access all cells by row and column name (instead of by position).

;; Parse the CSV, get a data-table object.

(defun parse-csv (file)
  "Parse CSV, return a data-table object with column names and rows."
  (let ((rows (csv:read-csv (str:from-file file))))
    (make-instance 'data-table:data-table :column-names (first rows) :rows (rest rows))))

;; Iterate on rows,
;; access columns with data-table-value :row row :col-name "the col name"

(defun get-all-days (dt)
  (remove-duplicates
   (loop for row in (data-table:rows dt)
         collect (data-table:data-table-value dt :row row :col-name "Date"))
   :test #'equal))

redditors also mentioned that cl-duckdb is useful and pretty fast at parsing.

There are more libs on awesome-cl.

[vindarel]

added data-table in CIEL, quick examples: http://ciel-lang.org/#/libraries?id=csv

Just used it to help friends in a non-profit, 't was quick n easy 💪

[vindarel]

quick blog post with usage example of cl-csv, data-table and lisp-stat's data-frame: https://dev.to/vindarel/read-csv-files-in-common-lisp-cl-csv-data-table-3c9n

comments: https://www.reddit.com/r/Common_Lisp/comments/1kht5ht/read_csv_files_in_common_lisp_clcsv_datatable/ (cl-suckdb, a faster parser…)