Tips for dealing with inconsistent number of columns

[harrisonmg]

For reasons outside of my control I often deal with partially populated CSVs - meaning that the header is fully populated but many lines are missing some trailing data without even commas to denote no data.

I have already patched out the warning that appears when moving the cursor to the lines in question, but Select operations still fail.

For my purposes, it would be completely appropriate to assume missing columns are just all empty string values. Does anybody have tips on how to do this easily? If not, I may look into modifying rainbow_csv further.

Thanks!

[mechatroner]

This sounds like a useful feature! There could actually be two ways to do it:

1. Add a command that would append missing commas to the end of each line where appropriate (although this solution may not be always suitable since it would require modification of the original file)
2. Add a new flag (e.g. --treat_missing_fields_as_empty) to RBQL which would append missing fields to each of the records processed by Select and then a special variable to vim which would be passed to RBQL as a flag if set.

Right now I, unfortunately, don't have enough spare time to work on this so if you want, feel free to create a PR, I would really appreciate your help!

[harrisonmg]

Great, I'll look into it when I get some time!