David Underdown

icon indicating an email [email protected]

icon company @digital-preservation @nationalarchives icon location Kew, London icon location Now a Data Engineer in @digital-preservation at @nationalarchives - formerly Senior Digital Archivist and prior to that senior DBA

Results 80 comments of David Underdown

Write out error reports during processing, rather than all at end

Not included in 1.1 in the end, so re-tagged as 2.0 for now

Will CSV Validator support column definitions wild card?

See digital-preservation/csv-schema#13 - this has been suggested by another user, but we don't have resource to implement at the moment.

Parsing schema before beginning data validation can take a significant length of time with complex schema, status bar should give more info on that stage

Thanks Adam. Re point 3, have to be a little bit careful as it is permitted to have a line break within a quote-wrapped field, so number of line breaks...

Extend checksum expression to optionally produce warning or error if checksum value is for an empty message

Not included in 1.1 in the end, milestone updated to 2.0 for now

noext recorded as noExt in schema language but noext in code

noExt is more consistent with other rules, should be changed in CSV Validator to match definition

Debug log

-t is a trace of how the schema is parsed - what you're seeing there aren't errors as such. It's possible there's something there though that might shed some light...

Generate command line params from GUI

But not for command line execution does it? The files with the GUI version merely start the GUI don't they, or am missing something?

Row Terminator

Not currently no, the underlying CSV library should handle common line endings. This would need to be added to the CSV Schema Language before it could be implemented in the...

OutOfMemoryError failure when process huge csv file.

Hi, as noted in the user guide, memory can be an issue with large files, particularly if you are using the uniqueness test, as the validator has to build a...

OutOfMemoryError failure when process huge csv file.

There were some big improvements in #135 - they would certainly help with your example as caching of regex was introduced among other improvements. This is included 1.1.5 and 1.2-RC2