hyper-api-samples icon indicating copy to clipboard operation
hyper-api-samples copied to clipboard

Request for IGNORE_ERROR=>NumErrors for COPY or external table

Open rferraton opened this issue 3 years ago • 2 comments

It would usefull (i think) to have an IGNORE_ERROR option for COPY and external table with à number of errors that can be ignored during the process.

rferraton avatar May 06 '22 19:05 rferraton

can you describe your use case in more detail? Which types of errors are you referring to? Individual values which are invalid? Rows which don't have enough columns? Complete files which are missing on disk? What is the benefit of specifying an exact number of expected incorrect values over just ignoring just all such errors?

In case you aren't aware of it, yet: ON_CAST_FAILURE allows you to ignore all cast errors for CSVs

vogelsgesang avatar May 06 '22 20:05 vogelsgesang

I am currently facing incompleted rows (a row full of NUL 00 ascii characters due to the mode of extraction of the original data...but nevermind) As we want to use external table with array we will have the problem in each file of the array, so we should have n "acceptables" errors no more...

It is a classical parameter you can find in commercial ETL/ELT solutions like informatica, datastage, sap bods... On theses tools you can sometimes have the possibility to set the parameter to -1 to allow errors without limit

rferraton avatar May 08 '22 06:05 rferraton