dbpedia-docs icon indicating copy to clipboard operation
dbpedia-docs copied to clipboard

inadequate (or incomplete) testing for error during bulk loading process

Open neradis opened this issue 11 years ago • 1 comments

The query used scripts / virtuoso / load_data.sql to spot errors during the loading of the datasets seems to be at least incomplete:

select * from DB.DBA.load_list where ll_state <> 2;

did not return a dataset loaded incompletely due to syntax errors in the source file when I loaded staging LGD dumps, but

select * from DB.DBA.load_list where ll_error is not null;

did.

neradis avatar Oct 05 '14 13:10 neradis

this was a copy-paste from VOS documentation (which is now updated)

Once the rdf_loader_run() is complete, you can check the DB.DBA.load_list to confirm all data sets were loaded successfully. This is indicated by an ll_state value of 2 and an ll_error value of NULL.

So ll_state <> 2 OR ll_error is not null should be better

jimkont avatar Oct 09 '14 06:10 jimkont