datawave icon indicating copy to clipboard operation
datawave copied to clipboard

Published 20 hours ago • verified publisher icon NationalSecurityAgency

1665: clean up failed job dirs

Open wgtong opened this issue 1 year ago • 3 comments

https://github.com/NationalSecurityAgency/datawave/issues/1665

wgtong avatar Aug 10 '22 13:08 wgtong

look into cleaning up file in the other exceptions within this class

wgtong avatar Aug 10 '22 20:08 wgtong

look into cleaning up file in the other exceptions within this class Perhaps it's better to clean up the dir at the stage in which the job flags the data to be reingested.

wgtong avatar Aug 11 '22 14:08 wgtong

look into cleaning up file in the other exceptions within this class Perhaps it's better to clean up the dir at the stage in which the job flags the data to be reingested.

Any thoughts? @hlgp

wgtong avatar Aug 15 '22 14:08 wgtong

TODO: integration test

wgtong avatar Aug 16 '22 14:08 wgtong

After discussing with hannah, IngestJob has code already to clean up the directory. Somehow the partitioner is escaping it.

wgtong avatar Aug 24 '22 15:08 wgtong

closing for now

wgtong avatar Sep 15 '22 20:09 wgtong