Mridul M
Mridul M
Took a quick pass through it, sorry for the delay. +CC @JoshRosen as well.
+CC @shardulm94, @robreeves
@JacobZheng0927, please add it as a unit test.
+CC @HeartSaVioR
Note that when driver crashes, the event file remains with `.inprogress` suffix. Not deleting these files would result in filling up the event directory - and eventually fail all jobs...
`reader.completed` is checking for the `IN_PROGRESS` suffix - which will be the case here @bluzy: and so with this PR, it will not clean up those files. (Some users/deployments have...
+CC @shardulm94
Your review covers what I would have added @dongjoon-hyun :-)