Make Avro file clean-up more robust

[iskakaushik]

          can there be an error in the middle of writing the file and lead to a file still being left around?

in that case we'll return the error, but not cleanup.

Similar cases below

Originally posted by @iamKunalGupta in https://github.com/PeerDB-io/peerdb/pull/767#discussion_r1418036041