AstroB
AstroB
@justas200 Hi! Do you have any news here? The issue is very bad at first glance
could it be that while writing files that have not yet been committed, nessie gc triggered and marked them as orphaned?
@dimas-b I suspect there may be a timing/concurrency problem in how Nessie’s GC handles data files. Consider this scenario: 1. A job is in the process of writing data files....
I noticed there is a --max-file-modification= flag for the expire operation in Nessie GC. Could this option resolve the concurrency issue described above, where freshly written (but not yet committed)...
@dimas-b So can it be the root cause of topic starter issue? If @justas200 using default >Defaults to the created timestamp of the live-content-set. Can timezone in env of nessie...