stocator icon indicating copy to clipboard operation
stocator copied to clipboard

Double slash in COS parquet path is being added

Open shivprakashy opened this issue 2 years ago • 1 comments

The parquet files are failing during rename operation because of double slash (//) in the path. This has occurred after merging https://github.com/CODAIT/stocator/issues/318

Solution: The rename for spark-events flow should be made conditional and the existing code is made default. handle rename differently for eventlog appstatus files

if (pathToKey(src).endsWith(".inprogress")) {
// changed logic if the 'rename' method is called on file ending ".inprogress" (this will be appstatus file)
...
} else {
// existing code
}

shivprakashy avatar Feb 08 '23 05:02 shivprakashy

Test Results updated here

shivprakashy avatar Feb 14 '23 14:02 shivprakashy