clip-video-encode icon indicating copy to clipboard operation
clip-video-encode copied to clipboard

set up resuming for failed runs

Open iejMac opened this issue 3 years ago • 4 comments

something like rom suggested where you generate input shard first then you just test if the output shard for that input shard already exists

iejMac avatar Oct 15 '22 21:10 iejMac

  • some additional json for if the shard is complete or not

iejMac avatar Oct 15 '22 21:10 iejMac

make it like in video2dataset using input_sharder and all

iejMac avatar Nov 20 '22 07:11 iejMac

or we can hack it for now: when you're done writing a shard, write a key_stat.json file as a signal for that shard being done

iejMac avatar Nov 20 '22 07:11 iejMac

based on found .json files in dataset folder update this https://github.com/iejMac/clip-video-encode/blob/44f2214032c9219663899cd4e3ef69724af6f619/clip_video_encode/clip_video_encode.py#L113

iejMac avatar Nov 20 '22 07:11 iejMac