just a note.

[PGTBoos]

not a bad idea at all, Dont you think the tensorflow / pytorch community would like to improve it with your idea? These days if you can compress video 3% more then it has a huge impact on data storage centers. If you can train an AI faster, 10%,... boy that's huge... think of the impact on data centers, training time. You should draw some attention from nvidea / intel.