Add regression test to CI

[msaroufim]

🚀 The feature

Regression test on GPU take about 30 min so could be potentially added to CI which takes about 20 min Regression test on CPU takes more than 12h but I suspect the majority of time is spent on BERT inference where it took 1-2s per inference

If we can get both to ~30min then it's realistic to add this to CI

Motivation, pitch

Right now none of our handlers are tested in CI so it can make detecting issues a pain where we only find issues at release. It's also not realistic to ask contributors to run a test suite that takes 12h to make minor changes to handler code

Alternatives

No response

Additional context

No response