Add model training GPU regression test

[EtiennePerot]

Description

Our only GPU regression tests are doing inference. We should have a test that does some training too.

Is this feature related to a specific bug?

#10879

Do you have a specific solution in mind?

#10879 has an example Dockerfile which is modeled after https://github.com/karpathy/llm.c/discussions/677.