Introduce torch.compile to all PyTorch examples

[champon1020]

As the previous discussion https://github.com/kubeflow/training-operator/pull/2024#discussion_r1521943323, I would suggest to introduce torch.compile in PyTorch examples, following the migration of PyTorch to v2 (#2024).

I think it should be guaranteed to work torch.compile, which is the main feature of PyTorch v2 and able to speed up computation and memory access, on PyTorch operator. However, as @tenzen-y mentioned at https://github.com/kubeflow/training-operator/pull/2024#discussion_r1521683099, there is a concern about supported GPUs (or I think also CUDA versions).

Please let me know your opinions!

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[tenzen-y]

/remove-lifecycle stale

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[andreyvelich]

/good-first-issue

[google-oss-prow[bot]]

@andreyvelich: This request has been marked as suitable for new contributors.

Please ensure the request meets the requirements listed here.

If this request no longer meets these requirements, the label can be removed by commenting with the /remove-good-first-issue command.

In response to this:

/good-first-issue

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[tenzen-y]

/remove-lifecycle stale