Enabling multitenancy tests with FSDP

[jyothisambolu]

Before submitting

• [ ] Was this discussed/agreed via a Github issue? (no need for typos and docs improvements)
• [ ] Did you read the contributor guideline, Pull Request section?
• [ ] Did you make sure to update the docs?
• [ ] Did you write any new necessary tests?

What does this PR do?

Fixes # (issue).

PR review

Anyone in the community is free to review the PR once the tests have passed. If we didn't discuss your PR in GitHub issues there's a high chance it will not be merged.

Did you have fun?

Make sure you had fun coding 🙃

[jyothisambolu]

Category	pytorch-lightning | pypi		pytorch-lightning | source		lightning | pypi		lightning | source
	G1	G2	G1	G2	G1	G2	G1	G2
HPU General tests	PASS	PASS	PASS	PASS	PASS	PASS	PASS	PASS
HPU torch compile tests	PASS	PASS	PASS	PASS	PASS	PASS	PASS	PASS
HPU FSDP tests	PASS	PASS	PASS	PASS	PASS	PASS	PASS	PASS
HPU Deepspeed tests	PASS	PASS	PASS	PASS	PASS	PASS	PASS	PASS
HPU precision test	PASS	PASS	PASS	PASS	PASS	PASS	PASS	PASS
Standalone-only single card tests	PASS	PASS	PASS	PASS	PASS	PASS	PASS	PASS
Multi card(2) HPU test	PASS	PASS	PASS	PASS	PASS	PASS	PASS	PASS
FSDP PT Multi card(2) HPU test	PASS	PASS	PASS	PASS	PASS	PASS	PASS	PASS
FSDP Fabric Multi card(2) HPU test	PASS	PASS	PASS	PASS	PASS	PASS	PASS	PASS
Testing HPU examples	PASS	PASS	PASS	PASS	PASS	PASS	PASS	PASS

[jerome-habana]

Please rerun the tests on latest master once

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.