feat: support lazy loading the lora module for reducing the loading p…

[thincal]

What does this PR do?

Fixes #433

Before submitting

• [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
• [ ] Was this discussed/approved via a Github issue or the discord / slack channel? Please add a link to it if that's the case.
• [ ] Did you write any new necessary tests?

Who can review?

@tgaddair