Support self-trained model

[yangelaboy]

trafficstars

Model description

hi, my company has trained a model of 7b， we want to deploy lorax with our model. Can you introduce key steps to support model in loraX？

Open source status

• [ ] The model implementation is available
• [ ] The model weights are available

Provide useful links for the implementation

No response

[tgaddair]

Hey @yangelaboy, thanks for offering to contribute your model to LoRAX!

For adding a new model architecture to LoRAX, the best examples are probably our PRs that added Phi, Qwen, and GPT2:

• Phi
• Qwen
• GPT2

The main things to add are:

• Basic model implementation in flash_<model>_modeling.py
• Initialization and LoRA loading rules in flash_<model>.py
• Register model for loading in models/__init__.py

Please let m know if you have any additional questions I can help answer!

[yangelaboy]

@tgaddair thx for your replying, I will get a try!