agentscope
agentscope copied to clipboard
modelscope
Model wrapper with local fine-tuning
name: Pull Request about: Create a pull request
Description
Added features to download models from hugging face model hub/load local hugging face model and fine-tune loaded model with hugging face dataset. Model loading and fine-tuning can happen both at the initialization stage and after the agent has been initialized (see README in agentscope/examples/conversation_with_agent_with_finetuned_model for details). Major changes to the repo include creating the example script conversation_with_agent_with_finetuned_model.py, adding a new model wrapper HuggingFaceWrapper in agentscope/examples/conversation_with_agent_with_finetuned_model/huggingface_model.py, and creating a new agent type Finetune_DialogAgent in 'agentscope/examples/conversation_with_agent_with_finetuned_model/finetune_dialogagent.py'. All changes are done in a new example directory agentscope/examples/conversation_with_agent_with_finetuned_model.
To test, run agentscope/examples/conversation_with_agent_with_finetuned_model/conversation_with_agent_with_finetuned_model.py by following the instructions in the README in the same directory.
Checklist
Please check the following items before code is ready to be reviewed.
- [x] Code has passed all tests
- [x] Docstrings have been added/updated in Google Style
- [x] Documentation has been updated
- [x] Code is ready for review
Need to disable E203 for pre-commit run. It automatically reformats `examples/load_finetune_huggingface_model/huggingface_model.py' line147 by adding whitespace before ':' and then flags it as an error.
The Description of this PR seems outdated, please update it accordingly.
Done.
Please see inline comments, and:
1. It's not clear who is responsible to fine-tune the model, the agent or model wrapper? Since the new model configuration has all required arguments for fine-tuning, why don't we just implement the fine-tuning functionality all in the new model wrapper class (e.g. the constructor function)? So that all other agent in AgentScope can re-use the model wrapper and fine-tune their local models without modifications.
Reloading and fine-tuning models after an agent has been instantiated need to introduce a new method to the agent class. If this is not required, this model wrapper in principle supports loading and fine-tuning for other types of agents.
Please see inline comments, and:
1. It's not clear who is responsible to fine-tune the model, the agent or model wrapper? Since the new model configuration has all required arguments for fine-tuning, why don't we just implement the fine-tuning functionality all in the new model wrapper class (e.g. the constructor function)? So that all other agent in AgentScope can re-use the model wrapper and fine-tune their local models without modifications.Reloading and fine-tuning models after an agent has been instantiated need to introduce a new method to the agent class. If this is not required, this model wrapper in principle supports loading and fine-tuning for other types of agents.
Yes, and my question is that why we need to reloading and fine-tuning models within the agent? In my view, once we create a huggingface model wrapper, we have already decided to fine-tune the model. So that the training can be finished automatically within the constructor function of the model wrapper as follows?
class HuggingFaceWrapper(ModelWrapperBase):
def __init__(self, *args, **kwargs):
# load model
self.model = self.load_model(xxx)
# fine tuning
self.fine_tune_model()
# ...
In this way, the agent doesn't need to do anything, and developers only need to set their model configuration.
Please see inline comments, and:
1. It's not clear who is responsible to fine-tune the model, the agent or model wrapper? Since the new model configuration has all required arguments for fine-tuning, why don't we just implement the fine-tuning functionality all in the new model wrapper class (e.g. the constructor function)? So that all other agent in AgentScope can re-use the model wrapper and fine-tune their local models without modifications.Reloading and fine-tuning models after an agent has been instantiated need to introduce a new method to the agent class. If this is not required, this model wrapper in principle supports loading and fine-tuning for other types of agents.
Yes, and my question is that why we need to reloading and fine-tuning models within the agent? In my view, once we create a huggingface model wrapper, we have already decided to fine-tune the model. So that the training can be finished automatically within the constructor function of the model wrapper as follows?
class HuggingFaceWrapper(ModelWrapperBase): def __init__(self, *args, **kwargs): # load model self.model = self.load_model(xxx) # fine tuning self.fine_tune_model() # ...In this way, the agent doesn't need to do anything, and developers only need to set their model configuration.
I have a long-term use case scenario in mind, where there might be new data becoming available after the agent has been deployed (i.e., continual learning setting). In this case, the agent might need to be fine-tuned (potentially multiple times) after deployment. One such example we touched on during the discussion is fine-tuning the agents from their interaction traces in a multi-agent setup. Perhaps I can rename the agent class to better reflect this use case?
Please see inline comments, and:
1. It's not clear who is responsible to fine-tune the model, the agent or model wrapper? Since the new model configuration has all required arguments for fine-tuning, why don't we just implement the fine-tuning functionality all in the new model wrapper class (e.g. the constructor function)? So that all other agent in AgentScope can re-use the model wrapper and fine-tune their local models without modifications.Reloading and fine-tuning models after an agent has been instantiated need to introduce a new method to the agent class. If this is not required, this model wrapper in principle supports loading and fine-tuning for other types of agents.
Yes, and my question is that why we need to reloading and fine-tuning models within the agent? In my view, once we create a huggingface model wrapper, we have already decided to fine-tune the model. So that the training can be finished automatically within the constructor function of the model wrapper as follows?
class HuggingFaceWrapper(ModelWrapperBase): def __init__(self, *args, **kwargs): # load model self.model = self.load_model(xxx) # fine tuning self.fine_tune_model() # ...In this way, the agent doesn't need to do anything, and developers only need to set their model configuration.
I have a long-term use case scenario in mind, where there might be new data becoming available after the agent has been deployed (e.g., continual learning setting). In this case, the agent might need to be fine-tuned (potentially multiple times) after deployment. Perhaps I can rename the agent class to better reflect this use case?
Okay, I understand your consideration. For me, agent.model.fine_tune() in continual learning setting is also acceptable. Whatever, please ensure the other agents can directly use this huggingface model configuration without modifying their code. Others plz see inline comments.
Further update of this pull request can be found here as it was moved to a new branch https://github.com/modelscope/agentscope/pull/240