function attach() could not work

[koishichann]

Following 'Multitask Modeling using OpenDelta' part in the document reports error 'NotImplementedError' on line:

delta_model.attach()

As tracked in the source code in function attach(), I guess the problem is at here:

if _delta_info['method'] == "replace": setattr(submodule, _delta_info["child_name"], _delta_info['delta_module']) elif _delta_info['method'] == "insert_sequential": self.insert_sequential_module(module=submodule,_delta_info=_delta_info) elif _delta_info['method'] == "insert_parallel": self.insert_parallel_module(module=submodule, _delta_info=_delta_info) else: raise NotImplementedError

I didn't find "insert_sequential" or "insert_parallel" in other codes but found "parallel" and "sequential". I think the codes may be like:

if _delta_info['method'] == "replace": setattr(submodule, _delta_info["child_name"], _delta_info['delta_module']) elif _delta_info['method'] == "sequential": self.insert_sequential_module(module=submodule,_delta_info=_delta_info) elif _delta_info['method'] == "parallel": self.insert_parallel_module(module=submodule, _delta_info=_delta_info) else: raise NotImplementedError

I changed like this and it worked on lora without problem until now. I dont know if this edition is right so please check this. Thanks for your contribution on OpenDelta.