function attach() could not work

[koishichann]

Following 'Multitask Modeling using OpenDelta' part in the document reports error 'NotImplementedError' on line:

delta_model.attach()

As tracked in the source code in function attach(), I guess the problem is at here:

if _delta_info['method'] == "replace":
setattr(submodule, _delta_info["child_name"], _delta_info['delta_module'])
elif _delta_info['method'] == "insert_sequential":
self.insert_sequential_module(module=submodule,_delta_info=_delta_info)
elif _delta_info['method'] == "insert_parallel":
self.insert_parallel_module(module=submodule, _delta_info=_delta_info)
else:
raise NotImplementedError

I didn't find "insert_sequential" or "insert_parallel" in other codes but found "parallel" and "sequential". I think the codes may be like:

if _delta_info['method'] == "replace":
setattr(submodule, _delta_info["child_name"], _delta_info['delta_module'])
elif _delta_info['method'] == "sequential":
self.insert_sequential_module(module=submodule,_delta_info=_delta_info)
elif _delta_info['method'] == "parallel":
self.insert_parallel_module(module=submodule, _delta_info=_delta_info)
else:
raise NotImplementedError

I changed like this and it worked on lora without problem until now.
I dont know if this edition is right so please check this. Thanks for your contribution on OpenDelta.