what is the order of modules inserted by `add_mix` in Transformer?

[XiaoqingNLP]

Hello, I would like to ask two questions as follows:

1. what determines the location of the module inserted by add_mix ? If not specified, is it inserted at the end?
2. why not save the zero optimizer ? Just to save disk space?

[Sleepychord]

If any hooks of the mixins are conflicted, an error is raised, except the hook having a @non_conflict decorator. The mixins objects are saved at the model(.module if deepspeed).mixins, which is a ModuleDict.