Some questions about conformer

[ll643989]

Dear author, I have used your conformer recently and its effect is really good. I would like to ask you the following questions:

1. I feel that the fitting speed of the model is slow, and the test accuracy rate has great fluctuation. Is there any other optimization suggestion？
2. I have been paying attention to your Transformer. Why are the CSP and spatial channel attention used in Transformer abandoned here I would appreciate it if you could answer my questions？

[eeyhsong]

hello @ll643989, Sorry for the late reply.

1. I haven't performed detailed experiments for the parameters of the optimizer. To stabilize the training processing, you can try different batch sizes, learning rates, beta1, beta2, etc.
2. CSP is an excellent way to separate non-stationary time series such as EEG. However, you know we commonly use end-to-end manner in constructing deep learning methods. So we discard the CSP module in Conformer. But I believe combining new DL methods and traditional ML strategies would be better.