About the improvement of ccnet

[shangqingchen]

I understand the question about the evaluation result of ccnet. Thank you very much for your reply. I also have another question about the improvement point of ccnet. Whether we can only pay cross attention to four nodes around the blue information point when we do cross-focus, can we improve the computing efficiency? I do a test as if it is feasible. I am eager to get your academic guidance. Thank you very much.

[speedinghzl]

It may be feasible. It's difficult to predict the result of the local and sparser attention. This is ICCV17 paper about local attention, hope it helps.

[lxtGH]

I disgree. No global information are embedded.

[shangqingchen]

Baidu ZhiHu has a CCNET article on it and a comment on it below. If you look at it, the overall situation will not be lost, and the author thinks it is feasible.

[shangqingchen]

I am making this improvement and will use the validation set to analyze the success of the improvement.

[lxtGH]

It is feasible if you stack more layers to get final global results.