mamba icon indicating copy to clipboard operation
mamba copied to clipboard

clarification on how to interpret kernel size for conv1d

Open ScottHoang opened this issue 1 year ago • 1 comments
trafficstars

Can we interpret the convolution kernel size as the context length? Would increasing kernel size allows longer range context?

ScottHoang avatar Aug 13 '24 04:08 ScottHoang

No, the context length is whatever sequence length you use as inputs. We typically use kernel size 2, 3, 4 for the conv1d

tridao avatar Aug 13 '24 05:08 tridao