thinc
thinc copied to clipboard
`SparseLinear` does not look up weights correctly
Reported in https://github.com/explosion/spaCy/discussions/11379
The weight matrix is allocated as follows:
model.set_param("W", model.ops.alloc((nO * length,), dtype="f"))
But indexed like this:
scores[clas] += weights[idx1 + clas] * value
scores[clas] += weights[idx2 + clas] * value
Where idx1/idx2 is in [0,length) and clas is in [0, nO). So the largest index ever used is (length-1) + (clas-1), not nO * length - 1. Depending on how we look at W, we should either do something like
scores[clas] += weights[(idx1 * nO) + clas] * value
or
scores[clas] += weights[idx1 + (clas * length)] * value