deep-learning-with-python-notebooks icon indicating copy to clipboard operation
deep-learning-with-python-notebooks copied to clipboard

question: why is multi-hot encoding appropriate for sequences?

Open mbutterick opened this issue 3 years ago • 0 comments

In Deep Learning with Python 2e, @fchollet says:

Multi-hot encode your lists to turn them into vectors of 0s and 1s. This would mean, for instance, turning the sequence [8, 5] into a 10,000-dimensional vector that would be all 0s except for indices 8 and 5, which would be 1s.

Wouldn’t this encoding essentially reduce the sequence to a set, which means losing information? For instance, how would the encoded representation of [8, 5] differ from [5, 8] or [8, 5, 8] (or any longer sequence of 8 and 5 elements)?

(This example arises in a tutorial about classifying movie reviews by whether they’re positive or negative. To make the example concrete, if word 5 is pretty and word 8 is awful, then there’s going to be a classification difference between pretty awful and awful pretty!)

mbutterick avatar Aug 25 '22 14:08 mbutterick