karpathy
Can these models also be used for classification?
If we had labels for these names, such as:
| name | is_palindrome | h_index | scrabble_score |
|--------+---------------+---------+----------------|
| anna | 1 | 4 | 4 |
| jake | 0 | 1 | 15 |
| bob | 1 | 7 | 7 |
| karen | 0 | 8 | 8 |
| andrej | 0 | 11 | 14 |
| ... | | | |
Can makemore-style generative models be modified to perform classification so I can feed in a new name like asdf and get a prediction for its h_index?
While a suggestion like "add this layer here" would absolutely be helpful, I'm secretly hoping someone will share a general, intuitive way to think about repurposing machine learning models for new tasks...
Normally our training examples are tokenized like this:
<S>bob<E><S>jake<E>
But I was thinking you could append special "label" tokens:
<S>bob<E><is_palindrome=1><S>jake<E><is_palindrome=0>
Maybe this is a silly idea, but I'm going to give it a try and see if it works. At least it won't require changing the model architecture very much.
Normally our training examples are tokenized like this:
<S>bob<E><S>jake<E>But I was thinking you could append special "label" tokens:
<S>bob<E><is_palindrome=1><S>jake<E><is_palindrome=0>Maybe this is a silly idea, but I'm going to give it a try and see if it works. At least it won't require changing the model architecture very much.
Did you have any luck with this?
