include_prefix=False does not exclude the prefix

[InconsolableCellist]

trafficstars

1. Generate text on a model with:

#!/usr/bin/python3        
                              
import gpt_2_simple as gpt2                          
                                         
sess = gpt2.start_tf_sess()
gpt2.load_gpt2(sess)                                                             
                     
text = gpt2.generate(sess,
        prefix="<|startoftext|>",                                            
        return_as_list=True,     
        include_prefix=False)[0]

Observed behavior: Output is generated text that starts with "<|startoftext|>"

Expected behavior: Whatever is included in prefix is used for generation but doesn't appear in the output. In this case "<|startoftext|>" doesn't appear.

This is a useful feature if you're trying to prefix the model with relevant text while omitting other parts, as one might do when the input ends up being longer and longer (like in a game).

I also tried with
truncate="<|endoftext|>" with no change in observed behavior. I tried this because of https://github.com/minimaxir/gpt-2-simple/blob/master/gpt_2_simple/gpt_2.py#L478

There appears to at least be a bug where include_prefix isn't even looked at unless truncate is also set.

Related to #42 and possibly #18 (but stated more clearly here, with steps to reproduce)

[InconsolableCellist]

PR: https://github.com/minimaxir/gpt-2-simple/pull/154

Will interfere with another pending PR: https://github.com/minimaxir/gpt-2-simple/pull/87/files

[HitLuca]

Any update on this?

[HitLuca]

wouldn't be straight-forward, given that you will be sure to have the prefix at the start of your generated text, to do something like this?

if not include_prefix:
    text = text.replace(prefix, "", 1)

[soneo1127]

I think gen_text = enc.decode(out[i].tolist()[len(context_tokens):]) is one solution

[xaedes]

I have the same problem. I just use this to strip the prefix away: result[len(prefix):]. Works for me

[NeonWizard]

This is still an issue. Looks to be because include_prefix is only respected when truncated is truthy.