Special Character Handling

Open AdamWP opened this issue 8 years ago • 1 comments

How does this project handle special characters such as currency symbols, apostrophes, percent symbols, etc (¢, $, %, ')? Can I just adjust the vocabulary and include these symbols in my dataset?

TARGET_VOCAB_SIZE = 26+10+3 # 0: PADDING, 1: GO, 2: EOS, >2: 0-9, a-z)

Jan 08 '18 02:01 AdamWP

Please help me on including the special characters

May 20 '18 17:05 balajiwix