markov
markov copied to clipboard
load issues for token length >= 128
If you attempt to load a previously saved chain where one of the tokens is >= 128 characters, the yaml parser throws an error that the key does not exist. Saving the chain to yaml does not have this problem.
I don't think this will be easy to fix, since the problem lies with the limits of yaml. Instead, either in feed
or save
, I believe an error should be thrown to prevent unloadable chains from ever been saved.
I've just bumped the version of serde_yaml
, so it is possible that changes on their end might have resulted in error production on serialization since the result is not a well-formed yaml file. If this is not the case, I would welcome a pull request that added a well-formedness check to save
.