hubot-markov
hubot-markov copied to clipboard
Add hubot brain storage module
Add new hubot_brain storage option to use built-in hubot persistence.
This makes it so you can use other plugins to persist hubot (like hubot-redis-brain) as a whole.
You may want to consider making this default.
This is setting the entire transition map into a single key-value pair within the brain's storage. The brain storage units that I've used (redis and postgresql) store that as a JSON blob in the associated backend. The markov model in the bot I run currently has about 2.5 million transition rows in its forward-chain model. Serializing and deserializing a JSON structure of that size on every received message would cause serious performance problems.
It's probably feasible to implement something like this backed by robot.brain.set
and robot.brain.get
with a key prefix instead. At least with the PostgreSQL brain variant I'm using, that would store each transition in a separate database row and do lookups with an indexed SELECT, which should be more scalable. (I'm not sure why I didn't do it that way to begin with - I think this module predates me finding/writing a PostgreSQL brain I was happy with?)
We should also alter the storage integration test suite to exercise this backend too.
I actually use the redis integration of markov myself. I agree that in the long-term that having it in a separate store is probably wise. Just seemed like a good option to have for small bots, or for fun development.
You won't hurt my feelings if you close this PR.