LongMem
LongMem copied to clipboard
Victorwz
...
Hi, apology for releasing the code a little bit late due to personal issues. The code is available to public now. I will immediately organize the documentation for easier reproduction and discussion.
it's all good. Was mostly teasing. The funny thing about this paper and codebase. I was literally (5 minutes before I got a twitter push notification) building a system in my head, I had asked myself a question about where the line between current rich context and otherwise compressed context (optimized for further reasoning or stored in memory), and from that question I came up with an idea for a way to summarize current context into high level concept/ memory... stored as vectors, and later trained into a model for retrieval in future conversations.
It feels like I'm in the matrix, to think on an app, and literally have it show up minutes later.
It would be nice to publish a diff against fairseq so the modifications can be seen.
I extracted a diff here: https://github.com/facebookresearch/fairseq/compare/v0.12.2...tmm1:fairseq:longmem?diff=unified