memory use

[lsmith77]

I am running into issues with memory use as I integrated your solution into an API.

Now I am pondering ways to reduce memory use and increase scalability. One idea I have is to move all these mappings into Redis. In theory this way I could load all of this into memory only once regardless of how many workers I have. As long as Redis manages to hold up and not introduce too much latency, this could in theory be a very memory efficient solution.

I need to implement this first but the question is if you would accept a PR that would optionally use Redis?

[gambolputty]

Hello! I only see a blank page when I click your link.

Interesting question, though. Did you find out what caused high memory usage/ why it did not scale?

I think redis would be too much for this small package. Another person suggested using SQLite (#6) and I think this would be the way to go. It's superfast and doesn't operate on a different process. Since there are no writing processes, even better.

[lsmith77]

yeah that could be interesting ..

it would also allow for fallback logic ..

f.e. if “Lehrperson” is missing, you could look for “ehrperson”, “hrperson”, “person”. this is something I have implemented as well already.

note this strategy can lead to false positives. f.e. “arbeitende” would lead to “ende”.