RAM is increasing while doing batches via API and workers

[severfire]

Bug Description

Hello,

I am doing calls via API, Server was using 7.3GB of RAM now it is using 8.33GB of RAM

Seems like there still is a leak in 1.1 version Seems like issue was also in the past.

Reproduction

Just running API calls. Installation was made via pip installation. SQLite db. Running 6 workers.

Expected behavior

RAM should not increase drastically while running API calls.

Who can help?

No response

Operating System

Ubuntu 22.04

Langflow Version

1.1

Python Version

None

Screenshot

No response

Flow File

No response

[severfire]

RAM usage grew to 19GB of RAM, abnormal

[dosubot[bot]]

Langflow version 1.1 has some known memory management practices, such as storing interactions in Message Logs for each session, which might contribute to memory usage ^[1]. However, specific memory leaks or issues in version 1.1 are not documented in the available information.

Given the situation where RAM usage is increasing significantly during API calls with multiple workers, you might want to consider the following:

1. Worker Configuration: The default number of worker processes is based on CPU count, which can impact memory usage. You can adjust the number of workers using the --workers CLI option or the LANGFLOW_WORKERS environment variable to see if reducing the number of workers helps manage memory usage better ^[2][3].

2. Session Management: Check if the session management and message logging are contributing to the memory increase. You might want to review how sessions are being handled and if there are unnecessary sessions or messages being stored.

3. Monitoring Tools: Although specific memory profiling tools are not mentioned, using external tools to monitor memory usage and identify potential leaks could be beneficial.

If these steps do not resolve the issue, further investigation into the code or reaching out to the community for insights might be necessary.

To continue talking to Dosu, mention @dosu.

---

^{Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Other}

[cbornet]

@severfire , do you have more details on which API calls you do ? Minimal steps to reproduce would be helpful.

[cbornet]

@serverfire several leaks have been fixed. Can you check again on main branch ?

[severfire]

Fixed.