filprofiler

filprofiler copied to clipboard

profiled process getting killed much too early on Mac by OOM detector

6

## Version information Fil: 2021.12.2 Python: 3.7.12 (default, Dec 20 2021, 11:33:29) [Clang 13.0.0 (clang-1300.0.29.3)] Additional context that could be valuable is that this is on MacOS Monterey on an...

petergaultney

how to cite?

2

I am using it to compare memory consumption for my code framework with others. how to cite?

pswpswpsw

Track memory from subprocesses

6

`multiprocessing` and other similar systems are basically the same program, just with multiple processes. That memory should be tracked together. Implementation ideas: 1. Each subprocess tracks its own info. 2....

itamarst

Basically like Jupyter, but need to start IPython in a special way (`filprofile python -m IPython` or something like that?).

itamarst

Figure out how to use Fil with Metaflow

1

`@conda` decorator might be good starting point?

itamarst

No one cares where NumPy is installed, probably, they care that NumPy was the one that allocated memory. So let's make the formatting a little less verbose.

itamarst

Logging the peak memory usage measurements

4

Hi, I need to measure the memory usage of operations on different matrix sizes. Is there a way that I can get the peak memory usage and log it into...

vsandinh

Insofar as Fil4prod uses this, reducing potential panics is important.

itamarst

Document usage with Airflow

1

Document how to use Fil with Airflow tasks. @timur-orudzhov could you explain a little bit how you run Airflow, and what kind of executors you use? Did you expect to...

itamarst

Document usage with Celery workers

2

Specifically, tracking memory usage on a task level.

itamarst