An API for interegating recorded data

[dnk8n]

Is there an existing proposal for this?

• [X] I have searched the existing proposals

Is your feature request related to a problem?

I enjoy the flamegraph implementations, etc meant for human eyes. What I would love to see is an API to programmatically pull out the desired metrics of potentially massive amounts of runs (different inputs, etc)

Describe the solution you'd like

I would love if Memray had functions to pull out specific useful data, like peak memory for various dimentions

Alternatives you considered

My current workaround (very hacky, but all I could figure out due to lack of options):

import subprocess
import json
import re
from bs4 import BeautifulSoup
from pathlib import Path
import numpy as np

def pad_arg(arg: int, leading_zeros: int = 6):
    return str(arg).zfill(leading_zeros)

peak_memory_matrix_raw = []
peak_memory_series_x = []
peak_memory_series_y = []
y_axis = range(1000, 11000, 1000)
x_axis = range(100, 1100, 100)
for row in y_axis:
    peak_memory_row_raw = []
    for col in x_axis:
        subprocess.run([f"memray flamegraph analysis/mem-pivot-{pad_arg(row)}-{pad_arg(col)}.bin"], shell=True)
        with Path(f"analysis/memray-flamegraph-mem-pivot-{pad_arg(row)}-{pad_arg(col)}.html").open('r') as html:
            soup = BeautifulSoup(html)
            script = soup.find('script',  {'type': 'text/javascript'})
            memory_records = json.loads(script.contents[0].strip().split('const memory_records = ')[1].split(';')[0])
            peak_memory = max([memory_record[1] for memory_record in memory_records])
            peak_memory_row_raw.append(peak_memory)
            peak_memory_series_x.append(row * col)
            peak_memory_series_y.append(peak_memory)
    peak_memory_matrix_raw.append(peak_memory_row_raw)
peak_memory_matrix = np.matrix(peak_memory_matrix_raw)

[dnk8n]

Also on a side note, I have noticed that the peak memory reading in the statistics section of the flamegraph webpage does not correspond to the peak value plotted on the graph. It is often the second highest value of the heap (orange) graph.

[clementfaisandier]

I second this request, seems like an essential feature.

My use case: I'm running dozens of tests I want profiled for analysis, but none of the recorded data can be easily extracted through API calls... This means I need a text-based parsing solution, or just manually enter the data to an array, both seem unnecessarily complicated given how well the rest of memray seems to work.