name is still unknown even id is set

[spacemanspiff2007]

I have put the pytest_benchmark_generate_commit_info conftest.py and properly return an id as per the example. However the generated .json is still in the format Windows-CPython-3.8-64bit/0001_unknown_date_time.json

How would a be able to set change the output file name format and remove the parent folder? I'd like to get it like this: 0001_my_id.json

Also how would I call the original functions from my hook so I can just remove some sensitive information without having to recollect all information myself?

[ionelmc]

Ooof, the Windows-CPython-3.8-64bit part is the machine id and that is configured way early before any of those hooks are called.

Perhaps there should be some options/hook to override that.

About the filename suffix (the _unknown_date_time) - you could override with --benchmark-save=$(git rev-parse --short HEAD) or similar. I know it's not ideal - a better way would be to add a hook to override the tag when --benchmark-autosave is used.

So that would be two new hooks. I wonder if a custom storage backend wouldn't be a better way here...

Can you give more details about all the customizations you have in your project?

[spacemanspiff2007]

Thanks for your reply!

My goal is to provide a graph of performance over time. Since I always have a main python version that is supported I'd like to compare performance regardless of python version and show it in a graph. Also I don't like to compare git versions, but rather main releases of the program. e.g.

0001_v0.1.0.json
0002_v0.2.0.json
0003_v0.3.0.json

Also how would I call the original functions from my hook so I can just remove some sensitive information without having to recollect all information myself?

Imho this is only a documentation issue. What is missing is the name and function which then can be called in the hook. I then can just redact sensitive information from the return value.

[ionelmc]

Ok what if I'd add support for using a custom storage object? Eg: --storage=my.project.module.MyCustomStorage://whatever and then your class could look like:

class MyCustomStorage(FileStorage):
  def __init__(self, value, logger, default_machine_id):
    # `value` would be "whatever"
  def save(self, output_json, default_git_id):
    # your custom thing here

[spacemanspiff2007]

But how will it work then if I want to compare performance between runs?

[ionelmc]

Well considering your custom storage would only override the save part (and dump everything in the same place) - everything would be compared to everything. You just wanted to just remove the interpreter id prefix path component right?

[spacemanspiff2007]

yes - that would work as intended. 👍