Move roofline plot generation to analyze mode

[sohaibnd]

This is to address https://github.com/ROCm/rocprofiler-compute/issues/435

Still, the only thing I need would be a way to generate the roofline PDFs for a given kernel after having recorded performance counters for all kernel.

I believe that would be a useful feature because we don't want to re-profile the whole program each time we want a roofline for a different kernel. This feature seems to already exist in the GUI, I would appreciate having the same capabilities built in the CLI

By moving the creation of the roofline plot pdfs to analyze mode on cli, we can create a roofline plots with different kernel filtering without having to re-profile the whole program.

[coleramos425]

@sohaibnd make sure you're also updating any CI tests that leverage the flags that are modified in this PR, i.e., https://github.com/ROCm/rocprofiler-compute/blob/c90b6a4b8e4149f9b213d5734dc94fdbb843372b/tests/test_profile_general.py#L1575

If this introduces a new "roofline" mode, we may want to add logic for this in tests to ensure it isn't being missed

[etiennemlb]

Hello, any new on that feature, it would be very useful to our use cases

[jayhawk-commits]

This PR has merge conflicts that need to be resolved. It cannot be automatically imported to the ROCm/rocm-systems super repo.

[vedithal-amd]

This will be tracked as an issue in new repo https://github.com/ROCm/rocm-systems/issues/1269