Support new library logic format

[b-shi]

Opening a new PR for https://github.com/ROCm/hipBLASLt/pull/1365 since there have been quite a bit of refactoring work done recently.

This PR makes a few changes to the library logic format and related for simplification purposes:

1. Fields in the library logic are now structured as key-value pairs (previously a list)
2. Removed ProblemType field in each solution - the global one is used instead
3. Add a DefaultSolution field in each logic file.
4. For each solution, parameters that are set to default their values are omitted in the logic file.
   • During library load, we reference DefaultSolution in each logic file to fill in the missing parameter values
5. Add a new field LibraryLogicVersion to track any logic file format changes in the future.
6. merge.py will automatically convert an older format library logic to the newer format.
7. The current implementation supports library logic in both a list or dictionary format.
   • I can update all the library logic to the dictionary format in this PR or do it in a separate PR.

[KKyang]

Is there a tool that can update the logic yaml to the latest format (not merging the yaml, just updating it)? Or can I just use merge.py and set the three folders to the same directory to refresh the yaml?

[b-shi]

Is there a tool that can update the logic yaml to the latest format (not merging the yaml, just updating it)? Or can I just use merge.py and set the three folders to the same directory to refresh the yaml?

Yes, currently merge.py will convert the older list format logic files to the newer format whenever it loads the yaml file (it will write back the updated file afterwards, here) so it can be used to refresh a library logic file. I also have a separate script which can update all the logic files, but that is meant as a one time thing. I would prefer to update all the logic files at once to avoid having different formats stored together.

[KKyang]

Is there a tool that can update the logic yaml to the latest format (not merging the yaml, just updating it)? Or can I just use merge.py and set the three folders to the same directory to refresh the yaml?

Yes, currently merge.py will convert the older list format logic files to the newer format whenever it loads the yaml file (it will write back the updated file afterwards, here) so it can be used to refresh a library logic file. I also have a separate script which can update all the logic files, but that is meant as a one time thing. I would prefer to update all the logic files at once to avoid having different formats stored together.

maybe we can have the script run in the CI and diff them, if there is a difference then CI fails. This will make sure the logic yamls are always up-to-date.

[b-shi]

Is there a tool that can update the logic yaml to the latest format (not merging the yaml, just updating it)? Or can I just use merge.py and set the three folders to the same directory to refresh the yaml?

Yes, currently merge.py will convert the older list format logic files to the newer format whenever it loads the yaml file (it will write back the updated file afterwards, here) so it can be used to refresh a library logic file. I also have a separate script which can update all the logic files, but that is meant as a one time thing. I would prefer to update all the logic files at once to avoid having different formats stored together.

maybe we can have the script run in the CI and diff them, if there is a difference then CI fails. This will make sure the logic yamls are always up-to-date.

Sure, I don't think we need a separate script though - we can add a check here and instead of parsing the older format, we would throw an error. This would make the build fail if any of the library logic files are using the older list format.

[b-shi]

There is an issue in merge.py related to module importing - fixing..

edit: fixed, thanks to @ellosel for the suggested fix!

[b-shi]

Or can I just use merge.py and set the three folders to the same directory to refresh the yaml?

I just checked this specifically while fixing the module import issue and yes it will work.

[b-shi]

Fixed some test failures from CI and ran tests locally (no issues).

@babakpst, @ellosel, @bstefanuk, @KKyang, any thoughts on including the library logic file changes in this PR as well? Or should this be done in a separate PR?

[jayhawk-commits]

Please resolve merge conflicts or close this PR to complete the task of importing PRs from this repo to the monorepo.