cbt icon indicating copy to clipboard operation
cbt copied to clipboard
verified publisher icon ceph

cbt: creating a common base class for fio subclasses

Open harriscr opened this issue 1 year ago • 4 comments

This is the first part of an effort to rationalise the 5 current fio classes into a single class structure. It will mean the users of cbt no longer need to know the subtle differences between the differing fio classes. The change adds typing for methods and variables (using mypy) and pep8 compliance (using ruff) to the fio* files changed.

This is my first pull request so I expect there is something I've missed in the process.

This has been tested by:

  1. running the unit tests in the test sub directory
  2. Validating that the client_endpoint_rbd_4K_rand_write.yaml configuration file used in teuthology results in the same generated fio cli with and without these changes
  3. Validating that the example yaml for fio (example-client_endpoints.yaml) generated the same cli to run fio with and without the changes

harriscr avatar Oct 11 '24 08:10 harriscr

Adding @sseshasa as well since the refactoring changes seem important

perezjosibm avatar Oct 14 '24 10:10 perezjosibm

@harriscr can you kindly post the snapshots showing the tests that you run? Also, it would be very useful the class diagrams that you got from pyreverse (new vs old).

I have some problems to understand what you mean by:

It will mean the users of cbt no longer need to know the subtle differences between the differing fio classes.

I'm guessing the main difference between each FIO subclass instance would be the underlying FIO engine (eg.libaio, librd, etc.) Or you mean "block" vs 'file" IO? Could you illustrate with an example please?

Thanks

perezjosibm avatar Oct 14 '24 10:10 perezjosibm

teuthology runs are: perf-basic - all passed https://pulpito.ceph.com/harriscr-2024-11-05_18:48:16-perf-basic-wip-harriscr-cbt-refactor-2024-10-17-0839-distro-default-smithi/

rados/perf - all passed https://pulpito.ceph.com/harriscr-2024-11-05_18:51:09-rados:perf-wip-harriscr-cbt-refactor-2024-10-17-0839-distro-default-smithi/

crimson-rados/perf - 2 failures - 'hit max job timeout' when running radosbench, which is not effected by these changes https://pulpito.ceph.com/harriscr-2024-11-05_18:49:41-crimson-rados:perf-wip-harriscr-cbt-refactor-2024-10-17-0839-distro-default-smithi/

harriscr avatar Nov 07 '24 09:11 harriscr

Here are the pyreverse diagrams: Classes before the change: classes_before Classes after the change: classes_after Packages before the change packages_before packages after the change: packages_after

harriscr avatar Nov 07 '24 14:11 harriscr