Proposal: Changing how we think about test cases and endpoints

[pgagne]

This proposal is a WORK IN PROGRESS

Currently in robottelo we have a few high level constructs that basically force us to put Test cases we right into buckets of on of API, CLI, and UI test modules and test cases (ie cli/test_activation_key.py)

This is mainly enforced by a few different things.

1. Most (all?) test cases are either subclassed from APITestCase or CLITestCase (see https://github.com/SatelliteQE/robottelo/blob/master/robottelo/test.py )

2. We store test cases in high level directories by endpoint `tests/foreman/{api,cli,ui,etc}

While this may of made sense at one time, I think moving forward we should rethink how we categorize test.

I often find myself coming up with ideas for test cases, then struggling to fit them into these narrow buckets, mostly due to the fact that those test cases transcend endpoints (ie they involve working in both the CLI, API, and maybe even the UI". While we have created multiple "special cases" for these, I think these sorts of test should be the rule, rather then the exception.

I propose that we move to a new system, perhaps categorizing test by high level features or categories. Here is one such way to catagorize them:

tests/foreman
- 

to properly categorize them we could have a directory call tests/foreman/features/ with subdirectories of each feature. Existing test cases can get copied into these test

[ntkathole]

It is not convincing as per benefits Vs efforts/changes required ratio. Can you please share more on benefits of changing directory structure as per above?