redhat-performance
HIL integration with QUADS -- Should it be client side, server side or both
I have some calls in quads-cli that make restful requests directly to HIL. There are some calls that make restful requests to HIL, fetch info from HIL and then using those parameters makes calls to QUADS.
I wanted to know which approach is worth polishing up. Also moving HIL logic server side seems to be difficult (but doable) because I do not seem to find any config file for server side. Where should I define the HIL attributes ? currently I am using the quads/config/quads.yml file to define the interface of HIL for QUADS.
and Lib/Quads.py does not have a notion of reading a config file.
Currently there is an option to turn HIL interaction on or off from the config file. There is no such logic available that Quads.py can take advantage of.
This is a design issue and needs discussion.
Option 1: one way is to not have any info in the client config yaml file and put all logic in Quads.py. every time we deal with nodes and networks related to HIL, we simply invoke the —allocator ‘HIL” (or —allocator QUADS) flag. By default it will be always ‘—allocator QUADS”
This may require major code changes, adding new api calls or modifying existing api calls.
Option 2: The way it is now. Where calls are made to Quads.py via the daemon and HIL is called from client directly. I feel this is doable for short term but it is not a long term solution.
In the long term some calls to HIL has to be on the server side. Query HIL can be done directly to HIL without QUADS server being bothered.
Let me know your thoughts on it.
Sorry for not responding to this sooner, a lot has changed with the code base but I want to keep this discussion and implementation points open, moving the milestone to 1.2 where it might be a great place to have it fleshed out and start testing it.
I have some calls in quads-cli that make restful requests directly to HIL. There are some calls that make restful requests to HIL, fetch info from HIL and then using those parameters makes calls to QUADS.
I wanted to know which approach is worth polishing up. Also moving HIL logic server side seems to be difficult (but doable) because I do not seem to find any config file for server side. Where should I define the HIL attributes ? currently I am using the quads/config/quads.yml file to define the interface of HIL for QUADS.
This is the right place as conf/quads.yml also contains bin/quads-server runtime options but it's also easily included as a configuration file object for any tool, bridge or library that might expand functionality like using HIL for provisioning.
e.g.
from quads.config import conf
if conf["HIL_some_option_enabled"]:
and Lib/Quads.py does not have a notion of reading a config file.
A lot has changed, best to look at 1.1 or master branch currently only, we're just keeping the 1.0 branch as the temporary default while we get 1.1/master battle-hardened, and we have one last R&D scale environment running 1.0.
Currently there is an option to turn HIL interaction on or off from the config file. There is no such logic available that Quads.py can take advantage of.
This is a design issue and needs discussion.
Option 1: one way is to not have any info in the client config yaml file and put all logic in Quads.py. every time we deal with nodes and networks related to HIL, we simply invoke the —allocator ‘HIL” (or —allocator QUADS) flag. By default it will be always ‘—allocator QUADS”
This may require major code changes, adding new api calls or modifying existing api calls.
Option 2: The way it is now. Where calls are made to Quads.py via the daemon and HIL is called from client directly. I feel this is doable for short term but it is not a long term solution.
See the new model / API in quads/api_v2.py for this, I think most of these points aren't relevant anymore due to the large amount of changes, refactoring and general improvements from 1.0 to 1.1 or current master branch:
https://github.com/redhat-performance/quads/blob/master/quads/api_v2.py
In the long term some calls to HIL has to be on the server side. Query HIL can be done directly to HIL without QUADS server being bothered.
Let me know your thoughts on it.
I think a great place for this to land is the ability to have systems managed by HIL (in replacement of or addition to Foreman). In this way with a conf change (or via the quads --host-type metadata) we can set certain systems backend provisioning workflow to be used by HIL while still having some use Foreman but on the front-end everyone can still use the same quads scheduling framework.
We had discussed early-on support for other backend provisioning mechanisms in but simply just don't have another one in production to drive the implementation of this. Maybe this is a good impetus to pick this back up.
It is good to know that there is still interest in integrating HIL with QUADS. Keep me in loop as things progress.
Regards, Sahil
On Thu, Mar 7, 2019 at 7:40 AM Will Foster [email protected] wrote:
Sorry for not responding to this sooner, a lot has changed with the code base but I want to keep this discussion and implementation points open, moving the milestone to 1.2 where it might be a great place to have it fleshed out and start testing it.
I have some calls in quads-cli that make restful requests directly to HIL. There are some calls that make restful requests to HIL, fetch info from HIL and then using those parameters makes calls to QUADS.
I wanted to know which approach is worth polishing up. Also moving HIL logic server side seems to be difficult (but doable) because I do not seem to find any config file for server side. Where should I define the HIL attributes ? currently I am using the quads/config/quads.yml file to define the interface of HIL for QUADS.
This is the right place as conf/quads.yml also contains bin/quads-server runtime options but it's also easily included as a configuration file object for any tool, bridge or library that might expand functionality like using HIL for provisioning.
e.g.
from quads.config import conf
if conf["HIL_some_option_enabled"]:
and Lib/Quads.py does not have a notion of reading a config file.
A lot has changed, best to look at 1.1 or master branch currently only, we're just keeping the 1.0 branch as the temporary default while we get 1.1/master battle-hardened, and we have one last R&D scale environment running 1.0.
Currently there is an option to turn HIL interaction on or off from the config file. There is no such logic available that Quads.py can take advantage of.
This is a design issue and needs discussion.
Option 1: one way is to not have any info in the client config yaml file and put all logic in Quads.py. every time we deal with nodes and networks related to HIL, we simply invoke the —allocator ‘HIL” (or —allocator QUADS) flag. By default it will be always ‘—allocator QUADS”
This may require major code changes, adding new api calls or modifying existing api calls.
Option 2: The way it is now. Where calls are made to Quads.py via the daemon and HIL is called from client directly. I feel this is doable for short term but it is not a long term solution.
See the new model / API in quads/api_v2.py for this, I think most of these points aren't relevant anymore due to the large amount of changes, refactoring and general improvements from 1.0 to 1.1 or current master branch:
https://github.com/redhat-performance/quads/blob/master/quads/api_v2.py
In the long term some calls to HIL has to be on the server side. Query HIL can be done directly to HIL without QUADS server being bothered.
Let me know your thoughts on it.
I think a great place for this to land is the ability to have systems managed by HIL (in replacement of or addition to Foreman). In this way with a conf change (or via the quads --host-type metadata) we can set certain systems backend provisioning workflow to be used by HIL while still having some use Foreman but on the front-end everyone can still use the same quads scheduling framework.
We had discussed early-on support for other backend provisioning mechanisms in but simply just don't have another one in production to drive the implementation of this. Maybe this is a good impetus to pick this back up.
— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/redhat-performance/quads/issues/179#issuecomment-470510488, or mute the thread https://github.com/notifications/unsubscribe-auth/ACSZ_g0zxsNjGH-BWauIycs5Xm4otiAHks5vUQjZgaJpZM4T6yjx .
-- --Sahil
Hey @SahilTikale it's been a while and QUADS 2.0 is now released. So much has changed (Flask vs CherryPy, Postgresql vs MongoDB etc).
https://github.com/redhat-performance/quads/releases/tag/v2.0.0
I think the discussion here is quite outdated at this point but we''d still love to see where any integration points may lie. I going to close this but we can reference it later. If you're still working on HIL take some time to look over the new 2.0 architecture and let's see if there are places we can integrate.