inventory queries and syndic

[eburghar]

Salt Syndic provides a use case where inventory queries could be useful.

Syndic helps you when you have a hierarchy of masters and you want to control minions associated with the other masters as if they were associated directly to the top master. Syndic act then as a kind of proxy. I control for example from one super master, several domains and each domain has a master with several associated minions.

If you want to control the configuration on the minions, you have to control the salt files (pillar, states, modules, ..) on their direct masters because minions retrieve states and pillar from them and not from the super master even with a configured syndic.

You simply have to copy a subset of the pillar, states and modules you want on the sub master. You do that obvioulsy a with salt formula which keep masters in sync with what you have on your super master (which has everything).

Now I found that reclass is better suited for that type of organization than pure salt (because there is a lot of similarity between domains), I am in the process of refactoring my plain sls files, and one thing i would need is a mean of accessing the full domain inventory in the syndic formula to be able to copy just what is needed to the masters (the submasters also have reclass obviously).

I will just for the moment implement a quick bridge between a state module and the reclass utility but it would be nice to have that directly in the reclass implementation.

[madduck]

Yeah, that's a nice use-case. Now it just needs to get implemented, and I can't seem to find the time :/

[eburghar]

Well madduck, I don't use salt alone anymore and rely heavily on reclass. I refactored everything around reclass and that helped to clarify a lot the whole architecture. Many thanks for sharing your work, it's brilliant. I'll try to do my best to participate now that reclass is an important part for us.

[madduck]

This sounds great. I'll try to assist whenever I can. I am also back in computer-land now after a few weeks of timeout, so hopefully more responsive…

There is an IRC channel on irc.oftc.net, maybe you want to join? I am asking specifically because @iluminite set up https://github.com/reclass and is also working, so best to coordinate…

[lottspot]

I think that first class support of salt syndics is something that we want, but I would be interested in more details about how reclass can help bridge that gap. If anyone is still interested in this feature, please feel free to add back to this thread

[fr6nco]

Hi, looks like a pretty old thread. Have anyone managed to make some updates regarding this?

I was thinking about creating a hybrid storage method, which will load the reclass locally, and if some of the classes would be missing, the reclass could send a REST / wget request to an external class storage and retrieve yml files form there.

Using this approach I could store local reclass inventory on the syndic (master) and query the space wide standard variables and classes form an external entity.

What do you think about this approach?

[eburghar]

We hit huge performance problems when using reclass directly with salt, so we decided to make a python script to generate pillar and top files per domain. We now only use git_pillar and fileverver on the syndic side which simplify a lot the overall design. A git post commit hook per domain fires a saltstack event, and pillar + fomulas are now updated within seconds (using salt states to copy formulas, nodes and classes to syndics took several minutes at best).

Using an extra layer (the script) between reclass and saltstack allows us to easily resolve problems that need introspection. by using __reclass__ in the generated dict, as well as cleaning up the pillar on the the syndic side (by removing the same __reclass__ key).

saltstack without reclass backend appears to consume less memory, be more reliable and lot more reactive. Also with git backend we have independent conf management per domain.