ignore some jobs on highstate page

[pju51]

Hello, it's not a bug, but questions or improvements. I am using the separate SaltGUI (1.28.0) solution in a nginx container (nginx:1.21.6). Everything works fine but I have 2 problems with the hightstate page:

1. my saltstack master uses a Gitlab as a backend, so to deploy a specific branch with salt, I use this command on the host as an example: salt-call state.apply saltenv=XXX pillarenv=XXX test=true (XXX = git branch) The problem is that the result of the command will also be displayed on the saltgui hightstate page, but we should only see the results of the master branch and not the result of my XXX branch, because only the master branch is important to define if the server is in highstate or not. Is it possible to hide these parasitic results in order to keep only the results of the master branch? or any other solution (like to show only the state.highstate and hide everything else) ?

2. I have a lot of minions and states, and in the highstate page, the result of the highstate is far too long: Is it possible to have only the final result ? and the detail if we click on the green/yellow/red circle ? or another proper solution. I already tried this solution: https://github.com/erwindon/SaltGUI#performance, but I didn't see any change, maybe because I use a separate SaltGUI. Screenshots

thanks a lot.

NOTE: nr. 2 is now in #459 NOTE: question about salt-syndic (below) is now in #460

[pju51]

Is there also a solution if we use several salt masters (multimaster with salt-syndic) ? (see #460)

[erwindon]

@pju51 you can ask unlimited questions, but please limit to 1 question per github-issue. I will ignore the 2nd question (long lines) and 3rd question (salt-syndic) in this issue. just copy those to new github issues please. (I cannot do that for you because then I become the owner of the issues)

[erwindon]

I'm not considering to be selective on the commands that were used (state.highstate/state.apply/etc) as that will differ too much per user.

I see the following solutions:

1. adjust the page to group the results per saltenv and/or per pillarenv
2. add a configuration parameter to the /etc/salt/master file that contains a list of saltenv to ignore (or include)

one of the consequences of both solutions is that it might not be sufficient to look for the latest 10 highstate related commands. e.g. the latest 10 jobs might have been for XXX leading to no results being shown. the only available alternative is to add metadata to the important jobs, so that they can be found more easily using the metadata search facility. unfortunately, function runners.jobs.list_jobs does not have a search parameter for saltenv (or pillarenv). the easiest solution for this additional problem is to change constant MAX_HIGHSTATE_JOBS into a configuration parameter.

From the proposed 2 solutions, I think that nr. 2 is the easiest one as there are no GUI changes involved. The detailed activities for nr. 2 are:

• add parameter saltgui_hide_envs / saltgui_show_envs
• ignore all jobs on page 'highstate' that have a saltenv named parameter, and which is (not) listed in these variables.
• make MAX_HIGHSTATE_JOBS a new configuration item saltgui_max_highstate_jobs (default=10)
• adjust the hidden page (ctrl-click on title 'SaltGUI') to show the 3 new parameters

the new parameters may be used to limit the jobs shown on the jobs-page, or on the jobs-panel. but there wouldn't be a way to make the jobs visible again. so I'll ignore that possibility for now.

[pju51]

Thanks for your answer, and I will create new tickets for my other questions, sorry.

from my point of view I think the results of state.apply should be in the job page and state.highstate in the highstate page. the solution to hide is not possible because we will constantly create new git branches, so I cant create a list, maybe a wildcard (*) is accepted ?

the other thing, if I run this command: salt-call state.apply STATE_NAME test=true I will have also the result in the hightstate page when it should be in the job page, because in my command I specified which state I want to deploy not all. A highstate status should not be linked to a particular state but to all states applied to a server, so the result of my command should not be in the highstate page

[erwindon]

the solution to hide is not possible because we will constantly create new git branches,

that's why I use 2 lists: saltgui_hide_envs / saltgui_show_envs only one of these is filled in in your case that would be "saltgui_show_env: master" and all other env-s are hidden

[pju51]

Ah yes can be a good solution.

I tested; I added saltgui_show_env: master in my /etc/salt/master.d/master.conf file on my saltstack master server then I restarted the services salt-api and salt-master but I still have the problem.

Did I miss anything?

on the hidden page I can't find this information added.

[erwindon]

Did I miss anything?

:-) you made the assumption that this was already created. but no, it is still only the solution proposal for the questions in this github issue

Ah yes can be a good solution.

I will go ahead to create this solution. It will take a few days to build that. I'll invite you to review the PR (a PR to myself) that I create for it

[pju51]

Did I miss anything?

:-) you made the assumption that this was already created. but no, it is still only the solution proposal for the questions in this github issue

Ah yes can be a good solution.

I will go ahead to create this solution. It will take a few days to build that. I'll invite you to review the PR (a PR to myself) that I create for it

my bad, yes, I thought it was already implemented and that I had missed something lol :D

ok I will review and test when its ready, no problem.

[erwindon]

note: changes will be on branch highstate-envs.

note: during build of this feature, I realized that there are several occasions where structured data is read back from the session storage. I added 2 utility functions to convert such data back to an object or an array.

note: during build of this feature, I realized that there are several occasions where allowlist+denylist matching was used. I added a utility function to compare an item to an allowlist and a denylist in one function call.

[erwindon]

@pju51: the configuration parameters have a slightly different name now: saltgui_show_saltenvs or saltgui_hide_saltenvs to prevent any confusion with pillar-envs; and to indicate that the parameter can take a list of names (a string value is considered to be a 1-item list)

[erwindon]

@pju51 Can you please give the new HighState page in this branch a try?

I guess that this solution should be extended to the "Issues" page. That page tries to find failed states (and a handful of other categories). But let's do this one page at a time.

[pju51]

@erwindon Thanks, I'll test tomorrow and let you know the result.

[pju51]

@erwindon So I added in my saltmaster config: saltgui_show_saltenvs: master then I restarted the services: salt-api + salt-master

After this I used your branch highstate-envs, but seems not work:

I always see the result of other branches in the highstate page.

also on the hidden page, I dont see the variablessaltgui_hide_saltenvs or saltgui_show_saltenvs, but maybe its normal?!

[pju51]

I also tested :

saltgui_show_saltenvs:
    - base

saltgui_show_saltenvs:
    - master

unfortunately same result, not work.

[erwindon]

You are using the correct branch. Even when the behaviour is broken, you should see the new setting on the (hidden) Options page.

just in case this is an effect of the browser-cache, can you SHIFT-refresh the relevant pages? In all known browers, shift-refresh ignores the brower-cache and reloads the actual page. Otherwise, I may not have pushed my changes, but I can only verify that in a few hours time...

[erwindon]

I did push out all changes, so that was not the problem. So I'm hoping that the SHIFT-refresh (or even the normal refresh) fixes the issue...

[pju51]

So I tested again, and after a redeployment of saltgui and a refresh of the hidden page, I have the new settings, but unfortunately I see no more results, I tested both parameters:

saltgui_show_saltenvs: base

saltgui_show_saltenvs: master

I can't get any more results on the highstate page, whether I use a gitlab branch or not in my salt command but I have the result in jobs page

also another problem, when I use your branch, I lose these parameters when I want to test (or apply) salt on a minion :

when I should have something like this:

[erwindon]

after a redeployment of saltgui and a refresh of the hidden page, I have the new settings

great!

also another problem, when I use your branch, I lose these parameters when I want to test (or apply) salt on a minion

can you take a look at the webbrower console for this one, to spot for any errors? my experience is that it is easier to work on a problem when additional problems are out-of-the-way. therefore I focus on this one first.

[erwindon]

can you take a look at the webbrower console for this one, to spot for any errors?

never mind, I already reproduced it here

[erwindon]

the problem was in the code that I cleaned up in relation to this issue, that is fixed now.

[erwindon]

there is something really strange in this screenprint. pillarenv and saltenv are not specified as named parameters, you seem to be passing a string with the literal value "saltenv=SOP-3967", which is a single string, not a named parameter.

for a named parameter, SaltGUI would display it as saltenv="SOP-3967" but "saltenv=SOP-3967" is just a single string

[erwindon]

note that as part of the solution, a decision had to be made on jobs that do not have a saltenv= parameter. these jobs are always displayed, no matter what the saltgui_show_saltenvs / saltgui_hide_saltenvs settings are. (feel free to propose alternative solutions)

the jobs in your screenprints do not have a named parameter saltenv=, and are thus subject to that rule

[pju51]

my command is litteraly like that: sudo salt-call state.apply saltenv=SOP-4116 pillarenv=SOP-4116 --state-output=changes --state-verbose=False where SOP-4116 is a gitlab branch name in my salt repository.

If I want to apply the master branch of gitlab from my salt repository, the command is : sudo salt-call state.apply --state-output=changes --state-verbose=False test=true

but if i use your new saltgui branch (highstate-envs) with any of these above commands , I see only the result in the job page, nothing in highstate page. normally if i don't add a saltenv, I should see the result, right?

the alternative solution for me is : the highstate page shows only the result of state.highstate, this module is created for that. cf: https://docs.saltproject.io/en/latest/ref/modules/all/salt.modules.state.html#salt.modules.state.highstate

[erwindon]

@pju51 thanks for clarifying and clearing my doubt. I will just use these commands and see where the problem actually is

[erwindon]

@pju51 I retried the first command, and when collecting information on the job, salt-api is returning data that suggests that ordinary string parameters have been used. The difference is that I typically only use the salt command, and rarely the salt-call command. with these facts, I'll have to adapt the code to also recognize string parameters that are actually key=value items...

[erwindon]

the updated code is available on the branch. the overview should now also use the saltgui_show_saltenvs / saltgui_hide_saltenvs for jobs that were started using salt-call @pju51 can you please try again?

[pju51]

I will test. sometime I use the salt command. It depends on the circumstances if I want to run the command from a master (salt) or directly on the minion (salt-call).

[pju51]

@erwindon ,I tested, but now I always have the result when I use saltenv or not :

• without saltenv:

• with saltenv

you'll notice that I ran the salt command and if I use the salt-call command on the minion, I get the same result.

I see an another problem, I explain, I have a schedule on all minions that every night execute state.highstate, it allows me to know which minions are in highstate but currently on saltgui the highstate page also returns the results when applying a particular state...so currently everything is mixed. Is it possible to create a new page dedicated only to state.highstate name global highstate or whatever ? and perhaps the simplest solution here would be not to change the highstate page, but to add this new dedicated page to state.highstate and nothing else.

[erwindon]

I see an another problem, I explain, I have a schedule on all minions that every night execute state.highstate, it allows me to know which minions are in highstate but currently on saltgui the highstate page also returns the results when applying a particular state...so currently everything is mixed. Is it possible to create a new page dedicated only to state.highstate name global highstate or whatever ? and perhaps the simplest solution here would be not to change the highstate page, but to add this new dedicated page to state.highstate and nothing else.

Can you please create a separate request for that? I'm thinking about a drop-down menu on the highstate page, like on the 'Jobs' page, so the tactics for the page can specified. (please do not reply on this comment here in this issue)

[erwindon]

note, you may want to hide some details from the images like you did with previous images...