ibm-mas
mas_appws_components: "base=latest,health=latest" no longer installs Base or Health
Collection version
28.0.0
Environment information
include_role:
name: "{{ subcomponent }}"
vars:
mas_config_scope: system
mas_app_id: manage
mas_app_channel: "9.0.x"
mas_appws_bindings_jdbc: system
mas_appws_components: "base=latest,health=latest"
loop:
- ibm.mas_devops.gencfg_jdbc
- ibm.mas_devops.suite_config
- ibm.mas_devops.suite_app_install
- ibm.mas_devops.suite_app_config
loop_control:
loop_var: subcomponent
... or something like that.
What happened?
A couple of weeks ago, Health stopped installing alongside Manage. It appears to be right around the time that @blechner committed this: https://github.com/ibm-mas/ansible-devops/commit/d18ef88f0a3201615fdefe5094202a54b5807b6b
His commit looks fine to me, and I can't pinpoint why this may have broken the role, but I thought it might be a good idea to sanity check those ternaries and defaults to make sure there's no regression.
We've probably stood up Manage + Health a thousand times successfully, so while there's a chance we broke it internally, my gut says it's the playbook (despite not knowing how or why).
Relevant log output
n/a
It doesn't, but I think I know why.
I hope @joaopauloksn or @durera can sanity check this, and then close this ticket:
According to this, the logic for this variable changed significantly in 28.0.0
The pre-foundation logic would have supported mas_appws_components: "base=latest,health=latest".
The new logic only supports the "base=latest,health=latest" syntax in the environment variable. The environment variable and its associated ansible variable are not identically formatted. This is contrary to most or all of the rest of this collection, which always populates the ansible variable from a direct lookup of the env var.
In the case of mas_appws_components, if using the ansible variable, you have to provide the full dict, post-templating, and bypass the templating step altogether:
mas_appws_components: { base: { version: latest }, health: { version: latest } }
I would have used a separate variable to maintain backwards compatibility, and I think there's still value in doing that for 28.0.1, if only to maintain the wide usage of the ansible_variable: {{ lookup('env', ENVIRONMENT_VARIABLE) | default('default') }} pattern. There's simply no good reason to diverge from that pattern here, and it will lead to confusion and complexity down the road.
I am also not sure what the hell you guys are trying to say in this line. Does the empty-string environment variable template to the literal string "Manage Foundation"? I certainly hope not. Does it template to an empty dict? Say that instead: {} (implies Manage Foundation)
Not to mention { base: { version: latest } } is invalid syntax. The appws_component filter actually renders { 'base': { 'version': 'latest' } }. These are the sort of nitpicks that really do matter.
Quite disappointed in this whole effort, but I suspect it's still salvagable.
@faangbait
Thanks for your feedback!
I'm updating the comments section as below, let me know if you think there’s anything else worth addressing here. @durera I appreciate your input as well!
Just to clarify, there hasn’t been any change in how the environment variable and the Ansible variable work today. We're not modifying the format of the mas_appws_components variable. It has always followed the structure: { 'base': { 'version': 'latest' }, 'health': { 'version': 'latest' } }.
You can see this in the existing code here: https://github.com/ibm-mas/ansible-devops/blob/968a152ccdbb85f6902683f8d89181c964e9916f/ibm/mas_devops/roles/suite_app_config/defaults/main.yml#L18
And the variable has always relied on the filter defined here to parse the environment variable:: https://github.com/ibm-mas/ansible-devops/blob/968a152ccdbb85f6902683f8d89181c964e9916f/ibm/mas_devops/plugins/filter/filters.py#L54C5-L54C21
I do agree that the pattern ansible_variable: {{ lookup('env', ENVIRONMENT_VARIABLE) | default('default') }} is a good practice, and maybe we can consider that as a future enhancement. But just to be clear, this PR did not introduce any change in behavior.
# ┌────────────────────────────────────────────────────────────┬────────────────────────────────────────────────────────────┐
# │ MAS_APPWS_COMPONENTS (env var) │ Result after parsing with ibm.mas_devops.appws_components │
# ├────────────────────────────────────────────────────────────┼────────────────────────────────────────────────────────────┤
# │ (not set) │ { 'base': { 'version': 'latest' } } │
# │ "" (empty string) │ {} (implies Manage Foundation) │
# │ "base=latest" │ { 'base': { 'version': 'latest' } } │
# │ "base=latest,health=latest" │ { 'base': ..., 'health': ... } │
# └────────────────────────────────────────────────────────────┴────────────────────────────────────────────────────────────┘
As far as I'm aware the behavior here has not changed at all since the start:
- the ansible variable needs to be an object
- we support a specific string format for the env variable to make it easier to pass in via env var than having to put e.g. a json object as a string
Comparing the handling of the var before/after v28:
Before: mas_appws_components: "{{ lookup('env', 'MAS_APPWS_COMPONENTS') | ibm.mas_devops.appws_components | default('{}', true) }}"
After: mas_appws_components: "{{ lookup('env', 'MAS_APPWS_COMPONENTS') | ibm.mas_devops.appws_components | default('{}', true) }}"
- 27.2.0 version: https://github.com/ibm-mas/ansible-devops/blob/27.2.0/ibm/mas_devops/roles/suite_app_config/defaults/main.yml#L18
- 28.0.0 version: https://github.com/ibm-mas/ansible-devops/blob/28.0.0/ibm/mas_devops/roles/suite_app_config/defaults/main.yml#L18
This part is working as expected, however there is a problem here, but it's been around for a long time it seems: https://github.com/ibm-mas/ansible-devops/commit/6e14ca7e0e14b04dfadd102ba13c741489a319a6#diff-1760f4331527147e8a3a867525f498e913a27aeb8c1515087befb18050348682
This change switched to using mas_app_components_manage, which I can't see anywhere that it's being set: https://github.com/search?q=repo%3Aibm-mas%2Fansible-devops%20mas_app_components_manage&type=code, but this change was introduced over a year ago.
This needs a deep investigation, urgently. I don't understand how this works at all right now based on what I've just reviewed.
- All the code in ibm-mas/cli and python-devops is setting
mas_appws_components/MAS_APPWS_COMPONENTS, - But for the last year this role seems to be ignoring that and instead trying to use
mas_app_components_manage, which I don't see being set anywhere. - It's entirely possible I've overlooked where it's being set as I've only been poking around the code for 20 minutes, but right now as I try to follow the code through it looks like we basically throw away the
MAS_APPWS_COMPONENTSenvironment variable when determining the Manage components, but I'm pretty sure we'd have noticed by now if this was the case so I hope I'm just missing something here.
@durera The code is not well designed IMO.
After researching again, here is what I've got:
1- Pod templates is called: https://github.com/ibm-mas/ansible-devops/blob/ff7d796a8743fdc6431cdb56a89b515091c220f3/ibm/mas_devops/roles/suite_app_config/tasks/manage/pre-config/main.yml#L8
2- Pod templates will create a fact (manageWorkspaceComponents) based on mas_appws_components
https://github.com/ibm-mas/ansible-devops/blob/ff7d796a8743fdc6431cdb56a89b515091c220f3/ibm/mas_devops/roles/suite_app_config/tasks/manage/pre-config/setup-pod-templates.yml#L16
3- mas_app_components_manage is a fact set here and will contain the values from manageWorkspaceComponents
https://github.com/ibm-mas/ansible-devops/blob/ff7d796a8743fdc6431cdb56a89b515091c220f3/ibm/mas_devops/roles/suite_app_config/tasks/manage/pre-config/main.yml#L15
In summary:
You don't find mas_app_components_manage being used because it is written as mas_app_components_{{ mas_app_id }}
The value flow is the following: mas_appws_components -> manageWorkspaceComponents -> mas_app_components_manage
Fair enough. I double-checked the git history and can confirm that we (always) used the wrong format for the ansible variable call.
I guess the mystery of why health stopped installing for us is yet to be determined, but if there's no regression, then it's not a big deal, and we can use the new format.
edit: I wish I could give a better timeline of when the issue arose. I said "a couple of weeks ago" because that's when it was raised internally to me. I seriously doubt it's been an issue for us for a year, but that's not out of the question.
I guess the mystery of why health stopped installing for us is yet to be determined
This is due to the change in the default:
- Previously the detault was base + health
- Now, the default is "there is no default" / which in Manage 9.1 onwards means that manage foundation (which is effectively the framework of manage without any components) will be installed, but in older releases is invalid usage.
Because foundation is defined by the absence of components, we can no longer have a default that says "if user doesn't choose any components then set base+health as the default".
When using MAS CLI interactive mode and selecting "No" to Manage installation, the CLI states "Foundation will be installed by default", but the foundation deployment actually fails due to missing foundation images.
spec: components: {} settings: deployment: serverBundles: - bundleType: foundation
CLI sets --manage-components "" when user selects "No" to Manage Empty components parameter tells operator to build no images Foundation server bundle requires foundation image to deploy Result: Configuration exists but no deployable image
Should foundation-only installations automatically build foundation images? Is there a proper CLI parameter for foundation-only installs? Should the CLI modify its "foundation by default" message if it won't actually work? This appears to be the same issue addressed in PR #1934 - confirming this is a known gap between CLI promises and actual functionality.
Any help will be appreciated :)
When using MAS CLI interactive mode and selecting "No" to Manage installation, the CLI states "Foundation will be installed by default", but the foundation deployment actually fails due to missing foundation images.
spec: components: {} settings: deployment: serverBundles: - bundleType: foundation
CLI sets --manage-components "" when user selects "No" to Manage Empty components parameter tells operator to build no images Foundation server bundle requires foundation image to deploy Result: Configuration exists but no deployable image
Should foundation-only installations automatically build foundation images? Is there a proper CLI parameter for foundation-only installs? Should the CLI modify its "foundation by default" message if it won't actually work? This appears to be the same issue addressed in PR #1934 - confirming this is a known gap between CLI promises and actual functionality.
Any help will be appreciated :)
@arun-ghodele can you give me more information? We have automation testing foundation installation daily. Can you please share any error message you are having? Do you have any specific customization in your environment? Which version of MAS are you installing and which applications are you including on it?
@joaopauloksn Thank you for your response. Here are the details, Let me know if you need specific logs, operator versions, or additional configuration details!
IBM Maximo Application Suite CLI Container v15.9 IBM Maximo Operator Catalog Selection - 9.1
Environment Specifications: OpenShift (4.16) Database: SQL Server Target: Mas Core deployment along with Facilities Customizations: None - standard installation
Installation Command Used mas install Selected "No" when prompted "Install Manage? [y/N]" Selected "Yes" when prompted "Install Real Estate and Facilities"
Error Messages: ManageWorkspace Status Conditions: { "type": "BuildReady", "status": "True", "reason": "Build Skipped", "message": "Build skipped - ManageFoundation." }
ManageWorkspace Deployment Status: { "type": "DeploymentReady", "status": "False", "reason": "Pending", "message": "UpdateDB/Maxinst: not initiated. Deployment foundation - not deployed" }
Overall Workspace Status { "type": "Ready", "status": "False", "reason": "Pending", "message": "Server Bundles not deployed in workspace" }
- mas install
- Select "No" when prompted "Install Manage?"
- Select "Yes" when prompted "Install Facilities?"
- Complete installation with default responses
- Observe ManageWorkspace fails to deploy foundation bundle
@arun-ghodele Build Skipped is fine because they want only foundation and did not select any components. The next one "UpdateDB/Maxinst: not initiated. Deployment foundation - not deployed" tells you that Maxinst/UpdateDB has not started. which is why the Foundation pod is not yet deployed.
You may want to look at the Manage Workspace (entitymgr-ws) operator logs and see if there are any clues there. Also look at the maxinst Pod (if it's created) and check the log there.
@joaopauloksn Thank you for the insight. It was helpful. I looked into the entitymgr-ws operator log and there is failure with foundation image build. could be due to missing roles or permissions?
oc describe manageworkspace demo Type: DeploymentReady Last Transition Time: 2025-10-15T19:34:59Z Message: unknown playbook failure Reason: Failed Status: True
logs demo-entitymgr-ws-558c4db8d-h5dp8 ok: [localhost] => {"ansible_facts": {"deploymentName": "manage-maxinst"}, "changed": false} TASK [manage-deployment : Deploy Maxinst foundation Admin Image] *************** task path: /opt/ansible/roles/manage-deployment/tasks/deployment/foundation/maxinst-foundation-deploy.yml:49 fatal: [localhost]: FAILED! => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} localhost : ok=84 changed=0 unreachable=0 failed=1 skipped=36 rescued=0 ignored=0