f5-aws-cloudformation icon indicating copy to clipboard operation
f5-aws-cloudformation copied to clipboard

Published 20 hours ago • verified publisher icon F5Networks

BIG-IP Version 15.1.2 fails to install appsvc3 and fails onboard

Open JeffGiroux opened this issue 4 years ago • 6 comments

Do you already have an issue opened with F5 support?

No

Description

Testing template deployments with customer who failed on a 15.1.2 AMI ID. Onboarding seems to get close, but then fails on installing appsvc3 package. I was able to replicate error when I use a 15.1.2 image. If I use 15.1.0.4, I am successful.

Template

https://github.com/F5Networks/f5-aws-cloudformation/tree/master/supported/failover/across-net/via-api/3nic/existing-stack/payg

Severity Level

3

Troubleshooting

  1. Using us-west-1 as my region
  2. Deploy template as-is and it will work fine. It references 15.1.0.4 AMI ids.
  3. Deploy again and use customImageId field with value=ami-0f8c6798cf6773d31 (15.1.2) and onboarding will fail

Relevant logs from 15.1.2 box

[admin@ip-10-0-0-239:Active:Standalone] aws # cat install.log | grep -i appsvc
verifying /config/cloud/f5-appsvcs-3.20.0-3.noarch.rpm
verified /config/cloud/f5-appsvcs-3.20.0-3.noarch.rpm
2020-12-01T21:51:22.403Z info: [pid: 17894] [scripts/onboard.js] /config/cloud/aws/node_modules/@f5devcentral/f5-cloud-libs/scripts/onboard.js called with /usr/bin/f5-rest-node /config/cloud/aws/node_modules/@f5devcentral/f5-cloud-libs/scripts/onboard.js --install-ilx-package file:///config/cloud/f5-appsvcs-3.20.0-3.noarch.rpm --install-ilx-package file:///config/cloud/f5-cloud-failover-1.6.1-1.noarch.rpm --wait-for NETWORK_CONFIG_DONE --signal ONBOARD_DONE -o /var/log/cloud/aws/onboard.log --log-level silly --no-reboot --host localhost --user admin --password-url file:///config/cloud/aws/.adminPassword --password-encrypted --hostname ip-10-0-0-239.us-west-1.compute.internal --ntp 0.pool.ntp.org --tz UTC --dns 10.0.0.2 --modules ltm:nominal --metrics cloudName:aws,region: us-west-1 ,bigIpVersion:15.1.0.4-0.0.6,customerId:c2fb325b59442c00c71e0600514ba562c3a086b50d316e0c9e0488beb53c4676d423e54310567e37a0bc307beb75d6317ac289f6be3ce6e24f7a6bedab6d418f,deploymentId:b73dd2f63c98506bf1b9b3344146b102edaa0cccb248830edf76ea14d8b33315718b9692e969af1967dcccaabd9ce3c0414594b148fb802c259f557382446021,templateName:f5-existing-stack-across-az-cluster-payg-3nic-bigip.template,templateVersion:5.9.0,licenseType:hourly -d tm.tcpudptxchecksum:software-only --ping 
2020-12-01T21:52:52.813Z info: [pid: 17894] [lib/bigIpOnboard.js] Installing package at path: /config/cloud/f5-appsvcs-3.20.0-3.noarch.rpm 
2020-12-01T21:52:53.126Z debug: [pid: 17894] [lib/bigIp.js] create localhost /shared/iapp/package-management-tasks {"operation":"INSTALL","packageFilePath":"/config/cloud/f5-appsvcs-3.20.0-3.noarch.rpm"}
2020-12-01T21:52:53.141Z debug: [pid: 17894] [lib/bigIp.js]  {"packageFilePath":"/config/cloud/f5-appsvcs-3.20.0-3.noarch.rpm","operation":"INSTALL","id":"719e3348-3279-43fc-ae3a-b6c4f86bbaa9","status":"CREATED","userReference":{"link":"https://localhost/mgmt/shared/authz/users/admin"},"identityReferences":[{"link":"https://localhost/mgmt/shared/authz/users/admin"}],"ownerMachineId":"28f72e64-ff60-4486-b4c3-14398b8f5c8b","generation":1,"lastUpdateMicros":1606859573133432,"kind":"shared:iapp:package-management-tasks:iapppackagemanagementtaskstate","selfLink":"https://localhost/mgmt/shared/iapp/package-management-tasks/719e3348-3279-43fc-ae3a-b6c4f86bbaa9"}
2020-12-01T21:52:53.149Z debug: [pid: 17894] [lib/bigIp.js]  {"packageFilePath":"/config/cloud/f5-appsvcs-3.20.0-3.noarch.rpm","operation":"INSTALL","id":"719e3348-3279-43fc-ae3a-b6c4f86bbaa9","status":"CREATED","userReference":{"link":"https://localhost/mgmt/shared/authz/users/admin"},"identityReferences":[{"link":"https://localhost/mgmt/shared/authz/users/admin"}],"ownerMachineId":"28f72e64-ff60-4486-b4c3-14398b8f5c8b","generation":1,"lastUpdateMicros":1606859573133432,"kind":"shared:iapp:package-management-tasks:iapppackagemanagementtaskstate","selfLink":"https://localhost/mgmt/shared/iapp/package-management-tasks/719e3348-3279-43fc-ae3a-b6c4f86bbaa9"}
2020-12-01T21:53:03.364Z debug: [pid: 17894] [lib/bigIp.js]  {"packageFilePath":"/config/cloud/f5-appsvcs-3.20.0-3.noarch.rpm","operation":"INSTALL","id":"719e3348-3279-43fc-ae3a-b6c4f86bbaa9","status":"FAILED","startTime":"2020-12-01T13:52:53.138-0800","endTime":"2020-12-01T13:52:53.153-0800","userReference":{"link":"https://localhost/mgmt/shared/authz/users/admin"},"identityReferences":[{"link":"https://localhost/mgmt/shared/authz/users/admin"}],"ownerMachineId":"28f72e64-ff60-4486-b4c3-14398b8f5c8b","generation":3,"lastUpdateMicros":1606859573153766,"kind":"shared:iapp:package-management-tasks:iapppackagemanagementtaskstate","selfLink":"https://localhost/mgmt/shared/iapp/package-management-tasks/719e3348-3279-43fc-ae3a-b6c4f86bbaa9"}
2020-12-01T21:53:13.687Z debug: [pid: 17894] [lib/bigIp.js] create localhost /shared/iapp/package-management-tasks {"operation":"INSTALL","packageFilePath":"/config/cloud/f5-appsvcs-3.20.0-3.noarch.rpm"}
2020-12-01T21:53:13.702Z debug: [pid: 17894] [lib/bigIp.js]  {"packageFilePath":"/config/cloud/f5-appsvcs-3.20.0-3.noarch.rpm","operation":"INSTALL","id":"8e732cdc-7c52-4abe-84a0-38a28545f892","status":"CREATED","userReference":{"link":"https://localhost/mgmt/shared/authz/users/admin"},"identityReferences":[{"link":"https://localhost/mgmt/shared/authz/users/admin"}],"ownerMachineId":"28f72e64-ff60-4486-b4c3-14398b8f5c8b","generation":1,"lastUpdateMicros":1606859593487728,"kind":"shared:iapp:package-management-tasks:iapppackagemanagementtaskstate","selfLink":"https://localhost/mgmt/shared/iapp/package-management-tasks/8e732cdc-7c52-4abe-84a0-38a28545f892"}
2020-12-01T21:53:13.714Z debug: [pid: 17894] [lib/bigIp.js]  {"packageFilePath":"/config/cloud/f5-appsvcs-3.20.0-3.noarch.rpm","operation":"INSTALL","id":"8e732cdc-7c52-4abe-84a0-38a28545f892","status":"STARTED","startTime":"2020-12-01T13:53:13.695-0800","userReference":{"link":"https://localhost/mgmt/shared/authz/users/admin"},"identityReferences":[{"link":"https://localhost/mgmt/shared/authz/users/admin"}],"ownerMachineId":"28f72e64-ff60-4486-b4c3-14398b8f5c8b","generation":2,"lastUpdateMicros":1606859593490157,"kind":"shared:iapp:package-management-tasks:iapppackagemanagementtaskstate","selfLink":"https://localhost/mgmt/shared/iapp/package-management-tasks/8e732cdc-7c52-4abe-84a0-38a28545f892"}
2020-12-01T21:53:23.722Z debug: [pid: 17894] [lib/bigIp.js]  {"packageFilePath":"/config/cloud/f5-appsvcs-3.20.0-3.noarch.rpm","operation":"INSTALL","id":"8e732cdc-7c52-4abe-84a0-38a28545f892","status":"FAILED","startTime":"2020-12-01T13:53:13.695-0800","endTime":"2020-12-01T13:53:13.706-0800","userReference":{"link":"https://localhost/mgmt/shared/authz/users/admin"},"identityReferences":[{"link":"https://localhost/mgmt/shared/authz/users/admin"}],"ownerMachineId":"28f72e64-ff60-4486-b4c3-14398b8f5c8b","generation":3,"lastUpdateMicros":1606859593501953,"kind":"shared:iapp:package-management-tasks:iapppackagemanagementtaskstate","selfLink":"https://localhost/mgmt/shared/iapp/package-management-tasks/8e732cdc-7c52-4abe-84a0-38a28545f892"}

JeffGiroux avatar Dec 01 '20 22:12 JeffGiroux

Had this in my GCP TF module too: https://github.com/memes/terraform-google-f5-bigip/issues/18

Download and install the RPMs to /var/config/rest/downloads/ to resolve

memes avatar Dec 01 '20 22:12 memes

This happens on 15.1.1 too. According to templates, only up to 15.1.0 has been validated so this makes sense. Please help push support for 15.1.1 and 15.1.2 by fixing the check for the rpm install location. Thx, Jeff

JeffGiroux avatar Dec 01 '20 22:12 JeffGiroux

Being tracked internally as ESECLDTPLT-2395

delgadillo22 avatar Dec 01 '20 23:12 delgadillo22

@shyawnkarim In case it helps with prioritizing, I have another customer that is also complaining about this. They cannot deploy 15.1.2 image but with 15.1.0.4 everything is okay.

@JeffGiroux if you have a workaround for this already can you let me know?

mikeoleary avatar Dec 08 '20 21:12 mikeoleary

The team had a sprint review this morning and this issue has been prioritized for next sprint, which starts December 22nd.

shyawnkarim avatar Dec 08 '20 21:12 shyawnkarim

@mikeoleary - taking the "HA across AZ" template example... https://github.com/F5Networks/f5-aws-cloudformation/tree/master/supported/failover/across-net/via-api/3nic/existing-stack/payg

*Note: not tested yet

Change RPM file download from this...

       "/config/cloud/f5-appsvcs-3.20.0-3.noarch.rpm": {
        "group": "root",
        "mode": "000755",
        "owner": "root",
        "source": "http://cdn.f5.com/product/cloudsolutions/f5-appsvcs-extension/v3.20.0/f5-appsvcs-3.20.0-3.noarch.rpm"
       },
       "/config/cloud/f5-cloud-failover-1.6.1-1.noarch.rpm": {
        "group": "root",
        "mode": "000755",
        "owner": "root",
        "source": "https://github.com/F5Networks/f5-cloud-failover-extension/releases/download/v1.6.1/f5-cloud-failover-1.6.1-1.noarch.rpm"
       },

To this...

       "/var/config/rest/downloads/f5-appsvcs-3.20.0-3.noarch.rpm": {
        "group": "root",
        "mode": "000755",
        "owner": "root",
        "source": "http://cdn.f5.com/product/cloudsolutions/f5-appsvcs-extension/v3.20.0/f5-appsvcs-3.20.0-3.noarch.rpm"
       },
       "/var/config/rest/downloads/f5-cloud-failover-1.6.1-1.noarch.rpm": {
        "group": "root",
        "mode": "000755",
        "owner": "root",
        "source": "https://github.com/F5Networks/f5-cloud-failover-extension/releases/download/v1.6.1/f5-cloud-failover-1.6.1-1.noarch.rpm"
       },

Then change the RPM install from this...

             "--install-ilx-package file:///config/cloud/f5-appsvcs-3.20.0-3.noarch.rpm --install-ilx-package file:///config/cloud/f5-cloud-failover-1.6.1-1.noarch.rpm",

To this (look for multiple occurrences if deploying HA)...

             "--install-ilx-package file:///var/config/rest/downloads/f5-appsvcs-3.20.0-3.noarch.rpm --install-ilx-package file:///var/config/rest/downloads/f5-cloud-failover-1.6.1-1.noarch.rpm",

Also, change the hash verification section too from this...

              "declare -a filesToVerify=(\"/config/cloud/f5-cloud-libs.tar.gz\" \"/config/cloud/f5-cloud-libs-aws.tar.gz\" \"/config/cloud/f5-appsvcs-3.20.0-3.noarch.rpm\"  \"/config/cloud/f5-cloud-failover-1.6.1-1.noarch.rpm\")",

To this...

              "declare -a filesToVerify=(\"/config/cloud/f5-cloud-libs.tar.gz\" \"/config/cloud/f5-cloud-libs-aws.tar.gz\" \"/var/config/rest/downloads/f5-appsvcs-3.20.0-3.noarch.rpm\"  \"/var/config/rest/downloads/f5-cloud-failover-1.6.1-1.noarch.rpm\")",

JeffGiroux avatar Dec 09 '20 00:12 JeffGiroux

Closing due to age. These legacy templates are now in maintenance mode and are being replaced by our next-generation templates available in the Cloud Templates 2.0 GitHub repo.

shyawnkarim avatar Nov 11 '22 23:11 shyawnkarim