Upgrading VMware Aria Suite Lifecycle | 8.12 to 8.14 | Deepdive |
Updated: Oct 23
Pre-Requisites
Take Snapshots
Review Release notes
Note : For VCF aware VMware Aria Suite Lifecycle environments , wait for respective PSPACK or Product Support Packs to be released
Important Logs
Pre-Upgrade & Upgrade Phase
/var/log/vmware/capengine/cap-non-lvm-update/worflow.log
/var/log/vmware/capenginecap-non-lvm-updateinstaller-<<timestamp>>.logPost Update Phase
/var/log/bootstrap/postupdate.log
/data/script.log
/var/log/vrlcm/vmware_vrlcm.logProcedure
We may use one of the repository methods to upgrade
Check Online
URL
CD-RomIn this example/demo , we will be using CD Rom method
Let's now deepdive and understand the uprgade procedure
Ignore the message which states that Suite Lifecycle is already upgraded. That will be removed in next version
Click on CD-Rom and then check for upgrades
It now reads the manifest and comes back stating that the uprgade is available
Give your conscent , agree that you have taken snapshot
Validations should go through
What do we check
Mandatory value check
Root password check
Disk space check on / filesystem
Requests in-progress check
VMware Aria Suite Lifecycle health check
VMware Identity Manager health checkNow click on Upgrade
Upgrade is now triggered
It would stay on 22% for a while as the rpm's are being verfified and staged into repo
Reference: /var/log/vmware/capengine/cap-non-lvm-update/worflow.logPrechecks
2023/10/20 11:09:42.479602 workflow_manager.go:84: Fetching metadata for workflow
2023/10/20 11:09:42.480148 workflow_manager.go:354: Updating instance status to Running
2023/10/20 11:09:42.480601 task_progress.go:24: Starting task update-precheck
*
*
2023/10/20 11:09:43.171868 task_progress.go:24: Reading appliance metadata complete.
2023/10/20 11:09:43.185705 workflow_manager.go:221: Task precheck_metadata completed
Staging
2023/10/20 11:09:43.186514 task_progress.go:24: Starting task stage
2023/10/20 11:09:43.250662 stage_plugin.go:124: Using stage directory: /storage/eb595996-31c9-432c-933e-fa354438df65/stage
2023/10/20 11:09:43.267429 progress.go:11: Staging update
*
*
2023/10/20 11:10:43.268446 task_progress.go:24: Updates staged successfully
2023/10/20 11:10:43.288182 workflow_manager.go:221: Task stage completed
Post staging , goes ahead with pre installation scripts
2023/10/20 11:10:43.288321 task_progress.go:24: Starting task ext-pre-install
2023/10/20 11:10:44.745400 task_progress.go:24: Finished executing pre install extension script
2023/10/20 11:10:44.757120 workflow_manager.go:221: Task ext-pre-install completed
Validates Installation
2023/10/20 11:10:44.757212 task_progress.go:24: Starting task validate-install
2023/10/20 11:10:50.381346 workflow_manager.go:221: Task validate-install completed
Moves to 55% when the rpm installation begins
Starts Installing RPM's
2023/10/20 11:10:50.381492 task_progress.go:24: Starting task install
2023/10/20 11:10:50.647395 progress.go:11: Installing RPMs
2023/10/20 11:10:50.658173 installer.go:44: Installing RPMs
2023/10/20 11:10:50.658307 task_progress.go:24: Installing RPMs
Remember, this is the stage where you can check install-<<timestamp>>.log as well. That's the lenghty log which installs all rpm's so will not describe it in detail
ufdio: 1 reads, 17154 total bytes in 0.000013 secs
D: ============== /storage/f5e77458-7b7f-49c8-a66f-d0bf3b0cca38/stage/apache-ant-1.10.12-2.ph3.noarch.rpm
D: loading keyring from pubkeys in /var/lib/rpm/pubkeys/*.key
D: couldn't find any keys in /var/lib/rpm/pubkeys/*.key
D: loading keyring from rpmdb
D: opening db environment /var/lib/rpm cdb:0x401
D: opening db index /var/lib/rpm/Packages 0x400 mode=0x0
*
*
*
*
D: closed db index /var/lib/rpm/Providename
D: closed db index /var/lib/rpm/Requirename
D: closed db index /var/lib/rpm/Group
D: closed db index /var/lib/rpm/Basenames
D: closed db index /var/lib/rpm/Name
D: closed db environment /var/lib/rpm
D: Exit status: 0
Once the installer log states Exit staus to 0 , it means that the upgrade part is now complete.
Workflow logs state that it has finished installing the RPM's
2023/10/20 11:15:27.232130 installer.go:32: Rebuilding RPM database
2023/10/20 11:15:27.875407 installer.go:57: Finished RPM installation
2023/10/20 11:15:27.875461 progress.go:11: Finished installing RPMs
2023/10/20 11:15:27.875762 task_progress.go:24: Finished installing RPMs
2023/10/20 11:15:27.946718 workflow_manager.go:221: Task install completed
Post this the post installation script starts
Reference: /var/log/vmware/capengine/cap-non-lvm-update/worflow.log
2023/10/20 13:10:40.708519 task_progress.go:24: Starting task ext-post-install
2023/10/20 13:10:41.324567 progress.go:11: Starting to execute post install extension script
2023/10/20 13:10:41.334470 command_exec.go:49: DEBUG running command: /bin/bash -c /var/tmp/f5e77458-7b7f-49c8-a66f-d0bf3b0cca38/post-install-script.sh 8.12.0.7 8.14.0.4 0
2023/10/20 13:10:41.334522 task_progress.go:24: Starting to execute post install extension scriptWhen it starts executing post install script, we need to check a 2 different logs, postupdate.log shown below to begin with
Reference: /var/log/bootstrap/postupdate.logDuring postupdate phase it checks
*** Begins Postupdate ***
*** RPM Checks ***
*** CAP User Creation ***
*** Disable Password Expiration ***
*** Update ulimit ***
*** Set Python ***
*** Postgres Configuration ***
*** Cleanup Inprogress Requests ***
*** Starts Services ***
2023-10-20 11:15:28 /etc/bootstrap/postupdate.d/25-start-services starting...
+ set -e
+ echo 'Reboot not required so starting all service.'
Reboot not required so starting all service.
+ systemctl daemon-reload
+ systemctl restart vpostgres
+ touch /var/log/bootstrap/reboot-required
+ systemctl restart vrlcm-server
+ systemctl restart blackstone-spring
+ cp -f /var/lib/vrlcm/dev-build-upgrade.sh /usr/local/bin/vrlcm-cli
+ chmod 700 /usr/local/bin/vrlcm-cli
+ cp -r /var/lib/vrlcm/nginx.conf /etc/nginx/
+ cp -r /var/lib/vrlcm/ssl.conf /etc/nginx/
+ systemctl reload nginx
+ rm -f /var/lib/vrlcm/SUCCESS
+ rm -rf /tmp/dlfRepo
+ [[ ! -f /etc/triggerLicenseUpdate ]]
+ touch /etc/triggerLicenseUpdate
+ [[ -f /var/log/vrlcm/status.txt ]]
+ rm -rf /var/log/vrlcm/status.txt_backup
+ cp -r /var/log/vrlcm/status.txt /var/log/vrlcm/status.txt_backup
+ rm -rf /var/log/vrlcm/status.txt
+ /var/lib/vrlcm/populate.sh
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
You would see the progress on postupdate will wait for sometime for populate.sh script to run
This. is the time you may check following logs
Reference : /data/script.log There are bunch of updates which go through in the database
checking all service status
checking services are running
checking dependent service status
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
^M 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0^M 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
*
*
delete sucess api count: 176:
delete failed api count: 0:
Starting postgres update script for vRSLCM specific tables
ALTER TABLE
ALTER TABLE
Disabling FIPS settings
ALTER TABLE
UPDATE 0After populate.sh script is complete, then go ahead and check postupdate.log
+ [[ -f /var/lib/vrlcm/SUCCESS ]]
+ echo 'Creating INPROGRESS to block UI from loading...'
Creating INPROGRESS to block UI from loading...
+ rm -rf /var/lib/vrlcm/SUCCESS
+ touch /var/lib/vrlcm/INPROGRESS
+ exit 0
2023-10-20 11:22:58 /etc/bootstrap/postupdate.d/25-start-services done, succeeded.
*** Service Startups Conclude ***
*** Cleaning-up PSPACK db's starts ***
*** Fill CAP product info ***
*** Updates upgrade status ***
*** blackstone upgrade starts ***
*** cleanup patch history ***
*** Enable fips mode ***
*** Create CAP update settings ***
*** Disable VAMI ***
*** Creates flag to reboot VA ***
*** Postupdate task is now complete ***Since postupdate task. is now complete , we shall now see the whole upgrade procedure completed by CAPENGINE and documented under workflow.log
2023/10/20 11:15:28.274925 command_exec.go:49: DEBUG running command: /bin/bash -c /var/tmp/eb595996-31c9-432c-933e-fa354438df65/post-install-script.sh 8.12.0.7 8.14.0.4 0
2023/10/20 11:22:59.237676
non_lvm_update_post_update_script_plugin.go:84: Post install extension script output Updating vami-sfcb.service - Removing vmtoolsd service dependency
already service is having restart policy
Finished installing version 8.14.0.4
2023/10/20 11:22:59.237734 progress.go:11: Finished executing post install extension script
2023/10/20 11:22:59.238148 task_progress.go:24: Finished executing post install extension script
2023/10/20 11:22:59.249143 workflow_manager.go:221: Task ext-post-install completed
2023/10/20 11:22:59.249404 task_progress.go:24: Starting task metadata_update
2023/10/20 11:22:59.870305 progress.go:11: Updating appliance metadata
2023/10/20 11:22:59.872860 task_progress.go:24: Updating appliance metadata
2023/10/20 11:23:00.044506 progress.go:11: Metadata update completed
2023/10/20 11:23:00.044835 task_progress.go:24: Metadata update completed
2023/10/20 11:23:00.059053 workflow_manager.go:221: Task metadata_update completed
2023/10/20 11:23:00.059324 task_progress.go:24: Starting task cleanup
2023/10/20 11:23:00.289545 progress.go:11: Removing stage path
2023/10/20 11:23:00.306077 task_progress.go:24: Removing stage path
2023/10/20 11:23:00.306810 cleanup.go:64: Removing update directory: /storage/eb595996-31c9-432c-933e-fa354438df65
2023/10/20 11:23:00.912667 cleanup.go:72: Successfully removed update location
2023/10/20 11:23:00.983155 workflow_manager.go:221: Task cleanup completed
2023/10/20 11:23:00.983189 workflow_manager.go:183: All tasks finished for workflow
2023/10/20 11:23:00.983208 workflow_manager.go:354: Updating instance status to Completed
This concludes the upgrade of VMware Aria Suite Lifecycle to version 8.14