Was working on an issue where vRA Upgrade from 7.5 to 7.6 failed due to an exception during the upgrade phase.
As we all know vRA upgrade has 3 phases
Pre-Update
Upgrade
Post-Update
Snippet from an environment where it worked
17/11/2020 14:16:30 [INFO] Update status: Running package installation
17/11/2020 14:16:30 [INFO] Running /opt/vmware/var/lib/vami/update/data/job/4/run_command
warning: /opt/vmware/var/lib/vami/update/data/package-pool/package-pool/vmware-tools-core-10.3.6-1.sles11.x86_64.rpm: Header V3 RSA/SHA1 signature: NOKEY, key ID 001e5cc9
Preparing packages for installation...
glibc-2.11.3-17.110.24.2
/sbin/ldconfig: /lib64/libhistory.so.6 is not a symbolic link
/sbin/ldconfig: /lib64/libreadline.so.6 is not a symbolic link
/sbin/ldconfig: /opt/vmware/support/lib64/libpng.so.3 is not a symbolic link
/sbin/ldconfig: /opt/vmware/support/lib64/libicudata.so.58 is not a symbolic link
/sbin/ldconfig: /opt/vmware/support/lib64/libcares.so.2 is not a symbolic link
/sbin/ldconfig: /opt/vmware/support/lib64/libicutest.so.58 is not a symbolic link
/sbin/ldconfig: /opt/vmware/support/lib64/libicui18n.so.58 is not a symbolic link
/sbin/ldconfig: /opt/vmware/support/lib64/libpng12.so.0 is not a symbolic link
/sbin/ldconfig: /opt/vmware/support/lib64/libcurl.so.4 is not a symbolic link
/sbin/ldconfig: /opt/vmware/support/lib64/libicuuc.so.58 is not a symbolic link
/sbin/ldconfig: /opt/vmware/support/lib64/libicutu.so.58 is not a symbolic link
/sbin/ldconfig: /opt/vmware/support/lib64/libicuio.so.58 is not a symbolic link
/sbin/ldconfig: /opt/vmware/support/lib64/libxml2.so.2 is not a symbolic link
vmware-tools-foundation-10.3.6-1.sles11
VMware-Postgres-osslibs-9.5.16.0-12585967
vmware-studio-appliance-config-3.0.0.6-190321154832
libopenssl1_0_1-1.0.2r-1.vmw.sles.12677144
vmware-tools-libraries-nox-10.3.6-1.sles11
libxml2-2.9.9-1.vmw.12267021
/sbin/ldconfig: /lib64/libhistory.so.6 is not a symbolic link
Snippet from an environment where it failed
10/12/2020 14:43:34 [INFO] Update status: Running package installation
10/12/2020 14:43:34 [INFO] Running /opt/vmware/var/lib/vami/update/data/job/3/run_command
warning: /opt/vmware/var/lib/vami/update/data/package-pool/package-pool/vmware-tools-core-10.3.6-1.sles11.x86_64.rpm: Header V3 RSA/SHA1 signature: NOKEY, key ID 001e5cc9
Preparing packages for installation...
glibc-2.11.3-17.110.24.2
/usr/sbin/glibc_post_upgrade: While trying to execute /sbin/ldconfig child terminated abnormally
error: %post(glibc-2.11.3-17.110.24.2.x86_64) scriptlet failed, exit status 112
vmware-tools-foundation-10.3.6-1.sles11
VMware-Postgres-osslibs-9.5.16.0-12585967
vmware-studio-appliance-config-3.0.0.6-190321154832
libopenssl1_0_1-1.0.2r-1.vmw.sles.12677144
vmware-tools-libraries-nox-10.3.6-1.sles11
libxml2-2.9.9-1.vmw.12267021
/var/tmp/rpm-tmp.34745: line 1: 5830 Aborted /sbin/ldconfig
error: %post(libxml2-2.9.9-1.vmw.12267021.x86_64) scriptlet failed, exit status 134
tomcat-template-8.5.37.0-1.vmw.11804273
VMware-Postgres-osslibs-server-9.5.16.0-12585967
krb5-1.6.3-133.49.113.10.1
/var/tmp/rpm-tmp.34745: line 1: 5835 Aborted /sbin/ldconfig
VMware-Postgres-libs-9.5.16.0-12585967
vmware-studio-vami-tools-3.0.0.6-12673961
vmware-studio-init-3.0.0.6-190321154832
Shutting down vami-sfcbd: done.
vmware-studio-vami-cimom-3.0.0.6-12673961
Detected x86_64 arch
Updating /etc/pam.d/vami-sfcb for x84_64
There were lots of ldconfig errors in /var/log/messages of appliance
grep -i ldconfig /var/log/messages
Keeping these errors in mind an upgrade will only succeed if we clear these out
Here's the plan which was followed to clear ldconfig exceptions or cache corruption
All thanks to Nikolay Boshnakov for sharing solution with the limited amount of data we collected at the time of an upgrade
Remediation Plan
Execute below command
strace ldconfig
This will fail with an abort exception
Browse to the following path
cd /var/cache/ldconfig/
You would see a file called aux-cache
Delete this aux-cache file
rm -rf aux-cache
Execute the below command again which will recreate the aux-cache file again
strace ldconfig
Monitor /var/log/messages , you should not see any ldconfig exceptions anyomore
Comments