Workaround instructions to address CVE-2021-44228 in vRealize Automation 7.6

Updated: Apr 7



 

Source of Truth : VMware KB: 87121 ( https://kb.vmware.com/s/article/87121 )


Note: I am trying to share screenshots / outputs from the steps mentioned in the knowledge base : 87121 by implementing them in my lab


You may download PDF version here



Workaround instructions to address CVE-2021-44228 in vRealize Automation 7.6.
.pdf
Download PDF • 1.73MB


 

Purpose


Notice: On December 14, 2021 the Apache Software Foundation notified the community that their initial guidance for CVE-2021-44228 workarounds was not sufficient.


We believe the instructions in this article to be an effective mitigation for CVE-2021-44228 and CVE-2021-45046, but in the best interest of our customers we must assume this workaround may not adequately address all attack vectors.



 

Resolution

The workarounds described in this document are meant to be a temporary solution only. Upgrades documented in the aforementioned advisory should be applied to remediate CVE-2021-44228 when available.



 

Workaround


Prerequisites

  • Backup the vRA appliance nodes

  • Snapshots ( My vRealize Automation environment is managed by vRSLCM so i'll use it to take snapshots )








  • Snapshot is now complete

  • Let's now move to the actual procedure of workaround implementation


Procedure


Note: Steps #2 - #3 apply to the embedded vRealize Orchestrator 7.6 instance. For external instances, see Workaround instructions to address CVE-2021-44228 in vRealize Orchestrator 7.

SSH into each vRA appliance node and run the following steps:


Step:1

Stop the vco-configurator service on each vRA node


Step:2

Run the following command to update vRO configuration on each vRA node:




base64 -d <<< "bG9nX21lc3NhZ2VfbG9nNGooKSB7CiAgZWNobyAiWyQoZGF0ZSAtLXV0YyAiKyVGVCVULiUzTloiKV0gJDEiIHwgdGVlIC1hIC92YXIvbG9nL3Ztd2FyZS92Y28vYXBwLXNlcnZlci92Y29fbG9nNGpfY3ZlLmxvZwp9Cgpsb2dfZXJyb3JfbG9nNGooKSB7CiAgbG9nX21lc3NhZ2VfbG9nNGogIkVSUk9SOiAkMSIKICBleGl0IDEKfQoKc2V0X2phdmFfb3B0KCkgewogIGxvY2FsIGZpbGU9IiQxIgogIGxvY2FsIGJha3VwX3N1ZmZpeD0iJChkYXRlIC0tdXRjICsiJVklbSVkJUglTSIpIgogIAoKICBpZiBncmVwIC1xICdEbG9nNGoyLmZvcm1hdE1zZ05vTG9va3Vwcz10cnVlJyAkZmlsZQogIHRoZW4gCiAgICBsb2dfbWVzc2FnZV9sb2c0aiAiVGhlIGphdmEgcHJvcGVydHkgbG9nNGoyLmZvcm1hdE1zZ05vTG9va3Vwcz10cnVlIGlzIGFscmVhZHkgc2V0IGluICRmaWxlLiIKICBlbHNlCiAgICBsb2dfbWVzc2FnZV9sb2c0aiAiQ3JlYXRpbmcgYmFjayB1cCBmb3Igc2V0ZW52IGZpbGUgaW4gJGZpbGUuJGJha3VwX3N1ZmZpeCIKICAgIGNwIC1mICIkZmlsZSIgIiRmaWxlLiRiYWt1cF9zdWZmaXgiCiAgICBsb2dfbWVzc2FnZV9sb2c0aiAiQWRkaW5nIC1EbG9nNGoyLmZvcm1hdE1zZ05vTG9va3Vwcz10cnVlIHRvIEpWTV9PUFRTIGluICRmaWxlIgogICAgcmVzPSQoYXdrICdGTlI9PU5SeyBpZiAoL15KVk1fT1BUUz0vKSBwPU5SOyBuZXh0fSAxOyBGTlI9PXB7IHByaW50ICJKVk1fT1BUUz1cIiRKVk1fT1BUUyAtRGxvZzRqMi5mb3JtYXRNc2dOb0xvb2t1cHM9dHJ1ZVwiIiB9JyAkZmlsZSAkZmlsZSkgfHwgbG9nX2Vycm9yX2xvZzRqICJGYWlsZWQgdG8gZWRpdCAkZmlsZSIKICAgIGVjaG8gIiRyZXMiID4gJGZpbGUKICBmaQp9Cgp1cGRhdGVfdnJvX3RvbWNhdF9zdGFydCgpIHsKICBsb2NhbCBmaWxlPSIkMSIKCiAgaWYgZ3JlcCAtcSAncGF0Y2hfYWxsX3BsdWdpbnNfdjIgPicgJGZpbGUKICB0aGVuIAogICAgbG9nX21lc3NhZ2VfbG9nNGogIk5vdGhpbmcgdG8gZG8uIEtCIGFscmVhZHkgYXBwbGllZCEiCiAgZWxzZQogICAgbG9jYWwgcmVzCiAgICBsb2NhbCBiYWt1cF9zdWZmaXg9IiQoZGF0ZSAtLXV0YyArIiVZJW0lZCVIJU0iKSIKICAgIGxvZ19tZXNzYWdlX2xvZzRqICJDcmVhdGluZyBiYWNrIHVwIGZvciB0b21jYXQgc3RhcnR1cCBjb25maWcgaW4gJGZpbGUuJGJha3VwX3N1ZmZpeCIKICAgIGNwIC1mICIkZmlsZSIgIiRmaWxlLiRiYWt1cF9zdWZmaXgiCgogICAgbG9nX21lc3NhZ2VfbG9nNGogIk1vZGlmeWluZyB2Uk8gdG9tY2F0IHN0YXJ0dXAgY29uZmlnIC0gJGZpbGUiCgogICAgaWYgZ3JlcCAtcSAnI2xvZzRqX2N2ZV93b3JrYXJvdW5kJyAkZmlsZQogICAgdGhlbgogICAgICBzZWQgLWkgJ3MvI2xvZzRqX2N2ZV93b3JrYXJvdW5kL1xuI2xvZzRqX2N2ZV93b3JrYXJvdW5kL2cnICRmaWxlCiAgICAgIHNlZCAtaSAnL2xvZ19tZXNzYWdlX2xvZzRqMl9jdmUgIlBhdGNoaW5nIGRvbmUuIiQve247cy8uKi99XG4jbG9nNGpfY3ZlX3dvcmthcm91bmRfZW5kL30nICRmaWxlCiAgICAgIHNlZCAtaSAnL3ZSTyBzZXJ2ZXIgc2VydmljZSBkaWQgbm90IHN0YXJ0IHdpdGhpbiB0aGUgZXhwZWN0ZWQgcGVyaW9kLiokL3tuO247bjtzLy4qL31cbiNsb2c0al9jdmVfd29ya2Fyb3VuZF9lbmQvfScgJGZpbGUKICAgICAgc2VkIC1pICcvI2xvZzRqX2N2ZV93b3JrYXJvdW5kLywvI2xvZzRqX2N2ZV93b3JrYXJvdW5kX2VuZC9jXGRlbGV0ZV9tYXJrZXJcJyAkZmlsZQogICAgICBwZXJsIC1pIC0wcGUgJ3MvKC4qKVxuLipkZWxldGVfbWFya2VyXG4vXDEvZzsnICRmaWxlCgogICAgICBzZWQgLWkgJy9sb2c0al9jdmVfd29ya2Fyb3VuZCA+L2QnICRmaWxlIAogICAgICBzZWQgLWkgJy9wYXRjaF9hbGxfcGx1Z2lucyA+L2QnICRmaWxlCiAgICBmaQogICAgc2VkIC1pICcvYmFzZTY0IC1kIDw8PC9kJyAkZmlsZSAKCiAgICBpZiAhIGdyZXAgLXFFICdhY3Rpb249InN0YXJ0InxTdGFydGluZyB0Y1NlcnZlcicgIiRmaWxlIgogICAgdGhlbgogICAgICBsb2dfZXJyb3JfbG9nNGogIlVuYWJsZSB0byBhcHBseSBwYXRjaDogVW5leHBlY3RlZCBmb3JtYXQgb2YgdlJPIHRvbWNhdCBzdGFydHVwIGNvbmZpZyAtICRmaWxlLiIKICAgICAgZXhpdCAxCiAgICBmaQoKICAgIGxvY2FsIHV0aWw9IiQoZGlybmFtZSAiJGZpbGUiKS9jdmVfdXRpbC5zaCIKICAgIGJhc2U2NCAtZCA8PDwgIkkyeHZaelJxWDJOMlpWOTNiM0pyWVhKdmRXNWtDbXh2WjE5dFpYTnpZV2RsWDJ4dlp6UnFNbDlqZG1WZmRqSW9LU0I3Q2dsbFkyaHZJQ0piSkNoa1lYUmxJQzB0ZFhSaklDSXJKVVpVSlZRdUpUTk9XaUlwWFNBa01TSUtmUW9LQ25CaGRHTm9YM0JzZFdkcGJsOTJNaWdwSUhzS0NXeHZZMkZzSUhCc2RXZHBibDl3WVhSb1BTSWtNU0lLQ1d4dlkyRnNJSEJzZFdkcGJsOXVZVzFsUFNJa0tHSmhjMlZ1WVcxbElDSWtjR3gxWjJsdVgzQmhkR2dpS1NJS0NXeHZZMkZzSUhSbGJYQmZjR0YwYUQwaUwzUnRjQzhrY0d4MVoybHVYMjVoYldVaUNnbHNiMk5oYkNCaVlXTnJkWEJ6WDNCaGRHZzlJaTkxYzNJdmJHbGlMM1pqYnk5aVlXTnJkWEJ6SWdvS0NXMXJaR2x5SUMxd0lDSWtZbUZqYTNWd2MxOXdZWFJvSWdvS0NYSnRJQzF5WmlBaUpIUmxiWEJmY0dGMGFDSUtDWFZ1ZW1sd0lDMXhJQ0lrY0d4MVoybHVYM0JoZEdnaUlDMWtJQ0lrZEdWdGNGOXdZWFJvSWlCOGZDQmxlR2wwSURFS0Nna2pJRU5vWldOcklHbG1JSFJvWlhKbElHbHpJR0VnYm1WbFpDQjBieUIxY0dSaGRHVWdkR2hsSUhCc2RXZHBiZ29KYVdZZ1ptbHVaQ0FpSkhSbGJYQmZjR0YwYUNJZ0xYaGtaWFlnTFhSNWNHVWdaaUF0Ym1GdFpTQW5iRzluTkdvdFkyOXlaUzB5S21waGNpY2dMV1Y0WldNZ0wzVnpjaTlpYVc0dmVtbHdJQzF6WmlBaWUzMGlJRnc3SUh3Z1ozSmxjQ0J2Y21jdllYQmhZMmhsTDJ4dloyZHBibWN2Ykc5bk5Hb3ZZMjl5WlM5c2IyOXJkWEF2U201a2FVeHZiMnQxY0M1amJHRnpjenNLQ1hSb1pXNEtDUWxzYjJkZmJXVnpjMkZuWlY5c2IyYzBhakpmWTNabFgzWXlJQ0pOYjJScFpubHBibWNnY0d4MVoybHVPaUFrY0d4MVoybHVYMjVoYldVaUNna0piWFlnSWlSd2JIVm5hVzVmY0dGMGFDSWdJaVJpWVdOcmRYQnpYM0JoZEdnaUNna0pabWx1WkNBaUpIUmxiWEJmY0dGMGFDSWdMWGhrWlhZZ0xYUjVjR1VnWmlBdGJtRnRaU0FuYkc5bk5Hb3RZMjl5WlMweUttcGhjaWNnTFdWNFpXTWdjMmdnTFdNZ0p5OTFjM0l2WW1sdUwzcHBjQ0F0Y1NBdFpDQWllMzBpSUc5eVp5OWhjR0ZqYUdVdmJHOW5aMmx1Wnk5c2IyYzBhaTlqYjNKbEwyeHZiMnQxY0M5S2JtUnBURzl2YTNWd0xtTnNZWE56T3lCMGIzVmphQ0F0ZENBeU1ESXhNREV3TVRBd01EQWdJbnQ5SWljZ1hEc0tDUWtvWTJRZ0lpUjBaVzF3WDNCaGRHZ2lJRHNnZW1sd0lDMXhJQzF5SUMxWUlDMUVJQ0lrY0d4MVoybHVYM0JoZEdnaUlDb3BJSHg4SUdWNGFYUWdNUW9LQ1FraklFWnBlQ0J5YVdkb2RITUtDUWxqYUc5M2JpQjJZMjg2ZG1OdklDSWtjR3gxWjJsdVgzQmhkR2dpQ2drSlkyaHRiMlFnTURZME5DQWlKSEJzZFdkcGJsOXdZWFJvSWdvSkNXeHZaMTl0WlhOellXZGxYMnh2WnpScU1sOWpkbVZmZGpJZ0lsTjFZMk5sYzNObWRXeHNlU0J3WVhSamFHVmtJSEJzZFdkcGJqb2dKSEJzZFdkcGJsOXVZVzFsSWdvSlpXeHpaUW9KQ1d4dloxOXRaWE56WVdkbFgyeHZaelJxTWw5amRtVmZkaklnSWs1dmRHaHBibWNnZEc4Z1pHOGdabTl5SUhCc2RXZHBiam9nSkhCc2RXZHBibDl1WVcxbElnb0pabWtLQ1hKdElDMXlaaUFpSkhSbGJYQmZjR0YwYUNJS2ZRb0tjR0YwWTJoZllXeHNYM0JzZFdkcGJuTmZkaklvS1NCN0NnbG1iM0lnWm1sc1pTQnBiaUF2ZFhOeUwyeHBZaTkyWTI4dllYQndMWE5sY25abGNpOXdiSFZuYVc1ekx5b3VaR0Z5Q2dsa2J3b0pDWEJoZEdOb1gzQnNkV2RwYmw5Mk1pQWlKR1pwYkdVaUlIeDhJR3h2WjE5dFpYTnpZV2RsWDJ4dlp6UnFNbDlqZG1WZmRqSWdJa1ZTVWs5U09pQkdZV2xzWldRZ2RHOGdjR0YwWTJnZ2NHeDFaMmx1T2lBa0tHSmhjMlZ1WVcxbElDUm1hV3hsS1NJS0NXUnZibVVLQ2dsc2IyZGZiV1Z6YzJGblpWOXNiMmMwYWpKZlkzWmxYM1l5SUNKUVlYUmphR2x1WnlCa2IyNWxMaUlLZlFvPSIgPiAiJHV0aWwiCgogICAgc2VkIC1pICdzLGV2YWwgZXhlYywnInNvdXJjZSAkdXRpbCInXG4mLCcgIiRmaWxlIgogICAgZ3JlcCAtcSAic291cmNlICR1dGlsIiAiJGZpbGUiIHx8IHsgbG9nX2Vycm9yX2xvZzRqICJGYWlsZWQgdG8gZWRpdCAkZmlsZS4gQmFja3VwIGNhbiBiZSBmb3VuZCBpbiAkZmlsZS4kYmFrdXBfc3VmZml4IjsgZXhpdCAxOyB9CgogICAgc2VkIC1yIC1pICcvYWN0aW9uPSJzdGFydCJ8U3RhcnRpbmcgdGNTZXJ2ZXIvYSBcXHRcdFx0cGF0Y2hfYWxsX3BsdWdpbnNfdjIgPj4gL3Zhci9sb2cvdm13YXJlL3Zjby9hcHAtc2VydmVyL3Zjb19sb2c0al9jdmUubG9nJyAkZmlsZQogICAgZ3JlcCAtcSAncGF0Y2hfYWxsX3BsdWdpbnNfdjIgPicgIiRmaWxlIiB8fCB7IGxvZ19lcnJvcl9sb2c0aiAiRmFpbGVkIHRvIGVkaXQgJGZpbGUuIEJhY2t1cCBjYW4gYmUgZm91bmQgaW4gJGZpbGUuJGJha3VwX3N1ZmZpeCI7IGV4aXQgMTsgfQoKICAgIGxvZ19tZXNzYWdlX2xvZzRqICJTdWNjZXNzZnVsbHkgbW9kaWZpZWQgdGhlIHZSTyB0b21jYXQgc3RhcnR1cCBjb25maWcgLSAkZmlsZSIKICBmaQp9CgoKKHNldF9qYXZhX29wdCAiL3Vzci9saWIvdmNvL2NvbmZpZ3VyYXRpb24vYmluL3NldGVudi5zaCIgJiYgc2V0X2phdmFfb3B0ICIvdXNyL2xpYi92Y28vYXBwLXNlcnZlci9iaW4vc2V0ZW52LnNoIiAmJiB1cGRhdGVfdnJvX3RvbWNhdF9zdGFydCAiL3Zhci9saWIvdmNvL2FwcC1zZXJ2ZXIvYmluL2luaXQuZC5zaCIpIHx8IGxvZ19lcnJvcl9sb2c0aiAiRmFpbGVkIHRvIGFwcGx5IHRoZSBsb2c0aiBDVkUgd29ya2Fyb3VuZCBmb3IgdlJPLiBGb3IgbW9yZSBkZXRhaWxzIHNlZSAvdmFyL2xvZy92bXdhcmUvdmNvL2FwcC1zZXJ2ZXIvdmNvX2xvZzRqX2N2ZS5sb2cuIg==" | sh -

Note: My environment has a standalone vRA so i'll be running this command only on one node. If there are more than one node then this has to be run on all of the nodes





Output


[2021-12-18T01:26:31.436Z] Creating back up for setenv file in /usr/lib/vco/configuration/bin/setenv.sh.202112180126 
[2021-12-18T01:26:31.470Z] Adding -Dlog4j2.formatMsgNoLookups=true to JVM_OPTS in /usr/lib/vco/configuration/bin/setenv.sh 
[2021-12-18T01:26:31.493Z] Creating back up for setenv file in /usr/lib/vco/app-server/bin/setenv.sh.202112180126 
[2021-12-18T01:26:31.505Z] Adding -Dlog4j2.formatMsgNoLookups=true to JVM_OPTS in /usr/lib/vco/app-server/bin/setenv.sh 
[2021-12-18T01:26:31.525Z] Creating back up for tomcat startup config in /var/lib/vco/app-server/bin/init.d.sh.202112180126 
[2021-12-18T01:26:31.535Z] Modifying vRO tomcat startup config - /var/lib/vco/app-server/bin/init.d.sh 
[2021-12-18T01:26:31.577Z] Successfully modified the vRO tomcat startup config - /var/lib/vco/app-server/bin/init.d.sh



Step:3

Run the following (on any single vRA Node) to update the vRO Control Center (not applicable to versions 7.2 and 7.3)



/usr/lib/vco/tools/configuration-cli/bin/vro-configure-inner.sh controlcenter-update



Output


Will vary if you have a distributed environment


[master] svra:~ # /usr/lib/vco/tools/configuration-cli/bin/vro-configure-inner.sh controlcenter-update 
Orchestrator's root folder: /var/lib/vco 
Orchestrator Configuration Tool. Version: 7.6.0.12923317 Build: 12923317 
Start 'controlcenter-update' command. 
Dec 18, 2021 1:29:23 AM org.apache.tomcat.jdbc.pool.ConnectionPool checkPoolConfiguration 
WARNING: initialSize is larger than maxActive, setting initialSize to: 4 
Dec 18, 2021 1:29:23 AM org.apache.tomcat.jdbc.pool.ConnectionPool checkPoolConfiguration 
WARNING: minIdle is larger than maxActive, setting minIdle to: 4 
Dec 18, 2021 1:29:23 AM org.apache.tomcat.jdbc.pool.ConnectionPool checkPoolConfiguration 
WARNING: maxIdle is smaller than minIdle, setting maxIdle to: 4 
'controlcenter-update' command finished successfully. 
The command does not need database configuration update.


Step:4

Run the following command to update vRA and vIDM configuration on each vRA Node



base64 -d <<< "bG9nX21lc3NhZ2UoKSB7CiAgZWNobyAiWyQoZGF0ZSAtLXV0YyAiKyVGVCVULiUzTloiKV0gJDEiIHwgdGVlIC1hICAgL3Zhci9sb2cvdm13YXJlL3ZjYWMvdmNhY19sb2c0al9jdmUubG9nCn0KCmxvZ19lcnJvcigpIHsKICBsb2dfbWVzc2FnZSAiRVJST1I6ICQxIgogIGV4aXQgMQp9CgpzZXRfamF2YV9vcHQoKSB7CiAgbG9jYWwgZmlsZT0iJDEiCgogIGlmIGdyZXAgLXEgJ0Rsb2c0ajIuZm9ybWF0TXNnTm9Mb29rdXBzPXRydWUnICRmaWxlCiAgdGhlbiAKICAgIGxvZ19tZXNzYWdlICJUaGUgamF2YSBwcm9wZXJ0eSBsb2c0ajIuZm9ybWF0TXNnTm9Mb29rdXBzPXRydWUgaXMgYWxyZWFkeSBzZXQgaW4gJGZpbGUuIgogIGVsc2UKICAgIGxvZ19tZXNzYWdlICJBZGRpbmcgLURsb2c0ajIuZm9ybWF0TXNnTm9Mb29rdXBzPXRydWUgdG8gVkNBQ19PUFRTIGluICRmaWxlIgogICAgZWNobyAnVkNBQ19PUFRTPSIkVkNBQ19PUFRTIC1EbG9nNGoyLmZvcm1hdE1zZ05vTG9va3Vwcz10cnVlIicgPj4gJGZpbGUgfHwgbG9nX2Vycm9yICJGYWlsZWQgdG8gZWRpdCAkZmlsZSIKICBmaQp9CgpkZWxldGVfam5kaV9jbGFzcygpIHsKICBsb2dfbWVzc2FnZSAiRGVsZXRpbmcgYWxsIEpuZGlMb29rdXAuY2xhc3MgZmlsZXMgZm91bmQgZm9yIGxvZzRqIDIueCB2ZXJzaW9ucyIKICBmaW5kIC8gLXhkZXYgLXR5cGUgZiAtbmFtZSAnbG9nNGotY29yZS0yKmphcicgLWV4ZWMgL3Vzci9iaW4vemlwIC1xIC1kICJ7fSIgb3JnL2FwYWNoZS9sb2dnaW5nL2xvZzRqL2NvcmUvbG9va3VwL0puZGlMb29rdXAuY2xhc3MgXDsgfCB0ZWUgLWEgL3Zhci9sb2cvdm13YXJlL3ZjYWMvdmNhY19sb2c0al9jdmUubG9nCn0KCihzZXRfamF2YV9vcHQgIi9ldGMvdmNhYy9zZXRlbnYtdXNlciIgJiYgZGVsZXRlX2puZGlfY2xhc3MgKSB8fCBsb2dfZXJyb3IgIkZhaWxlZCB0byBhcHBseSB0aGUgbG9nNGogQ1ZFIHdvcmthcm91bmQgZm9yIHZSQS4gRm9yIG1vcmUgZGV0YWlscyBzZWUgL3Zhci9sb2cvdm13YXJlL3ZjYWMvdmNhY19sb2c0al9jdmUubG9nLiI=" | sh -



Note: Warnings of the type "zip error: Nothing to do!..." indicate there is no need for patching the specified binary.


Output



[master] svra:~ # base64 -d <<< "bG9nX21lc3NhZ2UoKSB7CiAgZWNobyAiWyQoZGF0ZSAtLXV0YyAiKyVGVCVULiUzTloiKV0gJDEiIHwgdGVlIC1hICAgL3Zhci9sb2cvdm13YXJlL3ZjYWMvdmNhY19sb2c0al9jdmUubG9nCn0KCmxvZ19lcnJvcigpIHsKICBsb2dfbWVzc2FnZSAiRVJST1I6ICQxIgogIGV4aXQgMQp9CgpzZXRfamF2YV9vcHQoKSB7CiAgbG9jYWwgZmlsZT0iJDEiCgogIGlmIGdyZXAgLXEgJ0Rsb2c0ajIuZm9ybWF0TXNnTm9Mb29rdXBzPXRydWUnICRmaWxlCiAgdGhlbiAKICAgIGxvZ19tZXNzYWdlICJUaGUgamF2YSBwcm9wZXJ0eSBsb2c0ajIuZm9ybWF0TXNnTm9Mb29rdXBzPXRydWUgaXMgYWxyZWFkeSBzZXQgaW4gJGZpbGUuIgogIGVsc2UKICAgIGxvZ19tZXNzYWdlICJBZGRpbmcgLURsb2c0ajIuZm9ybWF0TXNnTm9Mb29rdXBzPXRydWUgdG8gVkNBQ19PUFRTIGluICRmaWxlIgogICAgZWNobyAnVkNBQ19PUFRTPSIkVkNBQ19PUFRTIC1EbG9nNGoyLmZvcm1hdE1zZ05vTG9va3Vwcz10cnVlIicgPj4gJGZpbGUgfHwgbG9nX2Vycm9yICJGYWlsZWQgdG8gZWRpdCAkZmlsZSIKICBmaQp9CgpkZWxldGVfam5kaV9jbGFzcygpIHsKICBsb2dfbWVzc2FnZSAiRGVsZXRpbmcgYWxsIEpuZGlMb29rdXAuY2xhc3MgZmlsZXMgZm91bmQgZm9yIGxvZzRqIDIueCB2ZXJzaW9ucyIKICBmaW5kIC8gLXhkZXYgLXR5cGUgZiAtbmFtZSAnbG9nNGotY29yZS0yKmphcicgLWV4ZWMgL3Vzci9iaW4vemlwIC1xIC1kICJ7fSIgb3JnL2FwYWNoZS9sb2dnaW5nL2xvZzRqL2NvcmUvbG9va3VwL0puZGlMb29rdXAuY2xhc3MgXDsgfCB0ZWUgLWEgL3Zhci9sb2cvdm13YXJlL3ZjYWMvdmNhY19sb2c0al9jdmUubG9nCn0KCihzZXRfamF2YV9vcHQgIi9ldGMvdmNhYy9zZXRlbnYtdXNlciIgJiYgZGVsZXRlX2puZGlfY2xhc3MgKSB8fCBsb2dfZXJyb3IgIkZhaWxlZCB0byBhcHBseSB0aGUgbG9nNGogQ1ZFIHdvcmthcm91bmQgZm9yIHZSQS4gRm9yIG1vcmUgZGV0YWlscyBzZWUgL3Zhci9sb2cvdm13YXJlL3ZjYWMvdmNhY19sb2c0al9jdmUubG9nLiI=" | sh - 
[2021-12-18T01:31:25.078Z] Adding -Dlog4j2.formatMsgNoLookups=true to VCAC_OPTS in /etc/vcac/setenv-user 
[2021-12-18T01:31:25.082Z] Deleting all JndiLookup.class files found for log4j 2.x versions 
zip error: Nothing to do! (/usr/lib/vcac/server/webapps/notification-service/WEB-INF/lib/log4j-core-2.11.2.jar) 
zip error: Nothing to do! (/usr/lib/vcac/server/webapps/approval-service/WEB-INF/lib/log4j-core-2.11.2.jar) 
zip error: Nothing to do! (/usr/lib/vcac/server/webapps/network-service/WEB-INF/lib/log4j-core-2.11.2.jar) 
zip error: Nothing to do! (/usr/lib/vcac/server/webapps/o11n-gateway-service/WEB-INF/lib/log4j-core-2.11.2.jar) 
zip error: Nothing to do! (/usr/lib/vcac/server/webapps/iaas-proxy-provider/WEB-INF/lib/log4j-core-2.11.2.jar) 
zip error: Nothing to do! (/usr/lib/vcac/server/webapps/console-proxy-service/WEB-INF/lib/log4j-core-2.11.2.jar) 
zip error: Nothing to do! (/usr/lib/vcac/server/webapps/vcac/WEB-INF/lib/log4j-core-2.11.2.jar) 
zip error: Nothing to do! (/usr/lib/vcac/server/webapps/branding-service/WEB-INF/lib/log4j-core-2.11.2.jar) 
zip error: Nothing to do! (/usr/lib/vcac/server/webapps/portal-service/WEB-INF/lib/log4j-core-2.11.2.jar) 
zip error: Nothing to do! (/usr/lib/vcac/server/webapps/workitem-service/WEB-INF/lib/log4j-core-2.11.2.jar) 
zip error: Nothing to do! (/usr/lib/vcac/server/webapps/healthbroker-proxy-server/WEB-INF/lib/log4j-core-2.11.2.jar) 
zip error: Nothing to do! (/usr/lib/vcac/server/webapps/event-broker-service/WEB-INF/lib/log4j-core-2.11.2.jar) 
zip error: Nothing to do! (/usr/lib/vcac/server/webapps/placement-service/WEB-INF/lib/log4j-core-2.11.2.jar) 
zip error: Nothing to do! (/usr/lib/vcac/server/webapps/reservation-service/WEB-INF/lib/log4j-core-2.11.2.jar) 
zip error: Nothing to do! (/usr/lib/vcac/server/webapps/endpoint-configuration-service/WEB-INF/lib/log4j-core-2.11.2.jar) 
zip error: Nothing to do! (/usr/lib/vcac/server/webapps/config-management-service/WEB-INF/lib/log4j-core-2.11.2.jar) 
zip error: Nothing to do! (/usr/lib/vcac/server/webapps/catalog-service/WEB-INF/lib/log4j-core-2.11.2.jar) 
zip error: Nothing to do! (/usr/lib/vcac/server/webapps/properties-service/WEB-INF/lib/log4j-core-2.11.2.jar) 
zip error: Nothing to do! (/usr/lib/vcac/server/webapps/forms-service/WEB-INF/lib/log4j-core-2.11.2.jar) 
zip error: Nothing to do! (/usr/lib/vcac/server/webapps/content-management-service/WEB-INF/lib/log4j-core-2.11.2.jar) 
zip error: Nothing to do! (/usr/lib/vcac/server/webapps/identity/WEB-INF/lib/log4j-core-2.11.2.jar) 
zip error: Nothing to do! (/usr/lib/vcac/server/webapps/software-service/WEB-INF/lib/log4j-core-2.11.2.jar) 
zip error: Nothing to do! (/usr/lib/vcac/server/webapps/ipam-service/WEB-INF/lib/log4j-core-2.11.2.jar) 
zip error: Nothing to do! (/usr/lib/vcac/server/webapps/advanced-designer-service/WEB-INF/lib/log4j-core-2.11.2.jar) 
zip error: Nothing to do! (/usr/lib/vcac/server/webapps/composition-service/WEB-INF/lib/log4j-core-2.11.2.jar) 
zip error: Nothing to do! (/usr/lib/vcac/server/webapps/management-service/WEB-INF/lib/log4j-core-2.11.2.jar) 
zip error: Nothing to do! (/usr/lib/vcac/tools/config/repo/log4j-core-2.11.2.jar)

Step:5


Restart the services on each vRA Node



service elasticsearch restart 
service vco-server status | grep PID && service vco-server restart 
service vco-configurator start 
service vcac-server restart





Note:

Below command can be used to stop and start all the services systematically. Not mentioned in the knowledge base article but can be followed


vcac-vami service-manage stop vco-configurator vco-server vcac-server horizon-workspace hzn-dots elasticsearch 
vcac-vami service-manage start elasticsearch hzn-dots horizon-workspace vcac-server vco-server vco-configurator



Step:6 ( not present in the kb , an extra step just for validation )

Verify if all vRealize Automation services are registered



curl --insecure -f -s -H "Content-Type: application/json" "https:/$HOSTNAME/component-registry/services/status/current?limit=200" | sed "s/}/\n/g" | grep -E -o ".serviceName.*serviceInitializationStatus.[^,]*" | sed "s/\"serviceTypeId.*,//g" | sed -e "s/\"//g" -e "s/:/=/g" -e "s/,/, /" | sed -e "s/serviceName\|serviceInitializationStatus\|=\|,\|null//g" | column -t | sort | cat -n





 

Validation


To validate that the workaround has succeeded, take the following steps on all nodes


Verify that all vco and vcac processes are running with the java property "log4j2.formatMsgNoLookups=true":




Monitor the log /var/log/vco/app-server/vco_log4j_cve.log file, until you see 'Patching done.'





Run the command below to verify that the JndiLookup.class is not present in any log4j jar file for 2.x versions



find / -xdev -type f -name 'log4j-core-2*jar' -exec sh -c '/usr/bin/unzip -l "{}" | grep org/apache/logging/log4j/core/lookup/JndiLookup.class' \;

The command output will be empty if the scripting was successful.




 


This completes implementation of the workaround in vRealize Automation 7.6



 


201 views0 comments