Over the last two months, I’ve been working on a VDI refresh engagement, involving technologies such as VMware Horizon View 5.2, vCNS (vShield) and McAfee Move, agentless solution. The customer previously used a standard McAfee agent solution, with the agent installed in every Windows virtual desktop. However they own the license for the agentless solution, and thought this would a suitable time to investigate this, to optimize the environment and provide the best performance, whilst continuing to protect their virtual desktops.
With the agentless solution, a virtual security appliance is run on each host, and performs all the scanning, plus definition and policy updates. This dramatically reduces the load on the host in terms of memory and CPU, and also the Windows virtual desktop, as there’s no requirement for the agent to be installed in Windows. The benefits are clear, with increased virtual desktop performance and anti-virus updates and scanning storms, a thing of the past. You can also manage the solution from the McAfee ePolicy Orchestrator software.
Therefore, after research and a trial period, I deployed the agentless solution and I thought I’d document the process and quirks\issues I found along the way, in case a future engagement involves this or a similar agentless solution. I hope others may also find this useful?
First of all, the McAfee Move solution and product guide can be found here: http://kc.mcafee.com/resources/sites/MCAFEE/content/live/PRODUCT_DOCUMENTATION/24000/PD24625/en_US/MOVE_AV_Agentless_300_Product_Guide_final.pdf
You can also review other external references at the end of this post.
The following graphic is taken from the above guide from McAfee, and outline the components involved in the solution.
- First step is deploying vCloud Networking and Security 5.1.2 (vShield Manager), into the management cluster, which is a separate instance managed by a different vCenter from the virtual desktops hosts.
- Download the vCNS appliance from VMware.com, and import the OVF and setup the relevant networking details using the deployment wizard via the vSphere client. The vCNS deployment and admin guides can be located here: https://www.vmware.com/support/pubs/vshield_pubs.htm
- Once the ‘vShield Manager’ VM is deployed and available on the network, connect using the URL of the appliance, using default credentials admin and password default
- Continue with the setup from the ‘Settings and Reports’ menu down the left, configuring the Lookup Service and other relevant settings such as NTP and SSL.
- Lookup Service – https://vcenterserver.domain.local:7444/lookupservice/sdk
- Now it’s time to update vCNS from 5.1.2 to 5.1.2a (latest maintenance release)
- Download the vShield upgrade bundle (from VMware.com) to a location where vShield Manager can browse to. The name of the upgrade bundle file should be:-
Note: Ensure the upgrade bundle is named as above, otherwise you’ll receive an error when trying to update vShield. I had to rename the file after download, in order to ensure this worked. This problem has been document by another blogger.
- From the vShield Manager Inventory panel, click Settings & Reports.
- Click the Updates tab.
- Click Upload Upgrade Bundle.
- Click Browse and select the VMware-vShield-Manager-upgrade-bundle-maintenance-5.1.2-997359.tar.gz file.
- Click Open.
- Click Upload File.
- Click Install to begin the upgrade process.
- Click Confirm Install. The upgrade process reboots vShield Manager, so you might lose connectivity to the vShield Manager. None of the other vShield components are rebooted.
- Verify the maintenance update has been applied
There are a couple of VMware KB articles to support this process.
- Upgrading to vCloud Networking and Security 5.1.2a best practices (2044458)
- Applying the 5.1.2-997359 vShield Manager Patch
After vCNS (vShield Manager) was upgraded to 5.1.2a, it was time to install the vShield Endpoint component onto each ESXi host. You can review the best practices guide from here https://www.vmware.com/files/pdf/VMware-View-AntiVirusPractices-TN-EN.pdf
- You can perform this step from either the vSphere Client or vShield Manager. I prefer to use the latter, just to verify things from the vShield Manager point of view. Browse to your host(s) and select, using the Inventory menu down the left and view the summary from the right hand side.
- Install the vShield Endpoint component and wait for verification the install has completed, you can view the various tasks from the vSphere client. Once completed successfully, you’ll observe a similar outcome to the below.
- Check the Endpoint Status, including the Events log.
- Perform this step for all remaining hosts.
- Now the vCNS and required components (Endpoint) have been installed, the McAfee Move Virtual Security appliances (VSA) can now be deployed.
In preparation for the McAfee Move virtual security appliances (VSA) on each ESXi host, McAfee ePolicy Orchestrator software, which acts as the management station for the solution needs to be deployed. The latest version 5.01 was downloaded and installed onto Windows Server 2008. This was performed by the customer, but it’s a simple process, selecting either a SQL Express or dedicated SQL Server installation, depending on the size of your environment. For more information, refer to the McAfee documentation or external references at the end of the post.
Following the ePO deployment, you need to install ‘Product Extensions’ to extend the functionality of the ePO software and allow for the Move agentless solution.
- Within ePO install the extensions via Menu>Software>Extensions>Install Extension
- Main product extension MOVE-AV-AL_EXT_3.0.0.zip
- License extension MOVE-AV-AL_License_EXT_3.0.0.zip
- Deploy the VSA using the McAfee MOVE-AV-Agentless OVF (MOVE-AV-AL-OVF_3.0.0.zip)
Note: When completing the details in the Properties section of the Deploy OVF template, I found some of these settings did not apply after the deployment of the VM. For example, if you set a new admin password for the svaadmin account, for some reason this password does not apply when first setting up the VSA at the console screen, you still need to use the default password of admin.
Also, settings such as the vShield and ePO did not apply either, and I had to re-enter these through the console.
- Complete the configuration of the appliance at the console screen from the vSphere client. I ran through the whole setup just to ensure all the relevant settings were correct, as some do not apply (for some reason?), from the deployment of the VSA from the OVF wizard.
- Confirm the setup is completed via the console, ensure you’ve setup and registered the appliance with vShield Manager and ePO server, to enable the appliance to communicate and function with both.
- From the vShield tab within the vSphere client or vShield Manager, confirm the Service Virtual Machine has been detected.
- Log into McAfee ePO (https://eposerver:843/console) and from the System Tree, browse to location
- My Organization>Lost&Found>none
- You should see your appliance, move (drag and drop) the VSA into your own Organization setup, from here you’ll be able to apply\assign Policies and Client Tasks. I setup a folder called ‘ESXi VSA’ which contains all the appliances, and the relevant polices are then applied to this location.
- You can also check the Executive Dashboard to ensure your VSA’s are in compliance.
- You can drill down further via ‘Compliant’ from this dashboard, this will list your VSAs. You can select a particular VSA, then explore and view all the properties and system details.
- Manage the policies via the Policy Catalog. Select Move AV agentless from the Product drop down list. My Default (Scan) and My Default (SVA) are the policies to manage and should be renamed accordingly.
- My Default (Scan) policy, you can enable or disable the following settings. I found most of the default settings to be fairly correct, but these should be tuned according to each unique environment. Here you can enable On-Demand scanning if desired, to perform a scan of all virtual machines protected by the VSA on the host, during a selected window.
- My Default (SVA) policy, the Authentication tab requires the relevant details of your vCenter Server and credentials. Use the ‘Test connection settings’ to verify this setting.
- Scan Settings tab, this enables further control of the On-Demand scan settings, which you can enable from the Default (Scan) policy. You can tweak these as desired, you may wish to change the Scan time window to suit your environment. You can also change the On-Demand scan time interval (default is 7 days).
The VM-based scan configuration allow further granular control, to group protected virtual machines, and then apply policies to these groups. You need to install the relevant Data Center Connector for vSphere, which discovers and imports both running and stopped machine instances from VMware vCenter to the McAfee ePO server. This product integrates the management feature of McAfee ePO with the VMware vCenter server, and displays the imported virtual machines and their protection status on McAfee ePO.
- I stumbled across a few issues throughout the deployment process. You may come across the following error when trying to register the Move appliance with vShield Manager.
‘No route to host’
I found this to be a setup issue, after a quick ping test of the appliance which failed, I double checked the appliance VM. On initial deployment of the OVF template, I mis-configured the management network, and selected the next VLAN in the list. After quickly editing the VM and selecting the correct vNetwork, I was able to ping my appliance and register with vShield Manager.
- The following error may appear on startup of the appliance. I found waiting for a couple of minutes, the appliance would boot and function as normal. Verify from vShield Manager and ePolicy Orchestrator.
- If there’s a reason to inspect the Move appliance logs, maybe the appliance isn’t communicating with ePolicy Orchestrator upon initial configuration, or the services won’t start or display an error.Run the following command
- ls /opt/McAfee/move/log/
You can use Linux commands like ‘tail’ to inspect the logs – http://www.folkstalk.com/2012/10/tail-command-examples-in-unix-linux.html
The logs are fairly self-explanatory, in my experience I used the mcafee_agent_registration.log and mvsvc.log, whilst trouble-shooting communication from the appliance to the ePO server.
Move-AV appliance – VM Resources
The standard deployment of the VM, is configured for 2GB and 2 vCPU by default. I couldn’t find any sizing guidance within the documentation, for example, recommended specification for 50+ VMs per host or 100+ VMs per host. The documentation stated a minimum of the above configuration. Therefore, initially I deployed with the default values.
On-Demand scans, where the appliance will scan all the VMs on the host, can be scheduled for a window of your choice (preferably out of production hours). The On-Demand scan setting is disabled by default. I enabled this to see how the appliance and host performed during a test period.
CPU – Resources of the appliance were maxed out during on-demand scans, which by default scans a maximum of two VMs. You could change this setting, however if you increase the number of concurrent scans, I would advise possibly looking at increasing the Move appliance to 4 vCPU. As long as On-Demand scans are occurring out of production hours, having a 4 vCPU Move appliance, which will no doubt make full use of those vCPUs (although I haven’t tested), should not affect other virtual desktops running on the host in terms of ESXi co-scheduling.
RAM – The operating system of the appliance tends to consume around 1.5GB RAM, however when On-Demand scans are taking place, the VM uses all of the RAM available. Towards the end of the scanning window, I found a couple of different appliances just locked up and crashed. The host was running with around 50 VMs.
I recommend increasing this to at least 4GB RAM, possibly 6GB or 8GB depending on the VM\host ratio.
Within vShield Manager, Service VMs (appliances managed by Endpoint), should not be visible from the Inventory, as management operations are not supported. However, I detected a few Move appliances which were visible and showing under the Summary page as Services ‘Unprotected’.
I could not find any errors or alarms in the logs from vShield Manager, or via the vShield tab on each host within vCenter using the client. From here, the Endpoint status was also good. However, from the General page, I noticed the ‘Service Virtual Machines’ listing as blank. Other hosts were listing the Service VM.
Logging into McAfee ePolicy Orchestrator, showed all appliances communicating and within compliance. However, I did notice from each of these three Move appliances, the ‘Threat Events’ within ePO was relatively empty for the last few days, in comparison to the other Move appliances.
Therefore, to resolve this issue, I had to unregister the Move appliance from vShield Manager, and then register the appliance again.
From the Move appliance console, login as svaadmin and run the below command to run the setup configuration script
- sudo /opt/McAfee/move/bin/sva-config
- Enter ‘no’ for configuring other items such as host and network
- Choose ‘unregister’ for vShield
Run the sva-config command again (use tab), enter ‘no’ for configuring other items.
This time when prompted for vShield, choose ‘Register’ and supply the details.
Waiting for a minute or two for registration to complete, then quickly switching to vShield Manager and selecting the VM, shows ‘Operations not supported on this virtual machine’. Wait for another minute, and the VM will disappear from the inventory (which is a good sign!).
To verify all components, within vShield Manager, under the Inventory, click on Datacenter. Under General, verify the Service VMs (Move appliances) are detected and running on each host. Also, double check Endpoint is enabled for all hosts.