Consistent Network Configuration across ESX hosts

One of the key settings from our VMware health check that needed adjusted was the network setup. The goal is to make all connections redundant and constitant across the configuration so I am going to post the recommended config for our particular ESX hosts which are Dell M710's. They have the capability to have 2 sets of Mezzanine cards, however we only have 1 set filled. Of those we are only using the onboard ethernet or A fabric and the ethernet mezzanine or B fabric.

Therefore per the healthcheck tool, here are the recommendations.

----------------

Minimize differences in the network configuration across all hosts in a cluster. Consistent networking configuration across all hosts in a cluster easies administration and troubleshooting. Also, since services like VMotion require portgroups to be named consistently in order for VMotion to work, it is important to have a consistent nnnnnzz

Also, use a consistent naming convention for virtual switches, portgroups, and uplink groups

Product/Version: vSphere 4

VMware vSphere 4 introduces VMware vNetwork Distributed Switches (vDS) and Cisco Nexus 1000V distributed switches which reduce administration time and ensure consistency across the virtual datacenter. Changes to the distributed virtual portgroup are consistently and automatically applied to all hosts that are connected to the distributed switch. Check the licensing requirements in order to determine if distributed switches can be used in the environment.

Consider using distributed switches if possible.

Network Design

Blade Module	VMNIC#	Connection Type
A0	VMNIC0	Service Console
A1	VMNIC1	VMotion
A0	VMNIC2	Fault Tolerance
A1	VMNIC3	VM Production
B0	VMNIC4	Mgmt Spare
B1	VMNIC5	VM Production

------------------------------

In order to accomplish this, here is how we setup it in VMware.

1. We created 2 Distributed Virtual Switches - a production one, and a Service Console One.

2.Under the Service Console switch create a port group for Fault Tolerance, this should be a very limited vlan or for administrative purposes only.

3. Under the Service Console switch create a port group for the Service Console, this should be a very limited vlan or for administrative purposes only.

4.Under the Service Console switch create a port group for Vmotion, this needs to only be on the virtual switch or across any switch ports that Vmotion would be used across. It is something the ESX hosts handle internal to VMware so it doesn't even need to be routable.

5. On the production Switch current port groups as needed for your environment.

Teaming and Failover Setup

1. You need to setup Teaming and Failover to make the uplinks redundant which can be done after you have added the vmnics to the uplinks.

2. Once that is completed edit the settings of each port group. Go to the Teaming and Failover section under policy.

3. Change the links based on the Chart at the top so the uplink that matches the vmnic above is the Active Uplink. Move the rest of the uplink to standby. See the screen shot below for an example.

When all this is done, the ESX host specific configuration, should look like the image below. If your switching configuration is setup correctly then VMware should be now be setup redundant and according to the Health Check best practices.

Vmware ESX Host Drive Space Best Practice

We recently had Vmware technicians come in to analyze our environment we had configured and tell us what needed changed and give us best practices for VMware in our environment. It was through Dell and ultimately there is a Virtualization Healthcheck tool that will go through and tell you the majority of this. That tool runs as a Guest OS host in your VMware enviroment.

This particular post lays out what optimum settings for virtual disk size when setting up and ESX 4.0 host. This is one setting that needs to be done before you go much further since changing it pretty much requires a rebuild of the ESX host.

Here are the settings we ended up using.

• /boot - ext3 1100MB (PreConfigured Space for upgrades)
• - swap 1600+MB (Change for maximum services console swap)
• / - ext3 16384 MB (Change for additional space in root)
• /var - ext3 8192 MB (Create this partition to avoid overfilling root with log files)
• /tmp - ext3 8192 MB (Create this partition to avoid overfilling root with temp files)
• /opt - ext3 8192 MB (Create this partition to avoid overfilling root with VMware HA log files)
• - vmkcore 100MB (Memory Dump for PSOD - Note I didn't have this option)
• Leave all remaining space on the local volume unpartitioned

That is the first major setup best practice that isn't immediately obvious by going through the steps, more to follow including network setup and reasoning since that is one of the toughest ones in our experience.