By BackToBasics

Introduction

In Part One of this series on Standard vSwitches I looked at creating internal, basic and NIC Teamed vSwitches – together with a configuration designed for load-balanced iSCSI communication. Following on from that this part looks at enabling the options for VMotion, Fault-Tolerance and High Availability. I also do a survey of the common settings on vSwitches and Portgroups. In the next part I will look at doing all this configuration using PowerCLI.

Creating an VMkernel Port for VMotion

1. Select vSwitch0 (or create a new vSwitch), and click the Add Host Networking button

2. Select VMkernel Network Adapter as the portgroup type

Screen Shot 2013-11-27 at 09.25.39.png

3. Confirm that you are selecting an existing standard vSwitch

Screen Shot 2013-11-27 at 09.26.10.png

4. In the connection settings type a friendly portgroup name, and if required a VLAN tag value.

5. Under Available Services, enabled the option X vMotion Traffic

Screen Shot 2013-12-13 at 14.44.38.png

6. Next configure a static IP address for the vMotion Traffic

Screen Shot 2013-12-13 at 14.46.31.png

Note: In order for vMotion to work, all the those need to be patched to the vMotion network, and configured with a valid IP address. This is generally a dedicated network with bandwidth allocated of 1Gps or higher. You can validate if vMotion has been enabled by checking the “Configuration” pane on the Summary Tab of an ESX host.

Screen Shot 2013-12-13 at 14.50.33.png

Creating an VMkernel Port for Fault Tolerance

VMware Fault-Tolerance (FT) is feature that allows all the activity taking place in one VM to mirror to a second VM on a different ESX host. In this situation the VMs are considered to be held in a “lock step”. Should the “Primary” VM fail, then the “Secondary” VM instantaneously takes over. One requirement for FT to work is a dedicate network which can provide 1Gps or higher bandwidth. FT is covered in greater detail in the High Availability section of the vmWIKI.

1. Select vSwitch0 (or create a new vSwitch), and click the Add Host Networking button

2. Select VMkernel Network Adapter as the portgroup type

Screen Shot 2013-11-27 at 09.25.39.png

3. Ensure that vSwitch0 is the existing standard vSwitch

Screen Shot 2013-11-27 at 09.26.10.png

4. In the connection settings type a friendly portgroup name, and if required a VLAN tag value.

5. Under Available Services, enabled the option X Fault Tolerance Logging

Screen Shot 2013-12-13 at 14.58.11.png

6. Next configure a static IP address for the Fault Tolerance Traffic

Screen Shot 2013-12-13 at 15.03.22.png

Note: You can validate if FT has been enabled by checking the “Configuration” pane on the Summary Tab of an ESX host. However, other requirements must also be met for this to be flagged as Yes. Firstly, the physical host must support the correct CPU chipset requirements, and the ESX host most be part of VMware High-Availability cluster. In the screen grab below a VMware High Availability cluster has been created.

Screen Shot 2013-12-13 at 15.15.28.png

Creating an VMkernel Port for Management/VMware HA Heartbeat

In order for VMware High-Availability to function without errors redundancy must be delivered to the management network. This ensure scenarios like “Split Brain” where two or more hosts consider each other as “down” because they have become unreachable across the network – are unlikely to happen. Network redundency could be provided by furnishing the management vSwitch with two VMnic plugged into two independent physical switches or by adding a second management network to different vSwitch to the one mainly used for Management

1. Select vSwitch0 (or create a new vSwitch), and click the Add Host Networking button

2. Select VMkernel Network Adapter as the portgroup type

Screen Shot 2013-11-27 at 09.25.39.png

3. Ensure that vSwitch0 is the existing standard vSwitch

Screen Shot 2013-11-27 at 09.26.10.png

4. In the connection settings type a friendly portgroup name, and if required a VLAN tag value.

5. Under Available Services, enabled the option X Management Traffic

Screen Shot 2013-12-13 at 15.20.28.png

6. Next configure a static IP address for the Management Traffic

Screen Shot 2013-12-13 at 15.22.07.png

Note: There is no special way of checking you have redundency to the management network – but VMware HA will warn you if it detects redundancy isn’t present.

Configuring Standard vSwitch Settings

Standard vSwitch support a rich array of different configuration settings together these settings could be regarded as policy. The settings can be on the vSwitch itself, or on the portgroups. A system of “over-rides” exists such that the vSwitch could have setting Y, but the portgroup would have setting X. This allows for scenario where you have rules, and exceptions to those rules. These setting extend to include functionality such as performance, security, traffic management and NIC Teaming and Failover parameters.

Settings on the vSwitch can be modified by selecting the ESX host and the vSwitch, and clicking the “pencil” icon to Edit Settings.

Screen Shot 2013-12-13 at 15.30.59.png

What follows below is explanation of each of the settings with examples of situations which would merit their modification:

Maximum Transmission Unit (MTU) aka Jumbo Frames

Screen Shot 2013-12-13 at 15.33.13.png

The MTU value (aka Jumbo Frames) controls the size of the Ethernet frame on the network. Increasing this value, increases the MTU allowing the Ethernet Frame to carry more real data for the same overhead. In some circumstances supporting a larger MTU can improve performance. Care must be taken when increasing beyond the default of 1500 Bytes. The physical switch must be reconfigured to support the large MTU size – additionally performance can be degraded if a large MTU ethernet frame encounters a device configured for a smaller MTU size. The subsequent packets are fragmented into a stream of smaller packets. There has been some debate as to whether the benefits of a large MTU makes a great deal of difference in 10g environments, and if the administrative penalty of enabling the feature is smaller than the perceived performance improvements. This complexity increases if your intention is not to just provide a large MTU to the ESX network traffic but to the virtual machines as well. Generally, performance does improve – and Michael Websters (VCDX) blogpost “Jumbo Frames on vSphere 5” is good starting point in understanding the benefits. For instance MTU could be enabled on desecrate communication for traffic such as vMotion or IP-Storage where the benefits may be more immediately realised.

Security Settings

Screen Shot 2013-12-13 at 15.43.53.png

WARNING:

Consistency is king. If two ESX hosts have different security settings this can cause functions like vMotion to break. All vSwitch and Portgroup ship with standard security settings called Promiscuous Mode, MAC Address Changes and Forged Transmits – these configured by default as Reject, Accept and Accept. It would be more secure to set all these options as reject – however care must be taken to not over tighten the security screws and unintentionally break process running inside the guest operating systems within the virtual machines.

By default Promiscuous Mode is set to reject – and this prevents packet capturing software installed to compromised virtual machine for being used to gather more network traffic to facilitate a hack. Nonetheless it could modified by a genuine network administrator to capture packets as part of network troubleshooting. Even with this option enabled it would not stop an administrator from receiving packets to the VM. Another reason to change this option to Accept if you want to run intrusion detection software inside a VM. Such intrusion detection needs to be able to sniff network traffic as part its process of protecting the network. Finally, a less well-known reason for loosening the security on promiscuous mode is to allow so called “Nested ESX” configurations. This is where ESX is installed into a VM. This generally done in homelab and testing environments, not generally recommended for production use.

MAC Address Changes and Forged Transmits are both related to MAC address. Certain availability allows the guest operating system to send out traffic even if the traffic is being sent using a MAC address that is different from the one assigned virtual machine. This can happen in certain clustering environments such as Microsoft NLB or Microsoft Clustering Services (MSCS) where a “virtual IP” address is assigned a “virtual MAC” address to allow inbound/outbound access to the cluster. Configuring Reject for MAC Address Changes and Forged Transmits would cause such software to malfunction, as such the setting is compromise between security and usability.

Traffic Shaping

Screen Shot 2013-12-13 at 17.25.40.png

By default traffic shaping is disabled on both the vSwitch and the portgroup. Once enabled it allows the administrator to control bandwidth used as exits the physical server. Traffic Shaping does not control traffic as enters the server, and for this reason many organization do not consider it advantag

NIC Teaming and Failover

Screen Shot 2013-12-13 at 16.15.53.png

The NIC Teaming and Failover settings control a rich array of possible configurations and options. Load-Balancing supports four different options including (1) Route based on IP Hash, (2) Route based on MAC Hash, (3) Route based on Originating Port and (4) Use explicit failover order. Route based on originating port essentially load-balancing controlled by a round-robin on the teamed network interfaces, and is the default because its highly compatible with many physical switch configurations. Route based on IP Hash is consider the most optimal for performance . However, for this method to function the physical switches must be enabled for the 802.3 (AD) Link Aggregation standard and IP data is used to select the best physical NIC with which to send the traffic. Although a load is placed on all the NICs the maximum throughput is limited to the maximum transfer of the given physical NIC.

With the Explicit Failover Order option it is possible mark on VMnic as Active and another as Standby. Such a configuration looks like this:

Screen Shot 2013-12-13 at 16.31.03.png

This option works in conjunction with the Failback option of Yes/No. With Failback set to Yes. If VMnic0 failed, VMnic1 would take over – when VMnic0 became available then traffic would failback (return to) the Active NIC. If the setting is changed to No, even if the Active NIC becomes available again – the traffic stays with the vmnic1 device until such time that it experiences a failure. Such a configuration guarantees dedicated allocation of bandwidth and is highly reliable. However, its is an uncommon configuration as many consider dedicating a NIC into a standby mode a waste of precious bandwidth. A configuration where all the physical vmnics are marked as Active looks like this from the web-client

Screen Shot 2013-12-13 at 16.41.41.png

Whereas a Active/Standby configuration is represented like so – with the yellow line indicating which vmnic is taking the network load currently:

Screen Shot 2013-12-13 at 16.40.02.png

Network Failover Detection controls how VMware ESX discovers that a physical network path is unavailable. It supports two options Link Status Only and Beacon Probing. Link Status Only merely checks the NIC and its primary connection to the physical switch. It cannot validate if further upstream paths are valid. In contrast Beacon Probing sends out a discover packet to check the state of the networking. In theory it is a more intelligent method of detecting the true state of the network. However, in practise many switching architectures including those from Cisco often don’t benefit from the use of beacon probing. Consult your switching vendors best practises and recommendations before enabling beacon probing.

Notify Switches this option when set to Yes, sends out a reverse ARP (RARP) packet which helps keep the physical switch aware of changes occurring in the network. Typical leaving the default in place is the best practise as it helps with the vMotion process when VMs are being moved across the network. Although the virtual MAC address of the VM does not change when it is moved from one host to another – the MAC addresses of the physical world certainly are different from one ESX to another. Microsoft NLB software when configured in a unicast mode is incompatible with Notify Switches set to Yes. If Microsoft NLB is configured with multi-cast functionality then this is not a problem. Therefore if you are using Microsoft NLB within a VM using Multi-cast is recommended. Alternatively, a portgroup can be created merely for the Microsoft NLB cluster with the Notify Switches set to No.

Configuring Standard Portgroup Settings

The settings of a portgroup can be access by selecting the underlined text of the portgroup, and clicking the “pencil” icon to edit the settings:

Screen Shot 2013-12-13 at 17.08.51.png

Many of the portgroup settings are exactly the same as the settings found on the vSwitch. The only difference is that effect only the virtual machines or vmkernel traffic configured for those portgroups. For example the security settings on a portgroup are exactly the same as the vSwitch – and merely allow the administrator to over-ride the settings from the vSwitch. Nonetheless there are some settings that are unique to the portgroup that may require reconfiguration dependent on the organizations requirements.

Screen Shot 2013-12-13 at 16.59.30.png

Note: Here the security settings on the portgroup are the same as can be found on the vSwitch. Notice the “override” options which allow portgroup settings to act as an “exception to the rule” that is found on the vSwitch.

VLAN Tagging Settings

Screen Shot 2013-12-13 at 17.07.40.png

VLAN Tagging is the Standard vSwitches primary method to support the access of many VLANs on a physical switch with a small number of physical interfaces required on the ESX host. VLAN Tagging enabled when a portgroup is defined. However, thought operator error the wrong VLAN Tag may have been assigned, and so it can be modified after the wizard has completed. The portgroup also supports the use of VLAN ID 4095. This is required when enabling “Virtual Guest Tagging” (VGT). This is where the tagging takes place inside the guest operating system running within the virtual machine. VGT isn’t a common configuration as it essentially “turns off” the ESX tagging process in favour of one managed by the virtual machine. However, there certain circumstance where this is desirable. For example if you are running a virtual appliance that acts a firewall – it maybe easier to manage the VLAN configuration within the firewall management software than using VMware ESX. Another more tangental use case is when “Nested ESX” is being used. This when VMware ESX has actually being installed into a virtual machine. This generally used in lab and testing environments to save on hardware costs. In order for the VLAN Tagging function to work within the Nested ESX environment, VGT is required.