By BackToBasics

Well, its the start of a new “chapter” in the Back To Basic series. This time focusing on networking. Hang on I hear you say, didn’t you cover that way back when. Your right I covered Standard Switches (vSwitches?) and ploughed on into Storage (so I get my Synology and IOMega NAS setup)… But how would you do all that with distributed switches instead.

Introduction to Distributed vSwitches

VMW-DGRM-vNTWRK-DIST-SWTCH-lg.jpg

Distributed vSwitches were first introduced in vSphere 4.0 – and since then various enhancements have been made as each subsequent release of vSphere has been released. At their heart, Distributed vSwitches are method of centralizing the management of the virtual network into single plane. Every VMware ESXi host added to Distributed vSwitch inherits its configuration, and those settings are stored within vCenter, rather than on the ESXi host itself. This means adding new portgroups for a new VLAN for a cluster of ESXi hosts are relatively trivial affair. The VMware ESXi host “caches” its Distributed vSwitch to local storage so if the vCenter is unavailable for whatever reason network communications are unaffected. However, no management of the Distributed vSwitch is possible until it is restored. For this reason some virtualization admins prefer that infrastructure VMs such as vCenter, SQL, Domain Controller and other VMware services and appliance remain on standard vSwitch to allow for continued management even if vCenter is offline.

Distributed vSwitch also off some features which are easier to configure than with Standard vSwitches, such as adjusting the MTU for Jumbo Frame support – in addition there are some unique features these include:

  • Private Virtual LAN support (PVLAN)
  • Port Binding
  • Traffic Shaping for both inbound and outbound traffic
  • Port Policies and Port Mirroring
  • Network IO Control and Network Resource Pools
  • NetFlow
  • Network Rollback and Recovery
  • Health Check
  • Enhanced LACP Support
  • Additional load-balancing options on a Distributed Portgroup called “Route based on physical NIC Load”

This part covers the basics of creating the Distributed Switch, Adding Hosts and creating portgroups for both virtual machine and VMware ESXi host networking…

UPDATE:

Since the first publication of this blogpost I’ve added some videos. The first is a “Show Me How” video which demo’s how to create a Distributed Switch using “Template Mode” in the vSphere Web Client, it then continues to show how to add VMKernel ports also using template mode to bulk assign the IP addresses required.

Native Format

The second video is a “Discuss The Options” video with @chriswahl  (Social media adventurer, virtualization whisperer, and global event speaker. VCDX 104. Author of  &) In this video he talks about whether you should distributed switch exclusively, or whether you should use a combo of Standard & Distributed Switches, after we talk about the different network control available (load-balancing/distribution, LAGs, NOIC, Jumbo Frames and so on) and which pay dividends and which aren’t helpful in your environment.

 

 

Creating Distributed Switch

1. In the Web Client, right-click a Cluster or DataCenter, and select New Distributed Switch

Screen Shot 2014-02-11 at 13.39.28.png

2. Type a friendly name for the Distributed Switch. Names must be unique to the enter vCenter namespace, and commonly reflect a cluster of VMware ESXi hosts. Distributed vSwitch can span clusters, but many virtualization administrator prefer to curtail the scope to the host in a clusters. This means administration can be limited impact an smaller number of physical hosts. In this case DSswitch-GoldCluster01 was used to reflect the naming structure envisioned for the VMware Clusters that will be created.

Screen Shot 2014-02-11 at 13.49.46.png

3. Next, select what version of Distribute Switch you wish to use. Older formats are supported for backwards compatibility in situations where vCenter is managing a cluster of VMware ESXi hosts that have yet to be upgraded to the latest version.

Screen Shot 2014-02-11 at 13.49.55.png

4. Select the number of uplinks to be assigned. The default here is 4 vmnics. In our case vmnic0/1 have been assigned to a Standard vSwitch (vSwitch0), leaving another two physical NICs available (vmnic2/3). These physical adapters will be assigned to the Distributed Switch when we add the VMware ESXi hosts to the Distributed Switch in a later wizard. NIOC allows for advanced methods of triaging and prioritising network traffic which makes it a more sophisticated method than using Traffic Shaping on its own. Finally, you can create an initial portgroup. If you do wish for VLAN tagging to be supported this done by modifying the settings of the portgroup after it has been created in this wizard.

Screen Shot 2014-02-11 at 13.50.16.png

5. Click Finish to complete the wizard. Notice how this wizard indicates further steps including adding VMware ESXi hosts to the Distributed Switch and

Screen Shot 2014-02-11 at 13.50.27.png

After a short while the Distributed Switch should appear in the Web Client like so:

Screen Shot 2014-02-11 at 13.50.51.png

Adding VMware ESXi hosts

Note: When this article was first published I wasn’t using the “Template Mode” method of adding hosts into the Distributed Switch. This method is much more admin efficient and involves less clicking!

Once a Distributed Switch has been created we can make it accessible to the VMware ESXi hosts, and select which physical vmnics will back the movement of network traffic out to a physical switch.

1. You can add VMware ESXi hosts, by right-clicking the Distributed Switch and selecting Add and Manage Hosts…

Screen Shot 2014-02-11 at 17.03.38.png

2. Select Add Hosts

Screen Shot 2014-02-11 at 17.05.53.png

3. Click the green plus + to add hosts into the list. Enable the option to Configure identical network settings on multiple hosts (Template Mode). Template mode allows the administrator to select one reference VMware ESXi host and have its settings applied to all other hosts added to the Switch. This option massively reduces the amount clicking to complete the configuration – but does assume the physical configuration such as vmnic number is identical.

Screen Shot 2014-02-23 at 14.36.03

4. Select the server to be the template host, in our case we selected esx01nyc.corp.com

Screen Shot 2014-02-23 at 14.40.54

5. In our case we deselected Manage VMkernel Adapters, but keep Manage Physical Adapters selected. This is because will look at migrating both virtual machine network and VMkernel Networking from Standard vSwitches to Distributed Switches later

Screen Shot 2014-02-11 at 17.08.31.png

6. Select a free physical network adapter, in out case vmnic2, and assign it an Uplink container

Screen Shot 2014-02-23 at 14.42.24

7. Repeat this process for all the vmnics you wish to assign to the Distributed Switch, once completed – click the Apply to all button to apply this configuration to all the other VMware ESXi hosts

Screen Shot 2014-02-23 at 14.44.47

7. vCenter will look at your management changes, and assess the likely impact on your infrastructure.

Screen Shot 2014-02-11 at 19.36.17.png

Clicking Finish will trigger the adding of the ESXi host to the Distributed Switch, and you can confirm the hosts are correctly added and connected from

Screen Shot 2014-02-12 at 08.50.46.png

Creating and Modifying Virtual Machine Portgroups

Note: Advanced settings and options on both the Distributed Switch and Portgroup will be covered later.

The default portgroup created along side the Distributed Switch can be easily modified to support features such as VLAN tagging, and additional portgroups for VLANs can be easily added with the Web Client.

1. Right-click the target Distributed Switch, and select New Distributed Port Group

Screen Shot 2014-02-12 at 09.32.50.png

2. Type in a friendly and unique name for the portgroup

Screen Shot 2014-02-12 at 09.09.13.png

3. Assign the VLAN Tagging option, and type in the VLAN ID

Screen Shot 2014-02-12 at 09.12.09.png

4. Click Finish to create the portgroup

Screen Shot 2014-02-12 at 09.26.56.png

Creating VMKernel Portgroup

As well as virtual machine networking, it is possible to use Distributed Switches for VMware ESXi host network as well. Although many VMware Admins still prefer to use Standard vSwitches for this type of functionality – there maybe case where the administrator is compelled to do use just Distributed Switches. For instance a physical server may only have two 10Gps/20Gps/40Gps NIC interfaces which need to be teamed up in one switch technology – as physical vmnics can only be used once per switch type.

Creating a VMkernal Portgroup is a two-stage process and involves interacting with the Distributed Switch, and with the VMware ESX host to configure the VMkernel’s IP settings which are unique to the VMware ESXi. In this example we will create a portgroup to enable the VMware Fault Tolerance.

1. Right-click the target Distributed Switch, and select New Distributed Port Group

Screen Shot 2014-02-12 at 09.32.50.png

2. Type in a friendly and unique name for the portgroup

Screen Shot 2014-02-12 at 10.05.22.png

3. Assign the VLAN Tagging option, and type in the VLAN ID

Screen Shot 2014-02-12 at 10.05.38.png

4. Click Finish to create the portgroup

Screen Shot 2014-02-12 at 10.46.18.png

Once the portgroup has been created then we can transition to creating a VMKernel Port.

5. Select the VMware ESX host, and under the Manage tab, and the Networking column – select the VMKernel Adapters type

6. Click the Globe icon, to Add Host Networking

Screen Shot 2014-02-12 at 13.51.04.png

7. In the wizard, select VMKernel Adapter

Screen Shot 2014-02-12 at 13.52.03.png

8. Select option to use an Existing Distributed portgroup, and click the Browse button

Screen Shot 2014-02-12 at 13.55.30.png

9. In this case under VMkernel port settings we selected Fault Tolerance Logging

Screen Shot 2014-02-12 at 14.25.37.png

10. Configure appropriate IP settings as befits your network

Screen Shot 2014-02-12 at 14.26.00.png

11. After clicking next, the Web Client should refresh, and show the new VMK portgroup with its assign Distributed Switch. These configuration steps need to be repeated for each host

Screen Shot 2014-02-12 at 14.00.18.png

Note: For the configuration status for FT to switch from No, to Yes – the VMware ESXi host must be part of High-Availability Cluster (HA)