Acknowledgement:
I’d like to acknowledge Ricky El-Qasem’s blogpost on the Veeam Blog. This was written in Sept, 2011 ago as Ricky has moved on to work for canopy-cloud.com
IMPORTANT:
Of course all this is just for lab testing purposes, remember vINCEPTION or Nesting as it is more commonly known – is unsupported…
One of the thing I want to do is running Windows 2012 HyperV inside a VMware vSphere 5.1 environment. There’s couple of reasons why:
Firstly, my big project at the moment is learning vCloud Automation Center or vCAC. There’s a couple of ways round pronouncing the acronym. Some people call it VC-AC (think of how you would say AC-DC) and others I’ve heard it pronounce CAC as “cake”. So take your pick. As you might know vCAC is to some degree hypervisor and cloud agnostic – although personally I think your better of using vSphere (but hell, I would say that wouldn’t I?). So I want to learn how to setup ALL the provisioning resource types – as you can’t really go about deploying VMs/vApps without any resources to point to (although apparently with vCAC there is a way of spoofing it to think it is… which I think might be rather interesting to investigate at a later date)
Secondly, we recently released “Multi-Hypervisor Management” (MHM for short, shouldn’t that be vMHM?) so I wanted to setup a temporary nested Windows 2012 HyperV environment to test that as well.
Thirdly, it’s occurred to me that as VMware’s Senior Cloud Infrastructure Evangelist (you gotta love these US job titles!) I can’t really do that in a VMware Bubble. So its time to start looking over the fence at what others are doing – because only by seeing what these systems are like for real, and can a truly see what differentiates VMware racehorse from the also-rans. It makes sense to start with Microsoft’s offering because lets face it there are biggest Elephant in the room. But once I’m done with them, I do want to look at other vendors too such as OpenStack(s).
Of course my focus has to be VMware for reasons which I think are patently obvious – and so for that matter I didn’t want to dedicate hardware in my lab to running them. Nested seems to be there way to go, and that’s what I’m doing for internal builds of vSphere whereas in the physical world – I’m running the latest and greatest GA code. I won’t be able to draw any conclusions from a performance perspective because it will be nested, not native. My interest is really in product functionality and integration.
Now down to business. If you try to install Windows 2012 Server OS and then enable the HyperV role the GUI or via PowerCLI with the proper settings you likely to get this error message. One of the complexities around Window 2012 HyperV is how even get this far. Do you install Windows with or without a GUI front-end (or Server Core as Microsoft calls it). Plus when you look at TechNet there’s even .ISO of things called “HyperV”. Is it me or is that just unnecessarily confusing – compared to dedicated hypervisor with no graphic front-end?
To do this with PowerCLI on Server Core you would use:
Install-WindowsFeature Hyper-V –Restart
That cmdlet would result in similiar response:
So to get this nesting to work we need to make some changes on the ESX hosts, and also the VM’s VMX file. Before you go any further you do realise this sort thang isn’t remotely supported. Thought so. But I thought I better say that to CYA…
STEP1: Enable VHV Allow on the ESX Host:
First you need open a console on the ESX host, and modify a text file. These way to do this is temporarily open SSH on the host, and PuTTY in. Once there’s run this command:
echo ‘vhv.allow = “TRUE” ‘ >> /etc/vmware/config
repeat and rinse for the all the remaining hosts in your cluster.
STEP2: Enable VMX Settings
Next we need to add two entries to the .VMX file called monitor.virtual_exec=hardware and hypervisor.cpuid.v0=FALSE parameters to the Nested Windows 2012 HyperV system
STEP3: Ensure CPU/MMU Virtualization is engaged
STEP4: Add a CPU Mask:
Finally, add a CPU mask to the Level1 register on ecx
—- —- —- —- —- —- –H- —-
From this point if you pCPU is of the right type be able to power on and enable the HyperV role – sadly for me my CPU still didn’t support this nested approach (although VMware ESX 5.x works perfectly fine).
Conclusions:
So for me its back to the drawing board. I guess what I could do is – take 4 of my 9 servers and install Windows HyperV to two of them, and Xen to the others. It’s not ideal. My Lenovo’s at the colo don’t have the blue widget that enabled remote console access. That means scheduling a visit to the colocation facility to install to physical. Plus I didn’t really want to dedicate physical hardware to this sort of thing – just spin it on demand, and power off when I’m done…