Manual Process:

http://blogs.technet.com/b/jhoward/archive/2013/11/06/hyper-v-generation-2-virtual-machines-part-8.aspx

PowerShell Process:

http://blogs.technet.com/b/jhoward/archive/2013/11/14/hyper-v-generation-2-virtual-machines-part-10.aspx

In the world of VMware, when a big new shinny version of vSphere is released it’s not uncommon to find that a new version of VMware Tools and a new virtual hardware level is available. Since vSphere 5.1 it’s possible to upgrade VMware Tools without a reboot, albeit with caveats. Upgrades of the virtual hardware aren’t mandatory – but you do need to power off the VM before editing the virtual machines settings. The nice thing about the upgrade of virtual hardware is the total absence of a torque screwdriver (that’s my little joke by the way).

Note: As we all know upgrading the virtual hardware exposes a new hardware level (which we now refer to as a “compatibility level” which is a combination of VMware Tools version and hardware level) with new features. In the case of hardware level 8 this includes 32-vCPU, 1TB RAM, 3D Graphics for Aero, USB 3.0 and UEFI virtual BIOS. The nice thing about the VMware method is we can defer the upgrade until the next normal guest OS shutdown – or “Patch Tuesday” as its known in the trade. 🙂

In case you don’t know Windows Server 2012 Hyper-V R2 introduced a new Gen2 VM definition. It supports UEFI, boot from SCSI devices rather than IDE, and removes some (but not all) legacy devices in their VM. Perhaps the biggest reason to use a Gen2 over Gen1 VM is the ability to resize disks on the boot disk – something an IDE/Gen1 was incapable of doing. Personally, I find all this quite bizzare. VMware vSphere VMs have always been able to boot from SCSI, and hot-extend of disks from the command-line and UI were added donkey’s years ago…

Sadly, our friends over on the Microsoft side of the fence don’t have it so easy when it comes to “upgrading” from Gen1 virtual machines, to Gen2 virtual machines – that were introduce mid-way in the life of Windows Server 2012 Hyper-V R2. The process (and I’m being generous by using that term) involves many, many steps if you are doing it manually using four different command-line tools, and many different PowerShell cmdlets. True, there is PowerShell script that “automates” some of this – but the critical bottleneck has to do with copying of the contents of the Gen1 VHD disk into a Gen2 VHDX disk – something Powershell cannot improve.  This conversion has to happen because Gen1 VMs were IDE based, and Gen2 are SCSI based – and Gen2 VMs do not support booting from IDE. From a performance perspective I don’t see how SCSI would be any better than IDE. It’s virtual after all – but it does show what a pain in the rear only supporting IDE has become. Even when given a blank slate to begin with, Microsoft made a poor decision back in their first iterations of virtualization.

So not only do you have to claim a maintenance window to carry out the process – the time the “conversion” takes is directly related to the speed of your storage and volume of data inside the virtual disk. The other thing to be careful with is – as this process uses “diskpart” you could easily repartition the wrong disk – so be cautious as this runs the risk of data lost. The last thing you want to be doing is reaching for your backup catalog whilst attempting a conversion. As with all Microsoft conversions/upgrade paths the story gets decidedly shaggier the more you get into the process… For instance even after doing the disk conversion – a new Gen2 VM has to be created, so any “custom” settings you had on the Ye Olde Gen1 VM will need documenting and configuring on the new VM… L With that said Howard’s script does handle this more efficiently.

Having worked through the manual process and the Powershell script, and considering the fact that this involves data crunching and extended maintenance windows – it’s my bet most Windows Hyper-V customers wouldn’t want to undertake this “conversion” process. Instead they would probably run with their existing Gen1 where possible, and adopt a policy that all new VMs based on Windows Hyper-V 2012 R2 would be Gen2 based. Another approach would be to invest in some 3^rd party tools to help in the conversion. This is all rather ironic to me. Folks who adopt Windows Hyper-V often bang on about cost savings – and they are unlikely to want to supplement entry-level virtualization with additional software. A case of pennywise and pound foolish, perhaps?

Sadly, I think that this Gen1 to Gen2 pain point diminishes the impact of the new Gen2 feature in the short-term… Of course you could always just reinstall Windows to a clean VM, and not bother with this conversion process. We have been here before in my previous article “What works best – Clean install or upgrade”. That’s a view echoed in John Howards post at the end of the process:

So that’s it. Although somewhat involved, that is what it takes to convert a workload from generation 1 to generation 2. You may, of course, decide that a clean installation may be simpler.

http://blogs.technet.com/b/jhoward/archive/2013/11/06/hyper-v-generation-2-virtual-machines-part-8.aspx

Okay. Deep breath. I’ve broken this down into 8 main steps – the truth is within these 8 steps are quite a few sub-steps. Remember the fastest way of doing this is to use the MSDN available PowerCLI script. I’m going try both methods to see how they fair. I think it’s important to know the manual process prior to automation in case the train comes off the tracks.

1. Disable the recovery environment in source VM
2. Shutdown source VM; Mount VHD; Capture the image
3. Create a new VHDX
4. Partition in the GPT format
5. Clone WIM to VHDX; Configure BCD Store
6. Clean up and Unmount VHD/VHDX files
7. Attached to new Gen2 VM
8. Rebuild recovery partition and re-enable recovery environment

PHEW!

Of course – they’re some important caveats around this process to be aware off:

• By cloning a Gen1 VM to be Gen2 VM – you have two copies of the same VM. You should only power one on at any time to prevent IP conflicts, and potential AD issues associated with computer accounts – or patch them into different networks or VLANs
• Even if you use the script – parts 1 and 8 must be done before hand

My tests were done using a Windows 2008 R2 instance running IIS. And this was my first mistake. I didn’t check to see what OS types are supported with the new Gen2 format. I just assumed Microsoft would support Windows 2008 R2 because it’s still so popular in customer environments. It turns out Gen2 VMs can only use Windows 2012 or Windows 8.

http://technet.microsoft.com/en-us/library/dn282285.aspx

If you do try to convert something older you will get “Boot Failed. EFI SCSI Device”. I guess I was still too much in my VMware bubble where virtual machine hardware levels are supported for a much broader range of guest operating systems. Hands up. I should have RTFM’d. 🙂

Over the weekend I watchd Marylyn Monroe in the classic “Some like it hot”. Like her I keep getting the “fuzzy end of the lollipop” when it concerns Microsoft technologies.

Step1: Disable Recovery Environment

IMPORTANT: Don’t bypass this step. The process of putting back the BCD configuration if you don’t is too horrible to consider.

There are very tight dependencies between Microsoft’s “Boot Configuration Data” (or BCD) and the recovery environment – if it is enabled it can play havoc with this “conversion” process. So its recommend to disable this feature on the source VM, with reagent.exe command:

reagentc /disable

reagentc /info can be used to check if Windows RE is “enabled” or set to “1” which indicates its enabled. Use reagentc /info to confirm it has successfully disabled the Windows RE feature

Note: This was taken from Windows 2008 R2 machine, Windows 2012/8 reagentc will report Enabled/Disabled as the status.

Step2: Shutdown source VM; Mount VHD; Capture the image

Next we can shutdown the source VM gracefully, and then mount the VHD and capture it to a temporary location. To mount the VHD you will need to do this from a machine that has access to the same storage as the Windows Hyper-V 2012 host. By my reckoning the easiest place to do this is from one of the Windows Hyper-V hosts itself – as it is unlikely your management PC/system will have access to a .CSV volume in the cluster (although it could be shared using SMB if you need to).  This mounting and capturing process is carried out from a PowerShell environment. For some reason, in a clean install of Windows Server 2012 with the Hyper-V Role enabled, Microsoft does not install the Hyper-V Module for Windows PowerShell. This can be fixed by adding it using the “Add Roles and Features Wizard”. Without it you will find that commands like get-vm just aren’t found…

The Powershell cmdlet mount-vhd can be used to mount the VHD file to a drive letter (the next available drive letter by the looks of things).

mount-vhd –readonly C:\ClusterStorage\Volume1\web01\win2008R2_disk_1.vhd

The capturing process is carried out using the “Deployment Image Servicing and Management Tool” or DISM for short. In my case this mounting process created a drive letter alias of I: in File Explorer

dism /capture-image /imagefile:C:\ClusterStorage\Volume1\temp\web01.wim /name:”Web01-Captured” /capturedir=”I:\” 

Note: There is NO space between /imagefile: and C:\

This process took about 3 minutes to create 2.48GB WIM image file from a VHD that was 40GB in the dynamic format with about 7.5GB of data (the OS partition). The transfer was from FC SAN (2GB) to the local disk. This came down to 1min 30sec. The time required to complete this image management is going to vary based on the speed of your storage; how heavily utilized the storage fabric is, and the quantity of data to be imaged…. Factor that into your maintenance window…

Step 3: Create a new VHDX

The step involves a number of PowerShell commands to create a new VHDX (for the new Gen2 VM).  This includes not just creating the VHDX file, but mounting and enumerating (drive numbers) it such that you can go on to create a partition (using diskpart) table along with drive letter assignments within the VHDX itself.

# This part creates and mounts the new VHDX…

New-VHD C:\ClusterStorage\Volume1\temp\web01gen2.vhdx –dynamic –size 30gb

Mount-DiskImage C:\ClusterStorage\Volume1\temp\web01gen2.vhdx

# This part gathers disk information.

$Target = Get-DiskImage C:\ClusterStorage\Volume1\temp\web01gen2.vhdx

($Target | Get-Disk).Number

ls function:[d-z]: -n | ?{ !(test-path $_) } | select -last 3

Step 4: Partition in the GPT format

Right, with those variables (the disk number is 7) and X: Y: Z: are free drive letters we can create a diskpart “answer” file with those variables inputted to be used with DiskPart. Before you do this – check the disk numbering correctly – otherwise you could be zapping the wrong disk – and your disk will look very empty afterwards. This is something to be very cautious about, it’s a risk operation if you get the wrong disk and could result in data loss if you fat-finger the number.

This script creates 3 partitions on disk7 (ascertained from the earlier PowerShell script).

• X: drive for the Windows RE Tools (300)
• Y: drive which is used for the Boot Configuration Datastore (100)
• Z: Drive is where the WIM file is cloned too (Rest of the disk)

select disk 7

clean

convert gpt

create partition primary size=300

format quick fs=ntfs label=”Windows RE tools”

assign letter=X

set id=”de94bba4-06d1-4d40-a16a-bfd50179d6ac”

gpt attributes=0x8000000000000001

create partition efi size=100

format quick fs=fat32 label=System

assign letter=Y 

create partition msr size=128

create partition primary

format quick fs=ntfs label=Windows

assign letter=Z

list part

This script above is saved to a text file called web01gen2.txt, and used to automate the diskpart command like so:

diskpart /s C:\ClusterStorage\Volume1\temp\web01gen2.txt

 

I had a weird pop message when the script ran because I have the Disk Management console open at the same time. I think this one of those benign refresh issues that Windows presents occasionally.

Step 5: Clone WIM to VHDX; Configure BCD Store

The next step is to clone or restore the WIM capture that was created earlier, and then ensure the resulting disk is bootable. This carried out with the DISM utility, as well as the bcdboot command.  This clone process took about 1minute 30seconds to complete.

dism /apply-image /imagefile: C:\ClusterStorage\Volume1\temp\web01.wim /applydir:Z:\ /index:1

Z:\Windows\System32\bcdboot Z:\Windows /s Y: /f UEFI

Note: I actually had a problem with bcdboot command as outlined in John Howard’s blogpost. It just came back with a response as if the command syntax was not recognised. When I tested this against a Windows 2012 R2 machine it was successful.

 

However, I found if I moved into the path of Z:\Windows\System32 and ran the bcdedit utility from there then it would execute properly. I have a feeling this has something to do with the environmental paths in my system.

Step 6: Clean up and Unmount VHD/VHDX

Next we need to clean up the environment and unmount the .VHD/VHDX. The “Remove-PartitionAccessPath” cmdlet is used to clean out references to the Y: drive in the environment. Apparently, if you don’t carry out this task a Windows bug can affect you where the VM tries to use these temporary drive letters that we used to do this virtual disk management. To be honest I’m not sure I really understand this bug. To quote John Howard’s blog:

…we need to perform a workaround for a bug in Windows to avoid the drive letter for the ESP on the new VHDX persisting after an unmount. (If you don’t do this, the drive letter will persist and you’ll have to reboot your parent partition to get rid of it).

Remove-PartitionAccessPath –DiskNumber 7 –PartitionNumber 3 –AccessPath Y:\

Dismount-DiskImage C:\ClusterStorage\Volume1\temp\web01gen2.vhdx

Dismount-vhd C:\ClusterStorage\Volume1\web01\win2008R2_disk_1.vhd

Step 7: Attached to new Gen2 VM

During my conversion process I created a temp location for working with the files. So one thing I need to do was first create a new Gen2 VM, and then relocate the new VHDX to that directory – so I would keep all my files together.

 

Next, I needed to make sure all the setting that had been applied to the previous Gen1 VM was applied to the Gen2 VM.  On top of the usual suspects of CPU, memory (and also those confusing Dynamic Memory settings!!!) and network assignments this included:

• Enabling a cloud capability
• Enabling High Availability
• Enabling Hyper-V Replica support

Note: I guess I could use the hardware profile to apply these settings assuming that a.) They are configured b.) They match the converted VMs settings – after all it could have been customized.

Once the Gen2 VM had been created I went back to the Windows Hyper-V 2012 host and deleted the VHDX created with the wizard (you can do this as single action for SCVMM both removing the VHDX reference as well as deleting if from the host), and then I drag-and-dropped the converted VHDX into the VM folder…

Next I used the big green + New button in SCVMM to indicating I wanted to “Use a local virtual hard disk available on the host” to browse for the VHDX and setting the option indicate that the VHDX “Contains the operating system for the virtual machine”.

Step 8: Test & Power On/Clean Up

I was successful in powering on my converted my machine. I did keep hold of my old Gen1 VM (in fact I deleted my Gen2 VM) so I could test this Powershell script method. It did occur to me you would have some clean up to do. You’d need to delete the temporary WIM and diskpart.txt file (although I think this could be reused for other conversions), and also delete or archive the original Gen1 machine in case at some later stage a Gen2 issue came up unexpectedly…

Step 9: Re-enable Recovery Environment

Once you are satisfied the converted VM boots – then you can re-enable the BCD recovery environment with:

reagentc /enable

reagentc /info

reagentc /boottore

 

Note: No, This is not a typo. The command switch is /boottore not bootstore! The boottore switch forces Windows RE to start automatically at the next boot. 

A reboot after enabling the recovery environment will cause Windows 2012 R2 to boot to the Windows RE UI. You can bypass this using the option “Continue”

Scripted Conversions

John Howard has very kindly written a Powershell script that automates (in part) this process. It is held on the MSDN site. However, as John says this isn’t supported by Microsoft officially:

Although I work for Microsoft and am a Program Manager in the Hyper-V team, I must point you to the disclaimer on my blog, the disclaimer in files associated with this project, and the license conditions at the top of this page before use. Convert-VMGeneration.ps1 and any associated files are provided “as-is”. You bear the risk of using it. No express warranties, guarantees or conditions are provided. It is not supported or endorsed by Microsoft Corporation and should be used at your own risk. http://code.msdn.microsoft.com/Convert-VMGeneration-81ddafa2#content

There’s a couple of unsupported scenarios that you can find out from using get-help from the .ps1 script like so:

• COM ports; Floppy; Empty and physical DVD drives; Legacy Network Adapters; RemoteFX; .VHD disks are skipped
• VMs with checkpoints cannot be converted
• VMs with multiple windows partitions on the source boot disk cannot be converted
• Additional data partitions on the source boot disk are not converted (although additional disks are)
• 32-bit guest operating systems cannot be converted
• Disable Hyper-V Replica on the VM if enabled
• Commit any checkpoints on the VM

I copied the ps1 file to the root C:\ on one of my Windows Hyper-V 2012 R2 servers and executed like so:

Convert-VMGeneration.ps1 -vmname “web01-2012R2-Gen1” -path “C:\ClusterStorage\Volume1”

Note: One small bug I found was the VM has to be on the same Windows Hyper-V 2012 host that you’re running the command from. If it’s on a different Windows Hyper-V 2012 host it comes back with a message saying it cannot find the VM.  Also the script supports a –wim switch to stop the .vim file being created in a local temp directory.

This process did indeed create a new Gen2 VM as promised, and it did power on – all I had to do after this was enable Hyper-V Replica and re-enable the Windows RE options.

Note: This Powershell script is running outside of SCVMM, so occasionally the newly created Gen2 VM doesn’t appear. You have to use the “Refresh Virtual Machines” option for it to appear if this happens. I measured the total time the process took with the Script – about 6-7minutes per VM. That’s not including disabling & enabling the Window RE process, and other steps not covered by the script.

I was curious to know if this script would convert all the drives mounted to the VM, or whether it would just remap the disk1 of Gen1 VM to Gen2 VM. After all the main reason for this conversion is to make the Gen2 bootable. I was also interested to know how much time this would take. So I added a second VHD to my VM, formatted and created a BigDataFolder (Its folder full of .bmp files – deliberately picked because image files tend not to compress very well). Of course, I’m being ironic here – out of 40GB dynamic disk, I populated it with about 33GB of data.

Using the Powershell script I found it correctly detected the OS partition and didn’t convert this BigDataDisk (which I think is a good thing!). The copying the Gen1 VM settings for the new Gen2 does successful map the disk1 to the Gen2 VM – however, it’s located in the original directory structure of the Gen1 VM. So you would need to relocate and remap it to the Gen2 VM before deleting the old Gen1 VM. Additionally, although the disk is mapped to the Gen2 VM, it doesn’t appear in File Explorer. This disk needs to be brought “online” with the Disk Management snap-in.

The big thing here is woe betide anyone who store large quantities of data on disk0/C: you will be facing a bigger conversion time on your hands. For shits and giggles (as we are fond of saying in the North-East of England), I thought it would be interesting to move my “BigDataFolder” to the C: drive until it was almost full, and then see how long that conversion took. For 40GB disk, with about 35GB of data it was about 15mins.

Conclusions

Well, as my Dad would say, “What a palaver”¹. I don’t imagine for one moment that Hyper-V users will carry out the manual tasks. They are going to use the script because that’s quicker. When they do so – they are doing that at their own risk.

It strikes me more effort was put into creating the new Generation 2 virtual machine, than thinking about how customers would migrate to the new format. As far as I can see customers are just better off not bothering with the conversion at all. Unless they can see an overwhelming and compelling case for Gen2 VMs OR alternatively they could just switch to virtualization platform that doesn’t come with these nutty limitations and weird process. I wonder who that could be? 😉

1. The Cambridge Dictionaries Online describes a palaver as “unnecessary work and trouble”