Welcome Back… to my second part of creating a Pay-As-You-Go backed Organizational vDC… Isn’t funny how you can construct a sentence that makes perfect sense to you, but to the outside world they’d be all WT*!
7. Select a network pool for this Organization vDC
Note: Remember this is a pool that is going to be used soley and exclusively by this “Test/Dev” Organization vDC within the CorpHQ Organization. That’s quite different from the Organizational Network that will be created at the end of this wizard, which will be accessible to ALL Organization vDCs. The vApps created in the Test/Dev vDC will backed by network pool using vCD-NI, but the external interface of the vCNS will be backed by one of my “External Networks” which will use VLAN tagging.
Earlier in this series – I created number of different networks pools – VLAN, vCD-NI and VXLAN backed pools. All of these allow the dynamic creation of networks whether it be by ye olde worlde “VLANs”, or by the more blisteringly modern methods of vCD-NI and VXLAN. In my mind VLANs are precious resource, not because I value them – but that my physical switch can create so few of them (64 VLANs maximum to be precise). With any resource that’s scarce, I’m mindful of allocating them indescriminately. In the case of a test/dev Organization vDC I think this is ideal use case for either vCD-NI or VXLAN based allocations – mainly because I expect a Test/Dev environment to create/destroy many networks in a given period, and they are unlikely to need direct connected configurations. My only issue with this configuration is the size of my vCD-NI pool was tiny in comparison to the default quota of 1000 (in fact all my allocations of network pools are tiny compared to the quotas). The alarm didn’t stop me from clicking next, and I assume if I’d left this unaddress I would have allocated all 100 vCD-NI networks to this Organization vDC. The easiest solution was to reduce the quota, and then perhaps reconsider the allocation of vCD-NI backed networks. They dont’ “cost” me anything, so why did I make the allocation so low of 100?
8. Configure the Edge Gateway…
When you run through this wizard for the 1st time for an Organization there is no vCNS Edge Gateway (the artist formerly known as vShield Edge) or any 3rd party network services registed to select. The next part of the wizard allows you to indicate whether you would like to deploy an vCNS Edge Gateway, and connect the Organization to an external network. The setting is optional, so you could bomb out of the wizard and create an Organization Network and other allocations seperately if you so wish. I think it makes more sense to persist with the wizard if your setting up an Organization vDC for the first time.
When you enable the option to “Create a new edge gateway” you will see the page refreshes to show additional options, additionally if you tick off any of the “Advanced Options” (say Configure IP setings for example) this adds these additional steps in the wizard on the left-hand side bar. In the configuration your asked to set the “Edge Gateway” name as it will appear in the inventory of both vCD and vCenter. I’m likely to have at least one per Organization, so I used the Organization prefix to enforce some kidn of uniqueness. My first gateway I will used to connect the Organization to the Internet (outbound only) hence the descritpion.
It’s possible to have to different sizes of Edge Gateway, the functionality is the same – and you can easily switch after the deployment from compact to full and vice-versa. In fact if I recall rightly, there are actually more than two sizing options in the UI of the vCNS Manager. As this is just a lab environment and a modest load is going to be placed on the Edge Gateway I opted for using the compact addition.
The vCNS Edge Gateway now comes in a high availability mode where you have two appliance which are in a active/passive configuration and mirror of each other. This would be ideal in a production environment, and definitely something I want to experiment with in the future. But for now a single vCNS Edge Gateway is all I really require.
Finally, I selected the options to configure both the IP of the appliance and the range of IP address it will service on the network. I didn’t both with setting a rate-limit option as I don’t consider this a priority in a lab environment, but again its something I want to go back and look at.
9. Select the External Network that the Edge Gateway will use, and select the Add button
This is relatively trival page in the wizard with one exception (for me at least) which is the option to “Use default gateway for DNS Relay”. I still haven’t really decided if DNS per-Organization should reside within the Organizational vDCs or where it should be elsewhere. Part of me wants LDAP/DNS to managable within the scope of their own Organizations. That’s because in my mind I want the Organization and its Virtual Datacenters to have the same management and functionality as if it was a physical environment. The other side of me thinks this is a bad idea. Tenants shouldn’t have to worry about the configuration/management of “infrastructure” systems like LDAP/DNS. Those should be externally provided, with the option of them managing outside of their cloud – for example for adding new users and computers into their environment. Additionally, given those LDAP services are required for them to be authenticated to gain access to vCloud Directory and their Organization – there’s something of chicken-and-egg anxiety for me there. If LDAP/DNS works inside their Organization – they could shut it down, and find themselves unable to login to their vCD Organization to power it back on! [Unless, of course there is a local user account for their Organization]. If you don’t enable this option here – you cannot use it later on in the wizard when you create an Organization Network. If you do enable it you have the choice of using OR not using later on in the “Create Organization VDC Network”, so I enabled it to give me the choice but actually made my primary DNS the servers I have on the external network hosting the main “Corp.com” domain.
10. Configure IP Settings…
The next step is to configure the EXTERNAL interface of the Edge Gateway. This is the NIC that will sit on the “External Network” I’ve called “The Corporate Network” in vCD which actually points to a portgroup on the Distributed vSwitch called “ExternalCorpNetwork”. You have two choices here – to either have vCD set the IP automatically from the pool of IP addresses assigned to the External Network OR to manually assign a IP address from the same pool. On my external network I have a Juniper Firewall, and at some stage I want to allow TCP ports outbound/inbound through it and then on to the Edge Gateway. To do that I’m going to have to use an IP address that doesn’t change, so it can be added to the rules on the Juniper. I can see that there will be cases where I might want to dynamically assign an IP from a pool. For example if I was creating a lot of networks in test/dev environment to internal corporate network, but when it comes to internet access for now I want to be bit more controlled in assigning these IP addresses. In my case I selected “Manual” and assigned the IP of 192.168.3.51 to the Edge Gateway from the pool. You’ll notice there’s no option here to set a subnet mask, default gateway – that’s because that was set on the “Externel Network” configuration earlier.
11. Sub-Allocate IP Pools on Edge Gateway…
NOTE: One thing I discovered later was that this IP sub-allocation pool range can actually INCLUDE the Edge Gateway external IP address. In my case the 192.168.3.50 address.
It is required to allocate a bundle of IP address to the Edge Gateway for other processes such as NAT or Load-Balancing, where additional IP address maybe needed. I decided to carve out a block of IP address from 192.168.3.51-192.168.3.57. Lets just unpack that last statement. These IP address are need for other services such as NAT or Load-Balancing – what that means is the IP addressed assigned at the “Configure IP settings” (in my case 192.168.4.1) isn’t used for NATing. So merely the act of having an external (192.168.3.51) and an internal address (192.168.4.1) isn’t enough to get traffic moving from the vApps on the External Network. As these services are enabled for the first time an IP address will be claimed.
IMPORTANT: So by default the vCNS Edge Gateway passes NO traffic from the vApp to the External Network. There are no NAT rules enabled for that to happen, and there are NO Firewall rules to allow communications outbound. All the vApps can do is speak to each other on the same Organization Network.
12. Create Organization vDC Network…
NOTE: If you read on you might find that actually enabling the “Use Gateway DNS” is a better option!
Note: This is Organization Network that’s been created here for the very first time. By selecting the “Share this network with other vDC’s in the Organization” it will be available not just to this “Test/Dev” Organization vDC, but ALL Organizational vDCs. Remember “external networks” (in my case called “The Internet” are backed by portgroups on the Distributed vSwitch that are manually created and assigned an VLAN ID before you even create them in vCD. One of the downsides of enabling the option to create a new Edge Gateway is that two quite seperate wizards are affectively being chained together – one creates a network for the Organizational vDC and thothe rcreates a network for the Organization. That’s two quite seperate discrete objects within vCloud Director. Now the important thing to remember here is this page is primarily about configuring the options for the “Internal” network – the vApp side of the network, which will speak to the external network via the vCNS Edge Gateway. Essentially, these pages are specifying the parameters that the vCD Edge Gateway appliance will need to allow VMs inside the Organization vDC speak to the wider network.
The next step involves creating an Organizational Network for the first time. Again I used my Organizational name as prefix for the name of the network, and enabled the option to “Share this network with other vDC in the Organization”. Without that option available the Organizational Network would be only available to the Organization Virtual Datacenter I’m creating here. I was quite pleased to see this option. It means each vDC could have its own discrete set of networking options only available to it.In this case I set the internal interface of the vCNS to be 172.168.4.1 with a subnet mask of 255.255.250.0, rather than using the 172.168.4.1 interface as the “DNS Gateway”, I specify the Primary/Secondary DNS manually together with the DNS Suffix. Looking back at this configuration I think it’s a mistake. By which I don’t mean its “technicall wrong” but a mistake in that I’m missing out on a good feature. Without the “Use Gateway Address for DNS” option enabled it means every VM in a vApp that recieves an IP address from the Edge Gateway would be configured for the DNS servers (192.168.3.130/192.168.3.131). What if that changes? What if the IP of the DNS altered? Why “hard-code” a IP configuration like this. It could make the vApp less portable in the sense that it could potentially be looking for these DNS servers if it was relocated elsewhere… Fortunately, the option can be re-enabled and the configuration of the vCNS Edge Gateway updated…
If you scroll down beyond the “DNS Suffix” options you will see the option for a “Static IP Pool” [if your working on small screen you may have to scroll if you working on a large screen you might not!]. This is used for the VMs inside the Test/Dev Provider vDC. There’s a couple of way of enabling them for IP. We could turn on the vCNS DHCP functionality, we could ask the vApp owners to manually specify the IP address within the Guest Operating system – OR we can create a Static IP Pool. This will give a VM in a vApp an IP address from the pool if the .OVF is configured claim one. This will mean a VM in a vApp would be assigned an IP address of 172.168.4.10 with subnet mask of 255.255.255.0 with its default gateway being 172.168.4.1 and the DNS servers being 192.168.3.130 and 192.168.3.131.
NOTE: Actually I ended-up using the 172.168.5.x range in my deployment. I realised later that 172.168.5.x is used elsewhere on my network – although they aren’t visable to each other, I didn’t want any overlapping networking ranges.
13. Finally, name the Organization vDC and set a description. You may wish to set this as “disabled” until you are happy with the configuration. For example if you were wanting to use the vCNS Edge Gateways functionality, rather than static pools – you might prefer to disable it until it has been correctly configured.
Some Pay-As-You-Go Thinking:
[This title is meant to be some thinking about PAYG, but actually quite like the idea of PAYG as being type of thinking… Thinking-As-You-Go TAYG]
So…. lots of settings. Where to begin? So here’s some thinking. In my experience of running a lab environment you rarely if ever run out of CPU. It’s memory every time. It kind of feels “odd” to place controls on CPU. There appears to be two compelling controls in a Pay-As-You-Go model either imposing a memory quota or VM quota. The VM quota seems draconian. Why shouldn’t developers be allowed to create as many VMs as they like – they could be making lots of very small ones after all. My worry is those developers creating lots and lots of VMs, and consumming all the memory available in the Provider vDCs, as the expense of other tenants. The other concern I have is wonderful that all these controls are – it’s always been my view is the knobs & buttons you press and engage – the hard it becomes to work out the affective control. That means if we set the barriers too high, and stop VMs being created too early because of soft setting – it might be difficult to see how lift that barrier. Finally, although I like the “typical number of vApps” estimatation (as it illustrates well the effect your changes in the PAYG configuration) it might not be a very best representation of how the resources will be consummed in the real world. As the System Admin of the vCD environment, and the creator of the Organization vDC do you really know how many small, medium or large VMs will be created or when?
There’s some temptations here which I’m trying to resist. For example, as I have 4 Organizations potententially using the “Silver” vDC – I could take the total resources available in the cluster (48GB), and simply divided – set a memory quota of 12GB to each Organization – in other words it would be like a quota of 1-host per Organization (its four node cluster). A second example would be just think (in my general experience) what the average number of VMs I usually see on a 12GB host before I see memory alarms – that’s usually about 8-9 VMs sat there doing nothing – so I could limit the number of VMs to 10 per “Test/Dev” Organization. The trouble with both of these approaches is that assumes each Organization will create/consume about the same about of Test/Dev VMs and they will be roughly the same size. I don’t think I can guarantee either of these assumptions. What I do know is that could regarded as an inefficent use of my resources, and I’m not really pushing the “over-commitment envelope” as much I could. So I’m going to opt for memory quota that is more intune with that way of thinking. The PAYG model added extra 30GB to its calculations – allowing almost the double the over-commit against memory available in the cluster. So I’m going to use that as my guide. I’m going to give memory quota to each Test/Dev Organizational Virtual Datacenter – 20GB each.
Finally, there’s another thing I’ve been thinking and its about the new Organization wizard, and enabling the vCNS Edge Gateway in the same workflow. It does make that wizard a very long one – I’m wondering if my life and this blogpost would have been simplier if I’d just created the Organizational vDC first, and then configured the network second. Splitting the process up into two discrete phases. I have a feeling that this ability to both create an Organization vDC and configure its networking at the same time, was added in vCD 5.1. That’s because my lab guide instructions for 1.5 make no mention of it. I guess for more experienced users this makes sense for them, but for noobs like me its a bit overwelming. The other thing that concerns me is the placement of the first vCNS Edge Gateway to an Organization. If like me you create a “Test/Dev” vDC first – and present to it your Silver Provider vDC and just Silver/Bronze storage – the Edge Gateway for the Organization is placed there as well. What if I wanted this appliance to be on the Gold Provider vDC and on Platinum Storage. The way to do that would be to handle the creation of the Organization Networking as seperate process. I’d be happy for my vApp Network based Edge Gateway to reside in the SAME cluster as the vApps (that makes sense to me), but with the Organizational based vCNS which is used by both the Gold and Silver Provider vDC it feels like I should be locate it where I like…