GeekAfterFive - Infrastructure as Code

home

Deepdive: vCloud vApp Networks

24 Apr 2012

There are three network "fence modes" (connection modes) in vCloud Director: Isolated, Bridged, and Nat Routed. These apply to both Org and vApp networks, but for now we are talking in the context of vApps. We'll also talk about the vApp network features, courtesy of vShield Edge: DHCP, Firewall, Static Routing, and NAT.

A quick note about terminology. The use of terms like "fence mode" are what you will see in the API. When applicable, I will refer to the Web UI term (in parenthesis) or with a screenshot.

Isolated

Definition: Isolated networks are completely separate networks. By themselves, there is no connection or communication to other networks. Because they are completely isolated, you can do normally frowned upon things like reuse the same subnet over and over in different isolated networks. ;)

Usage: Isolated networks are great for back-end communication, such as database traffic. In a web application, for instance, you could give your web servers a second interface on the isolated network that database server resides on. Another use for an isolation network is not isolation at all, but providing your own routing device. Your routing appliance could have one interface on the isolated vApp network, and then another interface on an Org level network. If you wanted features such as dynamic routing, VPN to a specific vApp, Web Application Firewall, etc, this would be the way to go.

Features: Because of their isolation, the only available vShield Edge feature is DHCP. Theoretically speaking, if DHCP were enabled and a vShield was instantiated, static routing should be available, but it's not. Firewall and NAT are not available because they both require connections to another network via vShield Edge.

Bridged

Definition: Bridged simply means you are directly connected to an Org network. In fact, the vCloud Web UI calls the connection "Direct" and shows up as the Org Network you are directly connected to.

Usage: Bridged (direct connected) networks are commonly used for VMs that need to be accessed from anywhere within your Organization, such as DNS or directory services.

Features: None (at the vApp level). The Org level of this network may have feautures available, depending on it's type.

NatRouted

Definition: Connecting a vApp network to an Org network results in a 'natRouted' mode. This automatically invokes the creation of a vShield Edge appliance to connect the two different networks. The appliance has an internal and an external interface. The internal interfaces is on the vApp network, and the external on the org network.

Usage: NatRouted networks are required for firewall, NAT, and static routing services provided by vShield Edge. Typical usage includes securely publishing applications to the organization or Internet, but there is also the concept of "fencing," Fencing a network essentially hides a VM's MAC and IP addresses from the rest of your Virtual Datacenter. As long as each VM has a NAT address to the Org network, you can have duplicate IP addresses internally in the vApp because it is "hidden" by the vShield Edge device. This won't work if you are using static routing.

Features: All of 'em! DHCP, Firewall, NAT, Static Routing

More on fenced mode...

There is a checkbox in the UI to "Fence vApp." This option takes a 'bridged' network, and turns it into 'natRouted.' It will show the network as "Fenced" in the UI. The VMs within this vApp still have Org Network IPs, but a vShield device is placed in between the VMs and the Org network. So essentially, you have duplicate subnets on the internal and external sides of the vShield Edge device. As a network guy, this gives me heartburn. Here's a screenshot to demonstrate a potential problem using the "fence vApp" option:

Basically the "test (fenced)" VM will never be able to communicate with "test" with the same IP address of 192.168.1.102. The reverse is not true because you will be communicating to the external "test (fenced)" address of 192.168.1.105.

My opinion? If you want to mask IP and MAC addresses, just use a natRouted type. No need to complexify when there is a perfect solution available.

Managing your vApp networks with PowerCLI

Bust out your vCloud API Schema Reference, because we're going to be playing with the NetworkConfigSection of the vApp! To get you going, you'll want to retrieve the NetworkConfigSection of the vApp, which can be done like so:

$networkConfigSection = (Get-CIVapp MyVApp).extensiondata.GetNetworkConfigSection()

Now we can start to explore. Using the schema as our guide, we can drill down to actual config.

$networkConfigSection.NetworkConfig[0].configuration

Note the index at the end of NetworkConfig. Even if you only have 1 network in your vApp, you still have to specify an index of the collection or pipe to a where-object {$_.networkname -eq "NetworkName"} FenceMode specifies the vApp network type, which can be one of: isolated, bridged, natRouted ParentNetwork specifies the Org network that the vApp network is connected to. This doesn't apply to isolated networks, can you guess why? ;) Features contains the DHCP, Firewall, NAT, and Static Routing configuration IPScope contains the gateway, subnet, DNS, and IP Pool. Pretty much all of this is modifiable (refer to the schema reference). Simply assign a new value to something and then run:

$networkConfigSection.updateServerData()

Let's create some new networks:

Isolated

$mynetwork = new-object vmware.vimautomation.cloud.views.vappnetworkconfiguration
$mynetwork.networkName = "MyIsolatedNetwork"

$mynetwork.configuration = new-object vmware.vimautomation.cloud.views.networkconfiguration
$mynetwork.configuration.fencemode = "isolated"

$mynetwork.Configuration.IpScope = new-object vmware.vimautomation.cloud.views.ipscope
$mynetwork.Configuration.IpScope.gateway = "192.168.3.1"
$mynetwork.Configuration.IpScope.Netmask = "255.255.255.0"
$mynetwork.Configuration.IpScope.Dns1 = "8.8.8.8"

$mynetwork.Configuration.IpScope.ipranges = new-object vmware.vimautomation.cloud.views.ipranges
$mynetwork.Configuration.IpScope.IpRanges.IpRange = new-object vmware.vimautomation.cloud.views.iprange
$mynetwork.Configuration.IpScope.IpRanges.IpRange[0].startaddress = "192.168.3.100"
$mynetwork.Configuration.IpScope.IpRanges.IpRange[0].endaddress = "192.168.3.200"

$networkConfigSection = $vapp.ExtensionData.GetNetworkConfigSection()
$networkConfigSection.networkconfig += $mynetwork
$networkConfigSection.updateserverdata()

Bridged

$orgNetworkName = "MyOrgNetwork"
$orgNetwork = (Get-Org).extensiondata.networks.network | where {$_.name -eq $orgNetworkName}

$mynetwork = new-object vmware.vimautomation.cloud.views.vappnetworkconfiguration
$mynetwork.NetworkName = $orgNetworkName

$mynetwork.configuration = new-object vmware.vimautomation.cloud.views.networkconfiguration
$mynetwork.configuration.fencemode = "bridged"

$mynetwork.Configuration.ParentNetwork = New-Object vmware.vimautomation.cloud.views.reference
$mynetwork.Configuration.ParentNetwork.Href = $orgNetwork.href

$networkConfigSection = $vapp.ExtensionData.GetNetworkConfigSection()
$networkConfigSection.networkconfig += $mynetwork
$networkConfigSection.updateserverdata()

natRouted

$orgNetworkName = "MyOrgNetwork"
$orgNetwork = (Get-Org).extensiondata.networks.network | where {$_.name -eq $orgNetworkName}

$mynetwork = new-object vmware.vimautomation.cloud.views.vappnetworkconfiguration
$mynetwork.networkName = "MyNatRoutedNetwork"

$mynetwork.configuration = new-object vmware.vimautomation.cloud.views.networkconfiguration
$mynetwork.configuration.fencemode = "natRouted"

$mynetwork.Configuration.ParentNetwork = New-Object vmware.vimautomation.cloud.views.reference
$mynetwork.Configuration.ParentNetwork.Href = $orgNetwork.href

$mynetwork.Configuration.IpScope = new-object vmware.vimautomation.cloud.views.ipscope
$mynetwork.Configuration.IpScope.gateway = "192.168.3.1"
$mynetwork.Configuration.IpScope.Netmask = "255.255.255.0"
$mynetwork.Configuration.IpScope.Dns1 = "8.8.8.8"

$mynetwork.Configuration.IpScope.ipranges = new-object vmware.vimautomation.cloud.views.ipranges
$mynetwork.Configuration.IpScope.IpRanges.IpRange = new-object vmware.vimautomation.cloud.views.iprange
$mynetwork.Configuration.IpScope.IpRanges.IpRange[0].startaddress = "192.168.3.100"
$mynetwork.Configuration.IpScope.IpRanges.IpRange[0].endaddress = "192.168.3.200"

# With no firewall
$mynetwork.Configuration.features += new-object vmware.vimautomation.cloud.views.firewallservice
$mynetwork.Configuration.features[0].isenabled = $false

$networkConfigSection = $vapp.ExtensionData.GetNetworkConfigSection()
$networkConfigSection.networkconfig += $mynetwork
$networkConfigSection.updateserverdata()

Note on the above, I explicitly told it to create the natRouted network without a firewall. By default, it will enable the firewall, but we'll talk about in the services section.

Removing Networks

(Extra handy if you want to remove an Org network reference from a vApp!)

$removeNetworkName = "MyNetwork"
$networkConfigSection = $vapp.ExtensionData.GetNetworkConfigSection()
$networkConfigSection.networkconfig = $networkConfigSection.networkconfig | where {$_.networkname -ne $removeNetworkName}
$networkConfigSection.updateserverdata()

Services

You'll find the services in the features collection here:

$networkConfigSection.networkConfig[0].configuration.features

Here's what the natRouted network features look like in Powershell. I did a gettype() afterwards to show you the object types for each service.

The Firewall Service This is really handy if you have a spreadsheet full of rules, or want to add an entire range of IPs (which The vCloud Director UI and API do not support that at this time.) As stated above, the firewall service is added by default in a new natRouted network. By default it's set to deny all inbound.

$fwService = New-Object vmware.vimautomation.cloud.views.firewallservice
$fwService.DefaultAction = "drop"
$fwService.LogDefaultAction = $false
$fwService.IsEnabled = $true

#Add two rules
$fwService.FirewallRule = New-Object vmware.vimautomation.cloud.views.firewallrule
$fwService.FirewallRule += New-Object vmware.vimautomation.cloud.views.firewallrule

#First Rule
$fwService.FirewallRule[0].isenabled = $true
$fwService.FirewallRule[0].description = "Web In"

$fwService.FirewallRule[0].protocols = New-Object vmware.vimautomation.cloud.views.firewallRuleTypeProtocols
$fwService.FirewallRule[0].protocols.tcp = $true

$fwService.FirewallRule[0].policy = "allow"
$fwService.FirewallRule[0].port = "80"
$fwService.FirewallRule[0].destinationIp = "192.168.2.10"
$fwService.FirewallRule[0].sourceport = "-1"
$fwService.FirewallRule[0].sourceip = "Any"
$fwService.FirewallRule[0].direction = "in"

#Second Rule
$fwService.FirewallRule[1].isenabled = $true
$fwService.FirewallRule[1].description = "SSH In"

$fwService.FirewallRule[1].protocols = New-Object vmware.vimautomation.cloud.views.firewallRuleTypeProtocols
$fwService.FirewallRule[1].protocols.tcp = $true

$fwService.FirewallRule[1].policy = "allow"
$fwService.FirewallRule[1].port = "22"
$fwService.FirewallRule[1].destinationIp = "192.168.2.10"
$fwService.FirewallRule[1].sourceport = "-1"
$fwService.FirewallRule[1].sourceip = "Any"
$fwService.FirewallRule[1].direction = "in"

If you are adding the firewall service to a new network, just paste the above code into the creation code. If you are adding to an existing network, you'll need some additional code. Warning, this currently deletes the old ruleset. This was meant to add firewall service where there was none before:

$vappName = "My vApp"
$vappNetworkName = "My vApp Network"
$networkConfigSection = (Get-CIVApp $vappName).extensiondata.GetNetworkConfigSection()
$vAppNetwork = $networkConfigSection.NetworkConfig | where {$_.networkName -eq $vappNetworkName}
$vAppNetwork.Configuration.Features = $vAppNetwork.Configuration.Features | where {!($_ -is [vmware.vimautomation.cloud.views.firewallservice])}

# Remove the NAT service if there is nothing in there (auto mapped), otherwise the update will error out. :(
if (!($vAppNetwork.Configuration.Features | where {$_ -is [vmware.vimautomation.cloud.views.natservice]}).natrule[0].description)
{
 $vAppNetwork.Configuration.Features = $vAppNetwork.Configuration.Features | where {!($_ -is [vmware.vimautomation.cloud.views.natservice])}
}

# Add our new ruleset
$vAppNetwork.Configuration.Features += $fwService
$networkConfigSection.UpdateServerData()

The DHCP Service

$dhcpService = New-Object vmware.vimautomation.cloud.views.dhcpservice
$dhcpService.DefaultLeaseTime = "3600"
$dhcpService.MaxLeaseTime = "7200"
$dhcpService.IsEnabled = $true

$dhcpService.IpRange = New-Object vmware.vimautomation.cloud.views.iprange
$dhcpService.IpRange.StartAddress = "192.168.2.2"
$dhcpService.IpRange.EndAddress = "192.168.2.50"

Like the firewall service, DHCP is automatically added to a new a new network, but is set to disabled. If you want to use this code to add DHCP to an existing network, you'll just need to modify the existing service, or remove the old service and add this one. You will also need to omit the NAT service section when updating an existing network or you'll encounter an error.

The Nat Service

If the NAT service box is checked (and it is by default), then the NatService will already be enabled and the NatService object will be there.

The NatType allows you to pick: ipTranslation and portForwarding. You must choose one or the other. :P

There is also a policy, which lets you choose what traffic will NAT (all, or inbound only): allowTraffic, or allowTrafficIn

Unfortunately, the NAT service is not yet fully implemented in the .NET SDK/PowerCLI, so you'll have to go straight REST API on this one...

Before we do, it's important to know the different NAT rule types:

OneToOneBasicRule: This simply maps an internal IP to an External IP. This is what I typically think of as a Static, or 1:1 NAT.

OneToOneVmRule: This maps a specific VM Nic to an external address. This is helpful if you change your internal IP and don't want to fiddle with NAT rules at the same time.

PortForwardingRule: Also a simple mapping rule to map a port on the external IP to an internal IP address.

VmRule: Odd naming, but it's port forwarding to the VM Nic, rather than an IP address.

Ok, so let's take a peek at our NAT rules using Powershell with the help of System.Net.Webclient:

$vApp = Get-CIVApp "vapp network testing"
$vAppNetworkName = "natRouted"

$netConf = $vapp.ExtensionData.GetNetworkConfigSection()
$netConfUri = $netconf.Href
$netConfType = $netconf.Type

$webclient = New-Object system.net.webclient
$webclient.Headers.Add("x-vcloud-authorization",$netconf.Client.SessionKey)
$webclient.Headers.Add("accept",$netConfType)
[xml]$netConfXML = $webclient.DownloadString($netConfUri)
&lt;pre&gt;

Basically, we just downloaded the XML representation of the NetworkConfigSection, bypassing the .NET SDK and communicating directly to the vCloud API. REST APIs ROCK.

We can now drill down to our Nat Rules:

($netConfXML.NetworkConfigSection.NetworkConfig | where {$_.networkname -eq $vAppNetworkName}).configuration.features.natservice.natrule

This will return something like:

You'll need to drill down one level further to view the rules. I am going to use a foreach to show all of them:

($netConfXML.NetworkConfigSection.NetworkConfig | where {$_.networkname -eq $vAppNetworkName}).configuration.features.natservice.natrule | foreach{$_.vmrule}

...and you'll end up with something like this:

So, now we can GET our NAT rules, how to we SET them? We have to get a bit more complicated and use [System.Net.Webrequest]. This is an example of creating a new NAT rule for port translation. OneToOne rules will be a bit different, so again, this is purely for example. If you need something specific and can't figure it out, I'd be glad to help.

$NatPort = "53"
$natRule = ($netConfXML.NetworkConfigSection.Networkconfig | where {$_.networkname -eq $vAppNetworkName}).configuration.features.natservice.natrule[0].clonenode(1)
$natRule.VmRule.ExternalPort = $NatPort
$natRule.VmRule.InternalPort = $NatPort
($netConfXML.NetworkConfigSection.Networkconfig | where {$_.networkname -eq $vAppNetworkName}).configuration.features.natservice.appendChild($NatRule)

$request = [System.Net.WebRequest]::Create($netConfUri);
$request.Headers.Add("x-vCloud-authorization",$netconf.Client.SessionKey)
$request.Accept="application/vnd.vmware.vcloud.task+xml"
$request.Method="PUT"
$request.ContentType = $netConfType

$postData = $netConfXML.outerxml
#adding strings to xml puts in silly newlines.
$xmlString = $postData.replace("&#xA;",")
[byte[]]$xmlEnc = [System.Text.Encoding]::UTF8.GetBytes($xmlString)
$request.ContentLength = $xmlEnc.length
[System.IO.Stream]$requestStream = $request.GetRequestStream()
$requestStream.write($xmlEnc, 0, $xmlEnc.Length)
$requestStream.Close()
$response = $request.GetResponse()
$responseStream = $response.getResponseStream()
$streamReader = new-object System.IO.StreamReader($responseStream)
$result = $streamReader.ReadtoEnd()
$streamReader.close()
$response.close()

The StaticRouting Service The Static Routing service controls routing within the vShield Edge.

$routeService = New-Object vmware.vimautomation.cloud.views.staticroutingservice
$routeService.IsEnabled = $true

$routeService.StaticRoute = New-Object vmware.vimautomation.cloud.views.staticroute
$routeService.StaticRoute[0].name = "test"
$routeService.StaticRoute[0].network = "192.168.1.0/24"
$routeService.StaticRoute[0].nexthopip = "192.168.2.1"
$routeService.StaticRoute[0].interface = "External"

Again, if you are adding this to an existing network, you'll need to omit the Nat Service, or you will encounter an error.

Wrap up...

WHEW, that's all I have for now. I like answering questions, so please post in the comments or get me on twitter.