This post was written in reference to the following platforms and versions
XenServer 5.x
XenServer 6.x
If you have just installed XenServer for the first time, or you have many XenServer installs under your belt you have obviously dealt with creating NIC bonds. There are a few things to think about when creating NIC bonds:
Does this bond include management interfaces?
What type of traffic is this bond going to host?
Does this bond connect to your SAN over iSCSI?
Types of Bonds
There are 2 types of bonds, active-active and active-standby. Active-active bonds are the default when you create a bond (you can later change the bond type through the CLI). Active-active bonds send traffic over both NIC's at the same type, so you gain better throughput (Note: VM's can only use 1 NIC at a time, even if they are in bond. XenServer load balances traffic among all VM's that are using the bond every 10 seconds and adjusts accordingly).
The other type of bond is active-standby. This bond only has 1 active NIC at any given time and uses the other NIC for fail-over scenarios. This type of bond is great for HA as well but does not give the same throughput as active-active bonds.
Management Interfaces and NIC Bonds, Why Care?
What are management interfaces? Management interfaces are XenServer networks with an IP address assigned to them. You set up the primary management interface when you first install XenServer, and you use this interface to connect via XenCenter. You can create additional management interfaces as well, for instances, to connect to your SAN via iSCSI (which we will discuss later). You can also create a management interface on a bond for high availability (HA), which is good, right? Well, not always. Management interfaces act in an active-standby mode on the bond. So, network throughput is not load-balanced across both NIC's, only 1 NIC is used at any given time. This also forces all other traffic that goes over this bond to be active-standby. So if you have a management interface network bond, only 1 NIC is being used at any given time no matter what traffic is passing through. The reason why you would want to care is if you also have your VM's use this bond for their traffic, thereby limiting their collective throughput.
So, a good idea would be to create a bond for your primary management interface and do not allow VM's to use this network bond. This enables HA for your XenServers management connection. You can then, create another bond (or bonds, depending on your requirements) to use for VM traffic, enabling redundant connections for your VM's as wells as better collective throughput.
Connecting to Your SAN via iSCSI
As discussed before, a management interface is a XenServer network with an assigned IP address. When you connect to your SAN via iSCSI (I am assuming this is over a different VLAN as it should) you need to assign an IP address, so you end up creating a management interface. This may not be the optimal configuration as we have also discussed before, management interface network bonds use the active-standby bond mode, so only 1 NIC will be handling iSCSI traffic at any given time. Obviously, you want to gain as much performance for your iSCSI traffic (there are methods in addition to gain better performance but are not in the scope of this document) but if you are only utilizing 1 NIC, that is not optimal. What does Citrix recommend? Whenever possible, Citrix recommends setting up multipathing. With multipathing, you gain the throughput of additional NIC's as well as the redundancy of multiple NIC's. That is great, but there is a gotcha. Some SANs do not support multipathing, so your only other option for a redundant connection is to use a management interface network bond.