Cisco ACE – Enterprise Load Balancing on a Stick using Source NAT – Part 1

Linear vs Stick Load Balancing.

For many companies, the use of a Load Balancer isn’t well understood. This article looks at the use of Source NAT to minimise the impact of deploying a load balancer in an existing corporate network with minimal interruption. I call this deploying your Load Balancer in “Stick Mode” (following the NAT on a Stick model and minimise the changes in your network.

Linear Deployment

This diagram outlines what I call the linear load balancing design:

Lb snat 1

People who are new to load balancing tend to use this model. It works ok and it meets the primary requirements for the routing and symmetric packet flow. One of the key factors for load balancer design is that the flow that is load balanced must return through the load balancer.A load balancer is usually a good router but a very expensive use of resources.

IP Address Modification

What happens when an IP Packet passes through a Load Balancer ? A simple Load Balancing VIP will simply modify the destination address. That is, the Virtual IP of the load balancer is the destination address from the client. When the IP flow hits the load balancer, it will examine the IP header, run through the configuration to select a Real Server, rewrite the header and then forward the packet. Aside from selecting from many possible server/destination, this is Destination NAT. Therefore the server sees the IP Source as the client, and must be able to route the packet back to client, via the Load Balancer, who will undo the NAT, and return the packet to the client.

Note that a full load balancing design does a lot more than simple NAT, but it’s a useful concept for basic visualisation.

Routing Required

Therefore you need to add some routing to your network

Lb snat 2

Problem with Linear Deployment

There are are number of problems with this type of design.

Lb snat 3

The biggest problem is that the Load Balancer becomes part of the network infrastructure, and must be able to handle all the traffic that flows to and from the servers. Given that Load Balancers (especially F5 Load Balancers) are expensive, this is not a good design. Consider what happens when you want to backup your servers ? That’s a lot of traffic that might impact the performance of your load balancer.

The second problem is that your servers must be physically in the right location for the Ethernet connection to the Load Balancer. In most data centres, this may require specific planning for space and power which isn’t always possible or easily done. For example, a server may already be live and not able to move, but the Load Balancing VLANs are on another switch.

The third problem is that the Load Balancer will get blamed for every problem that the server might be experiencing. Of course this is silly, but the server admins will be convinced that the LB is somehow magically affecting AD replication etc. Which then leads to this bad idea :

Lb snat 4

Some people might attempt setup something like the above system by adding a L3 function in front of the server and use policy routing or even policy based NAT to send non-balanced traffic by a different path. Obviously, trying to use routing will end up with a simple routing loop.

Use a second interface on the server

This is the most common solution to “Load Balancer as Router” problem.

Lb snat 5

This scenario causes major problems with the server since it now a router. That is, a routing table on the server decides which interface is used for traffic. Thus, a static route is added to the server for all “backup destinations” to use the secondary interface. This obviously is a support nightmare, since different packets go different ways according to which source or destination address is being used. Of course, any problems will still be blamed on the network because Server Admins often don’t understand routing very well (Which is fair enough, I say).

Other Posts in A Series On The Same Topic

  1. Cisco ACE - Enterprise Load Balancing on a Stick using Source NAT - Part 3 (14th February 2011)
  2. Cisco ACE - Enterprise Load Balancing on a Stick using Source NAT - Part 2 (9th February 2011)
  3. Cisco ACE - Enterprise Load Balancing on a Stick using Source NAT - Part 1 (8th February 2011)
  • Santino Rizzo

    Is the Source NAT coming in Part 2? That’s we use for most of our ACE deployments. Source NAT allows for ACE to be deployed regardless of server farm location (within reason), and only LB traffic flows through it.

    • http://etherealmind.com Greg Ferro

      Yeah, the post got kinda long and I want to a break from writing for a couple of days so i broke it into pieces.

  • Rick Arps

    Another workaround is to nat the backup/management traffic through another router. Routes to the servers are advertised through the nat router and not through the load balancer, fixing the routing issue, though leaving us with plenty of other headaiches to worry about.
    I’d much prefer Source NAT, but legacy applications based on user IP address, and management wanting source ip traffic analysis from iis logs make for an ugly solution.

    • Steve Busby

      Rick,

      It’s pretty trivial to inject src_ip into the packet headers so logs, etc can capture the original IP.

  • http://TACACS.org/ Jihn

    Greg – I feel I can safely assume you haven’t been following NANOG, because otherwise you would know that NAT solves nothing.

    :)

    I look forward to reading the other posts in this series.

  • Rob

    The ACE is the problem for all application data 98% of the time. It “normalizes” traffic by default. It does horrible things to the applications unless you disable it (no normalization under the interface). Kills ftp sessions, does not allow dynamic windowing,