Cisco FWSM and ACE resource allocation strategies


Introduction

If you are deploying a Cisco Application Control Engine (ACE) or Firewall Service Module (FWSM) using virtual contexts, you should be considering how to allocate the system resources between each virtual context that you create. When using virtualisation, you are sharing a finite amount of physical resources between the configured virtual systems and you should have a strategy on how to allocate those resources. Just hoping that they are enough isn’t necessarily the best option.

Possible Resource Allocation Models for Cisco FWSM and ACE.

I think that that there four general methods of allocating resources that can be considered.

Models Description
Custom Allocation A context is configured with specific limits on a per application/deployment basis. Possibly even with some sort of scientific intent.
Allocate Minimum In this model, the user configures a minimum number of resources to be available. The typical thinking here is that a system must have at least some resources to be able to work. Any context can burst up to the maximum physical resources available
No Allocation In this model every context is able to consume all resources available in the chassis.
Allocate Maximum The administrator configures a maximum amount of resources, which ensures that the context cannot consume all resources and thus cause loss of service in the other contexts.

Custom Allocation

Custom Allocation would work where all users o the firewall have a clear understanding of the protocol and application, and can take the time to ensure that the configuration matches the requirement. In most networks, this is not the case and thus is likely only under certain circumstances.

Allocate Minimum

Allocate Minimum means that every context is allocated at least some resources. However, when this is configured the total resource count is reduced. That is, if you allocate 5% to five contexts, then 25% of total resources are reserved, 75% of resources are available as a burst capacity. However when you allocate twenty contexts, there is no burst capacity left as all resources have been allocated (even if they are not used).

Unrestricted or No Allocation

No Allocation means that every context can consume all the resources available in the unit. This will often work quite well since most firewalls / load balancer’s do not use as many resources as people think. However, since there are no minimum reservation or maximum limits means that an overrun event can exhaust the available resources and every context will be affected on an intermittent basis.

Allocate Maximum

In this model, you allocate a maximum or peak level of resources for a virtual context. This way, you do not reserve or partition resources to a minimum and you ensure that a virtual context that attempts to consume too many resources does not impact all other contexts. However, this may lead to resources being unused without manual intervention and this makes it a contentious design choice.

My Preferred Model

The preferred model is to use Allocate Maximum. The primary considerations are:

  1. operational integrity
  2. monitoring capabilties
  3. easier to troubleshoot
  4. fails in a manner that you expect

“Better” Failure

The most significant advantage of the ‘Allocate Maximum’ is that when resource exhaustion occurs, the damage will be limited to the context that has resource overrun. For the case of minimum allocation, it would be possible for many contexts to experience intermittent failures before the underlying reason is detected (assuming that the gross resources of the unit have been exceeded). This creates conditions for a failure that has a very wide impact on the network.

Note also the Cisco ACE resources allocation of 5% minimum means that after 20 contexts all 100% of resources have been reserved and no more resources are available for the 21st context.

The alternative models do….

Both Allocate Minimum and No Allocation are reasonably difficult faults to detect (in the resource exhaustion) and thus negatively affect operational integrity of the network. Consider that resources for an FWSM are allocated in pools of connections, fixups, host and translation. Lets assume that the total number of translation slots (266144 maximum) have been consumed across several contexts. This will result in random packet loss across many firewalls, but only if translation is involved. Flows that are not NAT’d will not be affected.

It would be difficult to a correctly locate the fault in virtual firewall in the first instance (unless the monitoring / syslog is being performed correctly – which doesn’t always happen).

Custom Allocation

Once you start using your ACE or FWSM heavily, and you reach an average utilisation of about 40%, you are likely to begin customisation of your resources. In a typical Enterprise network, not all context will consume an equal amount of resources. For example, an Internet facing firewall will require enough xlate and conn resources. But there are less translation slots than there are connection slots. How do you develop a plan that correctly allocates resources according to the application.

For certain companies who have just one or two applications to support, they can understand the exact requirements. But for most organisations, it will not be possible to understand the application profile for each context and it’s unreasonable to expect to allocate resources on a custom basis.

The EtherealMind View

In one sense, the resource allocation models on the ACE and FWSM have a number of restrictions that are inflexible. The temptation to create a detailed resource plan should be avoided because it is hard to support and even harder to give to a junior engineer to look after. Working out numbers of NAT translates and connections for every context is a sure way to send yourself mad or spend endless hours tweaking resource configurations.

I propose a Model where you allocate a maximum amount of resources for each context. I have also created models where Bronze, Silver Gold equate to increasing levels of the maximum bound (and charge back accordingly). Note that setting maximum levels ensures that there are no reservations of resources with the ACE / FWSM operating system. This provides a better strategy for overall allocation for large number of virtual contexts in multi-tenant networks.

  • Niall McGee

    Greg,

    Really interesting article – good to see others grappling with the same issues as we are!

    I was wondering if you had any examples of how to configure the ACE for your “Allocate Maximum” model? I had understood that the ACE “limit-resource” command would only accept a maximum value of either “equal-to-min” or “unlimited”. Do you have a way of specifying an absolute or percentage maximum for a resource?

    Cheers,
    Niall.

    • Lt_dysan

      I agree as well. I’m not sure how do setup the “allocate max” model either. I don’t think the ACE can support that, even though it would make things easier to work with.