22nd May 2012

You are here: Home / Blog / Cisco IOS Order of Operation – Updated, Again

Cisco IOS Order of Operation – Updated, Again

9th September 2009 By 15 Comments

I am often searching information on the order of operation of the different features on an interface and the packet traverses the IOS software. All I can find on the Cisco.com is this not-suitable-for-every-case table.

Inside-to-Outside Outside-to-Inside
  • If IPSec then check input access list
  • decryption – for CET (Cisco Encryption Technology) or IPSec
  • check input access list
  • check input rate limits
  • input accounting
  • policy routing
  • routing
  • redirect to web cache
  • NAT inside to outside (local to global translation)
  • crypto (check map and mark for encryption)
  • check output access list
  • inspect (Context-based Access Control (CBAC))
  • TCP intercept
  • encryption
  • Queueing
  • If IPSec then check input access list
  • decryption – for CET or IPSec
  • check input access list
  • check input rate limits
  • input accounting
  • NAT outside to inside (global to local translation)
  • policy routing
  • routing
  • redirect to web cache
  • crypto (check map and mark for encryption)
  • check output access list
  • inspect CBAC
  • TCP intercept
  • encryption
  • Queueing

However, this list is the only that I can find and there are several things missing but it’s its the only “official version” I’ve seen. That is, there are others around (see below) but don’t have the Cisco name the bottom.

Networkers documentation

When I found this in my Networkers documentation I think this is pretty complete and so post it here for other to use. If you know of another location on the Cisco web site, please leave a comment so that I can reference it here.

Click the following for a larger diagram.

Update – Thanks to Pierky

“There is another table too, on 6200networks.com by Joe Har ris: http://6200networks.com/2008/09/30/ios-order-of-operation.”

I hope Joe doesn’t mind but I would like to put a copy here for my own reference. I would like to have some sort of confirmation from Cisco that this is more or less correct, but it certainly looks credible.

The author of this list (Craig Weinhold) said:

Thanks for posting the ASA order-of-operation — it’ll come in handy. You refer to the old IOS order-of-operation document — useful but very dated. David Smith and Greg Schudel have an updated version in their CiscoPress book “Router Security Strategies: Securing IP Network Traffic Planes”, but it neglected to touch on NVI’s, crypto clear-text ACLs, virtual reassembly, and other esoteric details. I worked with them last summer to come up with a more authoritative IOS order-of-operation for 12.4/12.4T. It’s necessarily incomplete, but it sheds light on many esoteric feature interactions. It’s attached below:

Big caveat: Some variations in feature ordering may occur in specific router platforms, IOS software releases, and switching paths (i.e.,CEF versus process-switched).

Ingress Features Egress Features
1. Virtual Reassembly * 1. Output IOS IPS Inspection
2. IP Traffic Export (RITE) 2. Output WCCP Redirect
3. QoS Policy Propagation through BGP (QPPB) 3. NM-CIDS
4. Ingress Flexible NetFlow * 4. NAT Inside-to-Outside or NAT Enable *
5. Network Based Application Recognition (NBAR) 5. Network Based Application Recognition (NBAR)
6. Input QoS Classification 6. BGP Policy Accounting
7. Ingress NetFlow * 7. Lawful Intercept
8. Lawful Intercept 8. Check crytpo map ACL and mark for encryption
9. IOS IPS Inspection (inbound) 9. Output QoS Classification
10. Input Stateful Packet Inspection (IOS FW) * 10. Output ACL check (if not marked for encryption)
11. Check reverse crypto map ACL 11. Crypto outbound ACL check (if marked for encryption)
12. Input ACL (unless existing NetFlow record was found) 12. Output Flexible Packet Matching (FPM)
13. Input Flexible Packet Matching (FPM) 13. DoS Tracker
14. IPsec Decryption (if encrypted) 14. Output Stateful Packet Inspection (IOS FW) *
15. Crypto inbound ACL check (if packet had been encrypted) 15. TCP Intercept
16. Unicast RPF check 16. Output QoS Marking
17. Input QoS Marking 17. Output Policing (CAR)
18. Input Policing (CAR) 18. Output MAC/Precedence Accounting
19. Input MAC/Precedence Accounting 19. IPsec Encryption
20. NAT Outside-to-Inside * 20. Output ACL check (if encrypted)
21. Policy Routing 21. Egress NetFlow *
22. Input WCCP Redirect 22. Egress Flexible NetFlow *
23. Egress RITE
24. Output Queuing (CBWFQ, LLQ, WRED)

* A note about virtual-reassembly

Virtual-reassembly causes the router to internally reassemble fragmented packets. It is enabled when an interface is configured with NAT, CBAC, or “ip virtual reassembly”. Operations above marked with a * will process the reassembled version of a packet. All other operations process the individual fragments. After virtual reassembly is complete, the router forwards the original fragments, albeit in proper order. This behavior is very different from PIX/ASA/FWSM and ACE which forward the reassembled packet.

Thus, even if virtual-reassembly is turned on, ACLs used for input access-groups and QoS still need to be aware of how ACLs interact with fragments (http://www.cisco.com/en/US/tech/tk827/tk369/technologies_white_paper09186a00800949b8.shtml).

Routing Features

1. Routing table lookup (if packet isn’t marked with a PBR next-hop) 2. tcp adjust-mss

And Another Update – PERFECTION

Craig Weinhold (author of the above list) has emailed an updated diagram. Which is totally outstanding. I shall consider this be comprehensive. Craig says:

Here is an updated OOO diagram using the same format and style as the
one from the networkers presentation (which was produced by David Smith
and Greg Schudel for their book). I did collaborate with them to fill
in some of the blanks.

Click for a full size image.

This post is copyright of Thropos Ltd ©2008-2011 at Etherealmind.com - contact | email: greg.ferro@packetpushers.net - twitter: @etherealmind | All rights reserved
Filed Under: Blog, Design, Operation Tagged With: , ,
About Greg Ferro

Greg Ferro is a Network Engineer/Architect, mostly focussed on Data Centre, Security Infrastructure, and recently Virtualization. He has over 20 years in IT, in wide range of employers working as a freelance consultant including Finance, Service Providers and Online Companies. He is CCIE#6920 and has a few ideas about the world, but not enough to really count.

He is a host on the Packet Pushers Podcast, blogger at EtherealMind.com and on Twitter @etherealmind and Google Plus

  • http://anomaly.cx Matt Johnson

    A larger version of that image would be great!

    • http://etherealmind.com Greg Ferro

      Matt,

      I fixed the image and you should be able to get a larger image now.

      • Graeme Danielson

        hi,  note that Joe’s old site 6200networks.com is no more. The domain is now parked on a page with nasty infections.  

        I ended up here as I was trying to re-find an excellent Order of Operations for pix/asa he used to have…alas.

  • Matt

    Can you please share what session you got that diagram from?

    • http://etherealmind.com Greg Ferro

      “Router Security Securing IP Control Planes BRKSEC-2105″ – This was from Cisco Networkers 2009 in Barcelona.

  • http://pierky.wordpress.com Pierky

    Hi Greg, this is my first comment on your blog, so I would like to thank you for your good work! :)

    The table you posted from Cisco is at this URL: http://www.cisco.com/en/US/tech/tk648/tk361/technologies_tech_note09186a0080133ddd.shtml

    At the end of the page we can find “Updated: Sep 30, 2008″.

    There is another table too, on 6200networks.com by Joe Harris: http://6200networks.com/2008/09/30/ios-order-of-operation/

    This list has more entries than your Net≠work≠ers table, for example it covers reverse crypto maps.
    This post also is dated September 30th, 2008!

    Who bids more? :)

    • http://etherealmind.com Greg Ferro

      Peirky

      Thanks for your info, I have updated the post to show the new information and made a copy of 6200networks information. While I like this list, it isn’t ‘official’ so I tend to go with the Networkers version unless proven or needed. I could wish that Cisco would publish a recognised version other than the Nat Order of operations. Something definitive would be good.

  • snetherland

    Greg,
    Thanks so much for this. I can’t tell you how often during the design and configure stage of a build I’ve had to scour the internet for an order of operations list. I am curious though where zone-based firewall and nat virutal-interfaces fit into this list. For instance, this specifically references when domain-based nat will be performed, but symmetric natting will first send a packet to the NVI and then route it again to the egress interface. Would this mean the nat decision would occur before the outbound WCCP redirect decision? And then with ZBFW your policies are not implemented until a packet crosses between zones. This, as best I can tell, would have to be after the routing decision unless you are using PBR. Would this mean, excluding packets destined to the Self zone, that the only time stateful packet inspection would occur is on the 9th order of the egress operation?

    Thanks again for this helpful reference.

    • http://etherealmind.com Greg Ferro

      I don’t have any answers to your questions at the moment. While we could wish for Cisco to publish something definitive, this is probably as good as it gets for now.

    • CraigW

      ZBF is tricky and I don’t know the answer. As you say, zone pairs are determined after PBR/routing. It would seem that ZBF ingress/egress policies are probably only applied at egress (or egress to “self”), using a cached copy of the original pre-NAT packet. But that’s just a guess. If you have the time and patience to go through a battery of tests, correlate ZBF behavior with NAT translations, ACL counters, and Netflow records, and you should get a good view of how it fits (please let me know).

      For “nat enable” (NVI), my OOO shows it taking place along with NAT inside-to-outside. I.e., WCCP should occur before “nat enable”

  • http://blog.ioshints.info Ivan Pepelnjak

    Well, as I’ve already (probably) commented to Joe’s post, the outside-to-inside NAT is sometimes before the input ACL. See this post in my wiki:

    http://wiki.nil.com/NAT_caveats_in_IOS_release_12.4T#Packets_rejected_by_inbound_ACL_generate_NAT_translations

    • http://etherealmind.com Greg Ferro

      I suspect that this is why Cisco doesn’t publish an absolute reference because the ACTUAL order varies from platform to platform, possibly different in code releases.

      Still, the above is a good guide until proven otherwise.

  • roberto taccon

    http://www.google.com/profiles/roberto.taccon#buzz

    CISCO IOS OOO (Order Of Operation)

    attached an updated diagram by Gregg Schudel

    the original diagram @ pag.120 Chapter 3: IP Network Traffic Plane Security Concepts on
    Router Security Strategies: Securing IP Network Traffic Planes
    http://www.ciscopress.com/bookstore/product.asp?isbn=1587053365

    http://lh3.googleusercontent.com/_pYguWUnPyho/S8g3bZWvDCI/AAAAAAAAADo/zTqaSSgb1ZA/IOS-OOO.PNG

  • pugalendi dhanapal

    Good Info , Great

  • pugalendi dhanapal

    Good , great info