I am often searching information on the order of operation of the different features on an interface and the packet traverses the IOS software. All I can find on the Cisco.com is this not-suitable-for-every-case table.
- If IPSec then check input access list
- decryption – for CET (Cisco Encryption Technology) or IPSec
- check input access list
- check input rate limits
- input accounting
- policy routing
- routing
- redirect to web cache
- NAT inside to outside (local to global translation)
- crypto (check map and mark for encryption)
- check output access list
- inspect (Context-based Access Control (CBAC))
- TCP intercept
- encryption
- Queueing
- If IPSec then check input access list
- decryption – for CET or IPSec
- check input access list
- check input rate limits
- input accounting
- NAT outside to inside (global to local translation)
- policy routing
- routing
- redirect to web cache
- crypto (check map and mark for encryption)
- check output access list
- inspect CBAC
- TCP intercept
- encryption
- Queueing
However, this list is the only that I can find and there are several things missing but it’s its the only “official version” I’ve seen. That is, there are others around (see below) but don’t have the Cisco name the bottom.
Networkers documentation
When I found this in my Networkers documentation I think this is pretty complete and so post it here for other to use. If you know of another location on the Cisco web site, please leave a comment so that I can reference it here.
Click the following for a larger diagram.
Update – Thanks to Pierky
“There is another table too, on 6200networks.com by Joe Har ris: http://6200networks.com/2008/09/30/ios-order-of-operation.”
I hope Joe doesn’t mind but I would like to put a copy here for my own reference. I would like to have some sort of confirmation from Cisco that this is more or less correct, but it certainly looks credible.
The author of this list (Craig Weinhold) said:
Thanks for posting the ASA order-of-operation — it’ll come in handy. You refer to the old IOS order-of-operation document — useful but very dated. David Smith and Greg Schudel have an updated version in their CiscoPress book “Router Security Strategies: Securing IP Network Traffic Planes”, but it neglected to touch on NVI’s, crypto clear-text ACLs, virtual reassembly, and other esoteric details. I worked with them last summer to come up with a more authoritative IOS order-of-operation for 12.4/12.4T. It’s necessarily incomplete, but it sheds light on many esoteric feature interactions. It’s attached below:
Big caveat: Some variations in feature ordering may occur in specific router platforms, IOS software releases, and switching paths (i.e.,CEF versus process-switched).
| Ingress Features | Egress Features |
|---|---|
| 1. Virtual Reassembly * | 1. Output IOS IPS Inspection |
| 2. IP Traffic Export (RITE) | 2. Output WCCP Redirect |
| 3. QoS Policy Propagation through BGP (QPPB) | 3. NM-CIDS |
| 4. Ingress Flexible NetFlow * | 4. NAT Inside-to-Outside or NAT Enable * |
| 5. Network Based Application Recognition (NBAR) | 5. Network Based Application Recognition (NBAR) |
| 6. Input QoS Classification | 6. BGP Policy Accounting |
| 7. Ingress NetFlow * | 7. Lawful Intercept |
| 8. Lawful Intercept | 8. Check crytpo map ACL and mark for encryption |
| 9. IOS IPS Inspection (inbound) | 9. Output QoS Classification |
| 10. Input Stateful Packet Inspection (IOS FW) * | 10. Output ACL check (if not marked for encryption) |
| 11. Check reverse crypto map ACL | 11. Crypto outbound ACL check (if marked for encryption) |
| 12. Input ACL (unless existing NetFlow record was found) | 12. Output Flexible Packet Matching (FPM) |
| 13. Input Flexible Packet Matching (FPM) | 13. DoS Tracker |
| 14. IPsec Decryption (if encrypted) | 14. Output Stateful Packet Inspection (IOS FW) * |
| 15. Crypto inbound ACL check (if packet had been encrypted) | 15. TCP Intercept |
| 16. Unicast RPF check | 16. Output QoS Marking |
| 17. Input QoS Marking | 17. Output Policing (CAR) |
| 18. Input Policing (CAR) | 18. Output MAC/Precedence Accounting |
| 19. Input MAC/Precedence Accounting | 19. IPsec Encryption |
| 20. NAT Outside-to-Inside * | 20. Output ACL check (if encrypted) |
| 21. Policy Routing | 21. Egress NetFlow * |
| 22. Input WCCP Redirect | 22. Egress Flexible NetFlow * |
| 23. Egress RITE | |
| 24. Output Queuing (CBWFQ, LLQ, WRED) |
* A note about virtual-reassembly
Virtual-reassembly causes the router to internally reassemble fragmented packets. It is enabled when an interface is configured with NAT, CBAC, or “ip virtual reassembly”. Operations above marked with a * will process the reassembled version of a packet. All other operations process the individual fragments. After virtual reassembly is complete, the router forwards the original fragments, albeit in proper order. This behavior is very different from PIX/ASA/FWSM and ACE which forward the reassembled packet.
Thus, even if virtual-reassembly is turned on, ACLs used for input access-groups and QoS still need to be aware of how ACLs interact with fragments (http://www.cisco.com/en/US/tech/tk827/tk369/technologies_white_paper09186a00800949b8.shtml).
Routing Features
1. Routing table lookup (if packet isn’t marked with a PBR next-hop) 2. tcp adjust-mss
And Another Update – PERFECTION
Craig Weinhold (author of the above list) has emailed an updated diagram. Which is totally outstanding. I shall consider this be comprehensive. Craig says:
Here is an updated OOO diagram using the same format and style as the
one from the networkers presentation (which was produced by David Smith
and Greg Schudel for their book). I did collaborate with them to fill
in some of the blanks.
Click for a full size image.
Loading ...
About Greg Ferro
A larger version of that image would be great!
Matt,
I fixed the image and you should be able to get a larger image now.
Can you please share what session you got that diagram from?
“Router Security Securing IP Control Planes BRKSEC-2105″ – This was from Cisco Networkers 2009 in Barcelona.
Hi Greg, this is my first comment on your blog, so I would like to thank you for your good work!
The table you posted from Cisco is at this URL: http://www.cisco.com/en/US/tech/tk648/tk361/technologies_tech_note09186a0080133ddd.shtml
At the end of the page we can find “Updated: Sep 30, 2008″.
There is another table too, on 6200networks.com by Joe Harris: http://6200networks.com/2008/09/30/ios-order-of-operation/
This list has more entries than your Networkers table, for example it covers reverse crypto maps.
This post also is dated September 30th, 2008!
Who bids more?
Peirky
Thanks for your info, I have updated the post to show the new information and made a copy of 6200networks information. While I like this list, it isn’t ‘official’ so I tend to go with the Networkers version unless proven or needed. I could wish that Cisco would publish a recognised version other than the Nat Order of operations. Something definitive would be good.
Greg,
Thanks so much for this. I can’t tell you how often during the design and configure stage of a build I’ve had to scour the internet for an order of operations list. I am curious though where zone-based firewall and nat virutal-interfaces fit into this list. For instance, this specifically references when domain-based nat will be performed, but symmetric natting will first send a packet to the NVI and then route it again to the egress interface. Would this mean the nat decision would occur before the outbound WCCP redirect decision? And then with ZBFW your policies are not implemented until a packet crosses between zones. This, as best I can tell, would have to be after the routing decision unless you are using PBR. Would this mean, excluding packets destined to the Self zone, that the only time stateful packet inspection would occur is on the 9th order of the egress operation?
Thanks again for this helpful reference.
I don’t have any answers to your questions at the moment. While we could wish for Cisco to publish something definitive, this is probably as good as it gets for now.
ZBF is tricky and I don’t know the answer. As you say, zone pairs are determined after PBR/routing. It would seem that ZBF ingress/egress policies are probably only applied at egress (or egress to “self”), using a cached copy of the original pre-NAT packet. But that’s just a guess. If you have the time and patience to go through a battery of tests, correlate ZBF behavior with NAT translations, ACL counters, and Netflow records, and you should get a good view of how it fits (please let me know).
For “nat enable” (NVI), my OOO shows it taking place along with NAT inside-to-outside. I.e., WCCP should occur before “nat enable”
Well, as I’ve already (probably) commented to Joe’s post, the outside-to-inside NAT is sometimes before the input ACL. See this post in my wiki:
http://wiki.nil.com/NAT_caveats_in_IOS_release_12.4T#Packets_rejected_by_inbound_ACL_generate_NAT_translations
I suspect that this is why Cisco doesn’t publish an absolute reference because the ACTUAL order varies from platform to platform, possibly different in code releases.
Still, the above is a good guide until proven otherwise.
http://www.google.com/profiles/roberto.taccon#buzz
CISCO IOS OOO (Order Of Operation)
attached an updated diagram by Gregg Schudel
the original diagram @ pag.120 Chapter 3: IP Network Traffic Plane Security Concepts on
Router Security Strategies: Securing IP Network Traffic Planes
http://www.ciscopress.com/bookstore/product.asp?isbn=1587053365
http://lh3.googleusercontent.com/_pYguWUnPyho/S8g3bZWvDCI/AAAAAAAAADo/zTqaSSgb1ZA/IOS-OOO.PNG