Thursday, March 18, 2010

Cisco IOS Order of Operation — Updated, Again

September 9, 2009 by Greg Ferro · 11 Comments 

I am often search­ing inform­a­tion on the order of oper­a­tion of the dif­fer­ent fea­tures on an inter­face and the packet tra­verses the IOS soft­ware. All I can find on the Cisco​.com is this not-​​suitable-​​for-​​every-​​case table.

Inside-​​to-​​Outside Outside-​​to-​​Inside
  • If IPSec then check input access list
  • decryp­tion — for CET (Cisco Encryption Technology) or IPSec
  • check input access list
  • check input rate limits
  • input account­ing
  • policy rout­ing
  • rout­ing
  • redir­ect to web cache
  • NAT inside to out­side (local to global translation)
  • crypto (check map and mark for encryption)
  • check out­put access list
  • inspect (Context-​​based Access Control (CBAC))
  • TCP inter­cept
  • encryp­tion
  • Queueing
  • If IPSec then check input access list
  • decryp­tion — for CET or IPSec
  • check input access list
  • check input rate limits
  • input account­ing
  • NAT out­side to inside (global to local translation)
  • policy rout­ing
  • rout­ing
  • redir­ect to web cache
  • crypto (check map and mark for encryption)
  • check out­put access list
  • inspect CBAC
  • TCP inter­cept
  • encryp­tion
  • Queueing

However, this list is the only that I can find and there are sev­eral things miss­ing but it’s its the only “offi­cial ver­sion” I’ve seen. That is, there are oth­ers around (see below) but don’t have the Cisco name the bottom.

Networkers doc­u­ment­a­tion

When I found this in my Networkers doc­u­ment­a­tion I think this is pretty com­plete and so post it here for other to use. If you know of another loc­a­tion on the Cisco web site, please leave a com­ment so that I can ref­er­ence it here.

Click the fol­low­ing for a lar­ger diagram.

Update — Thanks to Pierky

“There is another table too, on 6200net​works​.com by Joe Har ris: http://​6200net​works​.com/​2​0​0​8​/​0​9​/​3​0​/​i​o​s​-​o​r​d​e​r​-​o​f​-operation.”

I hope Joe doesn’t mind but I would like to put a copy here for my own ref­er­ence. I would like to have some sort of con­firm­a­tion from Cisco that this is more or less cor­rect, but it cer­tainly looks credible.

The author of this list (Craig Weinhold) said:

Thanks for post­ing the ASA order-​​of-​​operation — it’ll come in handy. You refer to the old IOS order-​​of-​​operation doc­u­ment — use­ful but very dated. David Smith and Greg Schudel have an updated ver­sion in their CiscoPress book “Router Security Strategies: Securing IP Network Traffic Planes”, but it neg­lected to touch on NVI’s, crypto clear-​​text ACLs, vir­tual reas­sembly, and other eso­teric details. I worked with them last sum­mer to come up with a more author­it­at­ive IOS order-​​of-​​operation for 12.4/12.4T. It’s neces­sar­ily incom­plete, but it sheds light on many eso­teric fea­ture inter­ac­tions. It’s attached below:

Big caveat: Some vari­ations in fea­ture order­ing may occur in spe­cific router plat­forms, IOS soft­ware releases, and switch­ing paths (i.e.,CEF versus process-​​switched).

Ingress Features Egress Features
1. Virtual Reassembly * 1. Output IOS IPS Inspection
2. IP Traffic Export (RITE) 2. Output WCCP Redirect
3. QoS Policy Propagation through BGP (QPPB) 3. NM-​​CIDS
4. Ingress Flexible NetFlow * 4. NAT Inside-​​to-​​Outside or NAT Enable *
5. Network Based Application Recognition (NBAR) 5. Network Based Application Recognition (NBAR)
6. Input QoS Classification 6. BGP Policy Accounting
7. Ingress NetFlow * 7. Lawful Intercept
8. Lawful Intercept 8. Check crytpo map ACL and mark for encryption
9. IOS IPS Inspection (inbound) 9. Output QoS Classification
10. Input Stateful Packet Inspection (IOS FW) * 10. Output ACL check (if not marked for encryption)
11. Check reverse crypto map ACL 11. Crypto out­bound ACL check (if marked for encryption)
12. Input ACL (unless exist­ing NetFlow record was found) 12. Output Flexible Packet Matching (FPM)
13. Input Flexible Packet Matching (FPM) 13. DoS Tracker
14. IPsec Decryption (if encrypted) 14. Output Stateful Packet Inspection (IOS FW) *
15. Crypto inbound ACL check (if packet had been encrypted) 15. TCP Intercept
16. Unicast RPF check 16. Output QoS Marking
17. Input QoS Marking 17. Output Policing (CAR)
18. Input Policing (CAR) 18. Output MAC/​Precedence Accounting
19. Input MAC/​Precedence Accounting 19. IPsec Encryption
20. NAT Outside-​​to-​​Inside * 20. Output ACL check (if encrypted)
21. Policy Routing 21. Egress NetFlow *
22. Input WCCP Redirect 22. Egress Flexible NetFlow *
23. Egress RITE
24. Output Queuing (CBWFQ, LLQ, WRED)

* A note about virtual-​​reassembly

Virtual-​​reassembly causes the router to intern­ally reas­semble frag­men­ted pack­ets. It is enabled when an inter­face is con­figured with NAT, CBAC, or “ip vir­tual reas­sembly”. Operations above marked with a * will pro­cess the reas­sembled ver­sion of a packet. All other oper­a­tions pro­cess the indi­vidual frag­ments. After vir­tual reas­sembly is com­plete, the router for­wards the ori­ginal frag­ments, albeit in proper order. This beha­vior is very dif­fer­ent from PIX/​ASA/​FWSM and ACE which for­ward the reas­sembled packet.

Thus, even if virtual-​​reassembly is turned on, ACLs used for input access-​​groups and QoS still need to be aware of how ACLs inter­act with frag­ments (http://​www​.cisco​.com/​e​n​/​U​S​/​t​e​c​h​/​t​k​8​2​7​/​t​k​3​6​9​/​t​e​c​h​n​o​l​o​g​i​e​s​_​w​h​i​t​e​_​p​a​p​e​r​0​9​1​8​6​a​0​0​8​0​0​9​49b8.shtml).

Routing Features

1. Routing table lookup (if packet isn’t marked with a PBR next-​​hop) 2. tcp adjust-​​mss

And Another Update — PERFECTION

Craig Weinhold (author of the above list) has emailed an updated dia­gram. Which is totally out­stand­ing. I shall con­sider this be com­pre­hens­ive. Craig says:

Here is an updated OOO dia­gram using the same format and style as the
one from the net­work­ers present­a­tion (which was pro­duced by David Smith
and Greg Schudel for their book). I did col­lab­or­ate with them to fill
in some of the blanks.

Click for a full size image.

Please rate this post:

  Why Rate Posts?
1 Star - It\\\'s Crud2 Stars - It\\\'s Tosh3 Stars - Something\\\'s missing4 Stars - Needs works5 Stars - Good Enough6 Stars - Good7 Stars - Excellent8 Stars - Brilliant9 Stars - Astonishing10 Stars - Awesomely Godlike? (5 votes, average: 10.00 out of 10)
Loading ... Loading ...

Comments

11 Responses to “Cisco IOS Order of Operation — Updated, Again”
  1. Matt Johnson says:

    A lar­ger ver­sion of that image would be great!

  2. Matt says:

    Can you please share what ses­sion you got that dia­gram from?

  3. Pierky says:

    Hi Greg, this is my first com­ment on your blog, so I would like to thank you for your good work! :)

    The table you pos­ted from Cisco is at this URL: http://​www​.cisco​.com/​e​n​/​U​S​/​t​e​c​h​/​t​k​6​4​8​/​t​k​3​6​1​/​t​e​c​h​n​o​l​o​g​i​e​s​_​t​e​c​h​_​n​o​t​e​0​9​1​8​6​a​0​0​8​0​1​3​3ddd.shtml

    At the end of the page we can find “Updated: Sep 30, 2008″.

    There is another table too, on 6200net​works​.com by Joe Harris: http://​6200net​works​.com/​2​0​0​8​/​0​9​/​3​0​/​i​o​s​-​o​r​d​e​r​-​o​f​-​operation/

    This list has more entries than your Net­work­ers table, for example it cov­ers reverse crypto maps.
    This post also is dated September 30th, 2008!

    Who bids more? :)

    • Greg Ferro says:

      Peirky

      Thanks for your info, I have updated the post to show the new inform­a­tion and made a copy of 6200networks inform­a­tion. While I like this list, it isn’t ‘offi­cial’ so I tend to go with the Networkers ver­sion unless proven or needed. I could wish that Cisco would pub­lish a recog­nised ver­sion other than the Nat Order of oper­a­tions. Something defin­it­ive would be good.

  4. snetherland says:

    Greg,
    Thanks so much for this. I can’t tell you how often dur­ing the design and con­fig­ure stage of a build I’ve had to scour the inter­net for an order of oper­a­tions list. I am curi­ous though where zone-​​based fire­wall and nat virutal-​​interfaces fit into this list. For instance, this spe­cific­ally ref­er­ences when domain-​​based nat will be per­formed, but sym­met­ric nat­ting will first send a packet to the NVI and then route it again to the egress inter­face. Would this mean the nat decision would occur before the out­bound WCCP redir­ect decision? And then with ZBFW your policies are not imple­men­ted until a packet crosses between zones. This, as best I can tell, would have to be after the rout­ing decision unless you are using PBR. Would this mean, exclud­ing pack­ets destined to the Self zone, that the only time state­ful packet inspec­tion would occur is on the 9th order of the egress operation?

    Thanks again for this help­ful reference.

    • Greg Ferro says:

      I don’t have any answers to your ques­tions at the moment. While we could wish for Cisco to pub­lish some­thing defin­it­ive, this is prob­ably as good as it gets for now.

    • CraigW says:

      ZBF is tricky and I don’t know the answer. As you say, zone pairs are determ­ined after PBR/​routing. It would seem that ZBF ingress/​egress policies are prob­ably only applied at egress (or egress to “self”), using a cached copy of the ori­ginal pre-​​NAT packet. But that’s just a guess. If you have the time and patience to go through a bat­tery of tests, cor­rel­ate ZBF beha­vior with NAT trans­la­tions, ACL coun­ters, and Netflow records, and you should get a good view of how it fits (please let me know).

      For “nat enable” (NVI), my OOO shows it tak­ing place along with NAT inside-​​to-​​outside. I.e., WCCP should occur before “nat enable”

    • Greg Ferro says:

      I sus­pect that this is why Cisco doesn’t pub­lish an abso­lute ref­er­ence because the ACTUAL order var­ies from plat­form to plat­form, pos­sibly dif­fer­ent in code releases.

      Still, the above is a good guide until proven otherwise.

Speak Your Mind

Tell us what you're thinking...
and oh, if you want a pic to show with your comment, go get a gravatar!