Blue Coat ProxySG VIP and Cisco switches need Multicast enabled

About VIP and SGOS

If you decide to use VIP for high availability on Blue Coat ProxySG, then you need to understand how it works. It is very similar to VRRP. That is, both units are configured to share the same IP address, and a priority is configured on each unit to define which ProxySG will be the master. They use multicast to broadcast their status.

You want to have two Blue Coat ProxySG, so you connect them to two switches so that redundancy is improved. Like so:

bctmcast01.png

Both ProxySG will then send multicast packets to advise the other ProxySG that it is up and the priority is contained in the multicast packet, along with the IP address of the VIP and so on. If you are going to have multiple VIPs in a single Layer 2 LAN segment, make sure you use different multicast addresses for each VIP cluster. Note also you can have some interesting design opportunities using three or four VIP per ProxySG for a active/active/backup design (maybe an article for another day, leave a comment if you are interested.).

Because you are using IP Multicast (2) your switch must be able to handle IP Multicast. There are two ways for a switch to handle IP Multicast, one is broadcast it to every port like a hub would, or to use Internet Group Multicast Protocol (IGMP). When broadcasting to every port, every host on that broadcast segment has to listen to the packets, receive into the IP buffer and read them before discarding the information.

bctmcast02.png

This causes significant CPU cycles to be lost to unwanted traffic flows. IGMP is protocol that is enabled on the switch that monitors multicast packet for certain information and decides whether to enable or disable the multicast flows on switch ports. To join a Multicast flow the host or endpoint will issue an IGMP join message for a given IP address, the switch will record this and any responses from other hosts and make sure that those ports will not be closed for that Multicast IP address. After some time, other ports that do not request to be part of the Multicast address will have the Multicast traffic for that address blocked. (3)

By default, Cisco switches have IGMP enabled (4) and they will build an IGMP snooping table for all ports on the switch. Consider the following diagram:

bctmcast03.png
  • Step 1 – both ProxySG issue a IGMP membership to their switches
  • Step 2 – switches record membership of the multicast group

What the Cisco switches do not do is notify their neighbour that they should forward the multicast traffic between SW1 and SW2. Which is exactly NOT what you would expect. The IGMP packets have been terminated by the switch and not forwarded. As a result, multicast traffic is not forwarded between the switches.

When using Blue Coat ProxySG you will know you have this problem because both VIPs will show as master.

bctmcast04.png

Cisco switches need a Multicast router (mrouter)

The best solution to this problem is to setup a IOS device somewhere as Multicast router. When the switch gets an IGMP report it will then relay the notification to the mrouter. The mrouter will then send IGMP messages that notify all switches about their Multicast group membership and everything will be well.

Sample IOS configuration

interface Vlan1
ip address 198.18.1.10 255.255.255.0
ip pim sparse-dense-mode
end

Switch1#show ip igmp snooping mrouter
vlan ports
—–+—————————————-
1 Router

Use MPLS VRF to create a Multicast only interface

Use an MPLS VRF to create a completely isolated IP interface and enable IP on that interface.

!
!enable multicast routing
!
ip multicast-routing
!
!this creates a standalone vrf
!
ip vrf proxy
rd 100:104
!
!now create an interface that is in the VLAN that the ProxySG are using.
!
interface Vlan3502
ip add 198.18.1.1 255.255.255.0
ip vrf forwarding proxy
ip pim sparse-dense-mode
!

Enable IGMP Querier Feature on a Layer 2 Catalyst Switch

From the Cisco web site

“IGMP querier is a relatively new feature on Layer 2 switches. When a network/VLAN does not have a router that can take on the multicast router role and provide the mrouter discovery on the switches, you can turn on the IGMP querier feature. The feature allows the Layer 2 switch to proxy for a multicast router and send out periodic IGMP queries in that network. This action causes the switch to consider itself an mrouter port. The remaining switches in the network simply define their respective mrouter ports as the interface on which they received this IGMP query.”

Switch2(config)#ip igmp snooping querier

Switch2#show ip igmp snooping querier
Vlan IP Address IGMP Version Port
————————————————————-
1 1.1.1.2 v2 Switch

I don’t want to use Multicast.

If you have some passionate gripe about not enabling Multicast on a single interface, you can create static ARP entries for the Multicast MAC address on every switch that is in the data patch. If you have multiple data paths, remember to configure that on every switch in all possible data paths.(5)(6).

I would only recommend this in small networks.

Switch1(config)#mac-address-table static 0100.5e6f.efef vlan 1 interface gigabitethernet 2/46 gigabitethernet 2/48

!— Note: This command should be on one line.

Switch1#show mac-address-table multicast vlan 1

vlan mac address type learn qos ports
—–+—————+——–+—–+—+——————————–
1 0100.5e18.010a static Yes – Gi2/46,Gi2/48

Switch2(config)#mac-address-table static 0100.5e18.010a vlan 1 interface
gigabit 1/47

!— Note: This command should be on one line.

Switch2#show mac-address-table multicast vlan 1
Vlan Mac Address Type Ports
—- ———– —- —–
1 0100.5e18.010a USER Gig1/47

Disable IGMP Snooping on All the Switches

If you disable IGMP snooping, all switches treat multicast traffic as a broadcast traffic. This floods the traffic to all the ports in that VLAN, regardless of whether the ports have interested receivers for that multicast stream.

Switch1(config)#no ip igmp snooping

Switch2(config)#no ip igmp snooping

References

(1)Cisco Web site – Multicast Does Not Work in the Same VLAN in Catalyst Switches. Excellent reference and this article is largely drawn from information here.

(2) Note IP Multicast and Ethernet Multicast are very different things. Note everyone realises this.

(3) Not all IGMP implementations are equal. Some vendors do this badly and some unmanaged switches don’t support IGMP at all. It is much cheaper to make a switch that does not support IGMP.

(4) This is not true of cheap switches – see (3), they disable IGMP to have more CPU cycles for day to day. Most people will never know the difference.

(5) In a big corporate LAN where your ProxySG are in different buildings, this could mean configuring static arp entries on dozens of switches.

(6) You must work out the MAC address from the Multicast address. Here is a link, there are others

About Greg Ferro

Greg Ferro is a Network Engineer/Architect, mostly focussed on Data Centre, Security Infrastructure, and recently Virtualization. He has over 20 years in IT, in wide range of employers working as a freelance consultant including Finance, Service Providers and Online Companies. He is CCIE#6920 and has a few ideas about the world, but not enough to really count.

He is a host on the Packet Pushers Podcast, blogger at EtherealMind.com and on Twitter @etherealmind and Google Plus

You can contact Greg via the site contact page.

  • Brian

    Hmm. Interesting usage of VRFs. If using all Cisco switches, why not just setup one L3 switch or router as a PIM-Sparse-mode RP, and listen for IGMP Membership reports from IGMP-enabled switches?

  • http://etherealmind.com Greg Ferro

    This comes back to the ‘I am scared of multicast’ debate. For many people, enabling multicast as global command is a scary thing.

    In big companies, the security and change control teams will fall into paroxysms of joyous condemnation of how insecure / risky it is. Which, of course, is mostly correct and in high security networks using a VRF is an effective way to solve this problem.

    In normal networks (which should be just about everyone, just use the code in the section “Cisco switches need a Multicast router (mrouter)” to keep it simple.

  • pjg

    Just to clear something up – IGMP and IGMP Snooping are 2 different things. IGMP is a L3 protocol used for joining/leaving multicast groups and querying group membership. IGMP Snooping is a switch feature that listens for IGMP messages so that instead of broadcasting out all L2 ports, the switch can build a table of which L2 ports are interesting in each multicast group. It might sound pedantic but as you say in note (2) IP multicast (L2) and Ethernet multicast (L3) are two different things and both need to be understood for a successful implementation.

    It would make more sense to me to enable PIM on the default gateway for the VIP subnet rather than having a separate VRF interface. I am used to configuring multicast on the whole network though, so it doesn’t scare me :)

    In my experience multicast only cause issues when someone who doesn’t understand the network and/or multicast configures it. SPF failures and L2 multicast are the most common gotchas I see. The worst one I had to troubleshoot had every edge switch configured as an IGMP Snooping Querier, with a query interval of 60 seconds. The end result was that every multicast group went to every switch, whether it really wanted it or not. These were HD streams too. You can imagine the change control to turn that off and configure multicast on the network properly on 165 switches which are providing IPTV to hundreds of rooms.

  • shannon

    I’d like to know the physical configuration of the this setup. Does this work using the Management Interface on the Bluecoat 900 or does this only work in the LAN interface?

  • Simon Wright

    Great article, helped me out – thanks.

    Did find that the static ARP entries are a no go for N7K as it won’t allow a multicast MAC in the statics, ended up using

    On each N7K; (and we are using OTV between all 4 of them with intermediate VDC)

    feature pim
    interface vlan111 (where my proxysg user interfaces sit and OTV extended)
    ip pim sparse-mode

    end

Subscribe For Weekly Updates by Email

Get a Weekly Summary of Latest Articles and Posts to your Email Inbox Every Sunday

Thanks for signing up. Look for the email from MailChimp & make sure you confirm your email address. You may need to check your spam or gmail settings to be sure of receiving the email.

Note: You can unsubscribe at any time using the link at the bottom of every email.