Routing

Hi,

I've got a redundant path between two EX4200-48T switches, each in independent a virtual chassis. They are connected directly with each other through a Layer 1 fiber connection on 10G uplink modules. This interface is in trunk mode and a pure L2 interface on both switches.

In addition to that, there is another interface on each switch which acts as consumer edge towards a VPLS L2 tunnel. On the switches these interfaces are L2 trunks with slightly different VLANs configured on it. The PE is a MX80 encapsulating packets and tunneling between the same switches again. 

That is, effectively I do have a loop on purpose, as I want to achieve failure resistance. I want to use the L1 link, and automated fail-over to the VPLS tunnel. Thus, I want to enable both links at the same time, and disable the VPLS port through MSTP. RSTP is enabled on all switches, the PE routers do not do anything fancy, other than capsulating the packets in MPLS. 

As soon as I bring up both interfaces, i.e. the VPLS port as well as the L1 fiber, I get broadcast storm and both switches tell me that both interfaces are active and forwarding. 

This is the CE interface configuration on the switch:

> show configuration interfaces xe-0/1/0       
mtu 9100;
unit 0 {
    family ethernet-switching {
        port-mode trunk;
        vlan {
            members [ ROUTING_3 PUBLIC INTERNAL ];
        }
    }
}

 This is the PE:

> show configuration interfaces xe-0/0/0 
flexible-vlan-tagging;
mtu 9100;
encapsulation flexible-ethernet-services;
unit 3 {
    /* L3 interface, VLAN is ROUTING_3 */
    vlan-id 3;
    family inet {
    /* ... */
    }
    family inet6 {
   /* ... */
    }
}
unit 512 {
    encapsulation vlan-vpls;
    vlan-id 512; /* PUBLIC */
    family vpls;
}
unit 1000 {
    encapsulation vlan-vpls;
    vlan-id-range 1000-4094; /* INTERNAL et al */
    family vpls;
}

 And this is the routing instance:

routing-instances {
    cloud-backend {
        instance-type vpls;
        interface xe-0/0/0.512;
        route-distinguisher XXX:4;
        vrf-target target:ASN:1;
        protocols {
            vpls {
                site-range 4;
                no-tunnel-services;
                site ce-switch {
                    site-identifier 1;
                    multi-homing;
                }
                connectivity-type ce;
            }
        }
    }
    cloud-internal {
        instance-type vpls;
        vlan-id all;
        interface xe-0/0/0.1000;
        route-distinguisher XXX:4;
        vrf-target target:ASN:2;
        protocols {
            vpls {
                site-range 4;
                no-tunnel-services;
                site ce-switch {
                    site-identifier 1;
                    multi-homing;
                }
                connectivity-type ce;
            }
        }
    }
}

Any idea why my VPLS facing interface on the switch does not seem to detect loops on the STP? Broadcast traffic generally works, as does unicast. I did also try to reproduce the setup on a L1/L1 combination, there the switches have no problem whatsoever to detect loops by means of STP.  

There are a few ways to resolve that issue, but your configuration that you show does not indicate that you have configured any such detection method. 

You can configure either a spanning tree protocol between PE and EX, active and backup links on the PE, Ethernet Ring Protection (ERP), or a link aggregation group (LAG).

In this case, I believe if you configure site-preference under both vlpls instances site ce-switch site identifier and configure high and low values so BGP can remove the loops. ,  

Thanks for your suggestion. Let me recap to understand:

• I could enable spanning tree on the CE facing interface on the MX. Is that correct? Why is that needed, shouldn't the MX just encapsulate packets, including STP BDPU and carry them over to the other switch which has STP enabled?
• A LAG: How would that work, one interface is a physical L1 fiber, one is a backup VPLS tunnel, both connected to different devices. Do you mean MC-LAG per chance?
• Having VPLS site identifier only helps for active/standby VPLS interfaces providing VPLS redundancy. Right? That's not going to help to detect a loop in BGP, since BGP can't and won't know about the physical (primary) link or do I miss something in your explanation?

In a Service Provider perspective, there maybe layer-2 aggregation switches interconnected to MPLS edge switch/routers, as the BPDU packets is one layer-2 PDU which travels in service provider network independly by multicast MAC address without label switching. In order to protect SP switches without any BPDU impaction sent from customers from CE side, SP edge router/switch should run bpdu-block function and assume the NTU is providered by the SP, so L2PT will be configured from each NTU (layer-2 CE) NNI interface for per VPLS instance.

BGP loop detection and VPLS loop protection are two things.

first is for loop protection from routing loop; second is sourced from layer-2 MAC address like horizon-split, but both of them can control the traffic avoiding loop traffic.

For future reference, the problem was solved. VPLS transparently encapsulates BDPUs, but only if they are matched to a VLAN which is actually encapsulated into VPLS.

However, both, (R)STP and MSTI is VLAN aware but use itself untagged frames. Thus, the MX router does not know what to do with that untagged packet and throws it away. To solve this problem two solutions do exist:

• use MSTI and use a native-vlan matching a VPLS tunnel to force-tag untagged frames
• use VSTP, not (R)STP/MSTI

Hello,

---

@at wrote:

For future reference, the problem was solved. VPLS transparently encapsulates BDPUs, but only if they are matched to a VLAN which is actually encapsulated into VPLS.

 

However, both, (R)STP and MSTI is VLAN aware but use itself untagged frames. Thus, the MX router does not know what to do with that untagged packet and throws it away. To solve this problem two solutions do exist:

 

• use MSTI and use a native-vlan matching a VPLS tunnel to force-tag untagged frames
• use VSTP, not (R)STP/MSTI

---

VSTP won't work if VLAN translation is involved as it includes VLAN-id in payload. If 802.1Q header does not match VLAN-id in VSTP payload, such BPDU is considered invalid.
I'd suggest to run STP betwen CE and PE and not rely on BPDU propagation across VPLS.

HTH

Thanks

Alex