Routing
Reply
Visitor
Ariel
Posts: 7
Registered: ‎08-21-2009
0

Dual Core Routers - VRRP - Dual Upstream - PIC Issue - M5

Hello all,

 

We have two M5's with this hardware:

 

ge-0/0/0 - used for VLAN supinterfaces to customers and infrastructure

ge-0/1/0 - used for VLAN subinterface to upstream 1 & 2 (one active per router)

so-0/3/0 - OC-12 cross connect between the M5's

 

Each router's ge's are connected to a 6509 per router. Each 6509 has differing connections. We run MPLS, IBGP, & OSPF for core operations. We have the two M5's as bgp peers to eachother and act as a route reflector core to 4 edge routers for circuits. Each edge router has a route reflector BGP session to each M5. Each Upstream fiber is in one 6509. Upstream 1, subint M5-1, 6509-01, etc. The goal was to lose any router PIC or 'any' switch and still maintain link with the remaining customers and gear.

 

We run VRRP on most (in progress) customer and infrastructure subinterfaces on ge-0/0/0 between the two M5's.

 

Today we experienced what appears to have been a hardware lockup that took the network mostly down that was solved by the reboot of the first M5. This was noted when we rebooted the 6509 to which it was connected and only the ge-0/1/0 interface went red. The M5 still believed that ge-0/0/0 was up. M5-1 was unreachable on interfaces on its ge-0/0/0 interface, including its routed core IP.

 

M5 2 is the VRRP master for all configured VRRP groups via 'priority 254'. We use a three address VRRP config (sample below).

 

The issue made itself known as follows:

 

Traffic destined out upstream 1 on M5 1 could not transit the router as ge-0/0/0 was not passing traffic, but it was seen as up to the router and the switch. Traffic would normally traverse both routers on a given subinterface with VRRP configured, which seems to show that VRRP is acting as a load balancing protocol with traffic that would prefer Upstream 1 or equipment linked to 6509-01 traversing the ge-0/0/0 subinterface on M5-1.

 

Is there a way to disable the quasi load balancing behavior of VRRP?I've attempted no use of 'accept-data' as well as explict configuration of 'no-accept-data' on the VRRP group.

 

My belief is that since ge-0/0/0 thought it was up, OSPF was still prioritizing that path, rather than the OC12 cross-connect. Due to circumstances not yet able to be rectified, the edge routers are all linked to 6509-01, the logical subinterface that is their default gateway and the subnet which their ethernet interface lives on is VRRP configured on both routers.

 

We've had a similar issue before on the other M5, perhaps linking to a FEB intermittent failure. The Chassisd log did not indicate any failures today, simply the logged entries when the router came back up.

 

 

Diagram:

 

M5-1 ---oc12-------   M5-2

ge0,ge1                       ge0,ge1

6509-1 ----gige----- 6509-2

upstream1               upstream2

customers               customers

edge routers           infrastructure

infrastructure

 

 

Thanks for any advice you all may have.

 

 

Sample interface config

 

M5-1

vlan-id xxx;
family inet {
    mtu 1500;
    policer {
        input x;
        output x;
    }
    address x.x.x.2/29 {
        vrrp-group 11 {
            virtual-address x.x.x.1;

            priority 100

            advertise-interval 1;
        }
    }
}
family mpls {
    mtu 1530;
}

 

vlan-id xxx;
family inet {
    mtu 1500;
    policer {
        input x;
        output x;
    }
    address x.x.x.3/29 {
        vrrp-group 11 {
            virtual-address x.x.x.1;

            priority 254
            advertise-interval 1;
            accept-data;
        }
    }
}
family mpls {
    mtu 1530;
}

Trusted Contributor
acooley
Posts: 117
Registered: ‎08-07-2010
0

Re: Dual Core Routers - VRRP - Dual Upstream - PIC Issue - M5

Does M5-2 have a higher priority? Vrrp won't load balance you have to make vrrp groups primary on M5-1 and primary on M5-2 to load balance. It doesn't do it like GLBP. It is strictly active/passive. You may want to turn pre-empt on M5-1 since he is priority of 100 and for him to take over for his neighbor with 254 pre-empt may be required.

-Adam
Copyright© 1999-2013 Juniper Networks, Inc. All rights reserved.