Routing
Highlighted
Routing

Multiple paths for VRFs questions ?

[ Edited ]
‎03-11-2020 07:44 PM

I come across some puzzles when I lab VRFs with multiple MPLS paths with different IGP.

This is my topology:

foo-vrf.JPG

Two VRFs on each site, JCust-1 and JCust-2

MX-PE1 and MX-PE2 have two lPs for their Lo0 interface

MX-PE1 <---------> MX-RR (route-reflector) <--------> MX-PE2 (ISIS as IGP) with one of the Lo0 IP advertised on each router.

MX-PE1 <---------> MX-P (P router) <--------> MX-PE2 (OSPF as IGP) with one of the Lo0 IP advertised on each router.

Each MX-PE router has two BGP groups with lo0 address as local-address.

My questions are :

when all the BGPs are up, the remote VRFs routes are hidden for one path.

JCust-1.inet.0: 6 destinations, 8 routes (6 active, 0 holddown, 2 hidden)
+ = Active Route, - = Last Active, * = Both

1.1.1.1/32         *[BGP/170] 01:36:15, MED 1, localpref 100, from 172.17.20.3
                      AS path: I, validation-state: unverified
                    > to 172.16.2.2 via ge-0/0/2.0, Push 300192
10.11.1.0/24       *[BGP/170] 01:36:14, localpref 100, from 172.17.20.3
                      AS path: I, validation-state: unverified
                    > to 172.16.2.2 via ge-0/0/2.0, Push 300192

JCust-1.inet.0: 6 destinations, 8 routes (6 active, 0 holddown, 2 hidden)
1.1.1.1/32 (2 entries, 1 announced)
TSI:
KRT in-kernel 1.1.1.1/32 -> {indirect(1048574)}
RIP route tag 0; no poison reverse
     nbr ge-0/0/0.11: 1.1.1.1/255.255.255.255, met: 1, nh: 0.0.0.0
         BGP    Preference: 170/-101
                Route Distinguisher: 10.11.1.1:1
                Next hop type: Unusable
                Address: 0x9293e84
                Next-hop reference count: 8
                State: <Secondary Hidden Int Ext ProtectionCand>
                Inactive reason: Unusable path
                Local AS: 65123 Peer AS: 65123
                Age: 1:37:21    Metric: 1
                Validation State: unverified
                Task: BGP_65123.172.18.20.1+55378
                AS path: I
                Communities: target:65111:1 rte-type:0.0.0.10:1:0
                Import Accepted
                VPN Label: 300048
                Localpref: 100
                Router ID: 172.18.20.1
                Primary Routing Table bgp.l3vpn.0
                Indirect next hops: 1
                        Protocol next hop: 172.18.20.1   
                        Label operation: Push 300048
                        Label TTL action: prop-ttl
                        Load balance label: Label 300048: None;
                        Indirect next hop: 0x0 - INH Session ID: 0x0

10.11.1.0/24 (2 entries, 1 announced)
TSI:
KRT in-kernel 10.11.1.0/24 -> {indirect(1048574)}
RIP route tag 0; no poison reverse
     nbr ge-0/0/0.11: 10.11.1.0/255.255.255.0, met: 1, nh: 0.0.0.0
         BGP    Preference: 170/-101
                Route Distinguisher: 10.11.1.1:1
                Next hop type: Unusable
                Address: 0x9293e84
                Next-hop reference count: 8
                State: <Secondary Hidden Int Ext ProtectionCand>
                Inactive reason: Unusable path
                Local AS: 65123 Peer AS: 65123
                Age: 1:37:21
                Validation State: unverified
                Task: BGP_65123.172.18.20.1+55378
                AS path: I
                Communities: target:65111:1
                Import Accepted
                VPN Label: 300048
                Localpref: 100
                Router ID: 172.18.20.1
                Primary Routing Table bgp.l3vpn.0
                Indirect next hops: 1
                        Protocol next hop: 172.18.20.1
                        Label operation: Push 300048
                        Label TTL action: prop-ttl
                        Load balance label: Label 300048: None;
                        Indirect next hop: 0x0 - INH Session ID: 0x0

[edit]

1. why the VRF routes from MX-PE1 <------> MX-P <-------> MX-PE2 are hidden.

2. how does  VRFs select the Path MX-PE1 <--------> MX-RR <--------> MX-PE2 ?

Notes:

All the loopback IPs are reachable via appropriate IGPs.

if I shut down MX-RR and configure lo0 IPs for the path (MX-PE1 <-------> MX-P <------->MX-PE2) to primary, the  VRF routes are not hidden any more. 

I try to understand why before I move to how to control which path is selected for a particular VRF (I asked in a different thread already before I noticed this behavior)

 

thanks a lot in advance for your coaching !!

 

All the configurations are:

set system host-name MX-PE1
set interfaces ge-0/0/0 vlan-tagging
set interfaces ge-0/0/0 unit 11 vlan-id 11
set interfaces ge-0/0/0 unit 11 family inet address 10.11.1.1/24
set interfaces ge-0/0/0 unit 12 vlan-id 12
set interfaces ge-0/0/0 unit 12 family inet address 10.11.1.1/24
set interfaces ge-0/0/0 unit 13 vlan-id 13
set interfaces ge-0/0/0 unit 13 family inet address 10.11.2.1/24
set interfaces ge-0/0/0 unit 14 vlan-id 14
set interfaces ge-0/0/0 unit 14 family inet address 10.11.2.1/24
set interfaces ge-0/0/1 unit 0 family inet address 172.16.1.1/30
set interfaces ge-0/0/1 unit 0 family iso
set interfaces ge-0/0/1 unit 0 family mpls
set interfaces ge-0/0/3 unit 0 family inet address 192.168.1.1/30
set interfaces ge-0/0/3 unit 0 family iso
set interfaces ge-0/0/3 unit 0 family mpls
set interfaces lo0 unit 0 family inet address 172.17.20.1/32
set interfaces lo0 unit 0 family inet address 172.18.20.1/32
set interfaces lo0 unit 0 family iso address 49.4234.1720.1702.0001.00
set routing-options autonomous-system 65123
set protocols rsvp interface all
set protocols mpls interface ge-0/0/3.0
set protocols mpls interface ge-0/0/1.0
set protocols bgp group ToMX-RR type internal
set protocols bgp group ToMX-RR local-address 172.17.20.1
set protocols bgp group ToMX-RR family inet unicast
set protocols bgp group ToMX-RR family inet-vpn any
set protocols bgp group ToMX-RR neighbor 172.17.20.3
set protocols bgp group ToMX-PE2 type internal
set protocols bgp group ToMX-PE2 local-address 172.18.20.1
set protocols bgp group ToMX-PE2 family inet unicast
set protocols bgp group ToMX-PE2 family inet-vpn any
set protocols bgp group ToMX-PE2 neighbor 172.18.20.2
set protocols isis export viaISIS
set protocols isis level 1 disable
set protocols isis interface ge-0/0/1.0
set protocols isis interface ge-0/0/3.0
set protocols isis interface lo0.0
set protocols ospf area 0.0.0.10 interface ge-0/0/3.0
set protocols ospf area 0.0.0.10 interface 172.18.20.1
set protocols ldp interface ge-0/0/1.0
set protocols ldp interface ge-0/0/3.0
set policy-options policy-statement BGP2Local term t1 from protocol bgp
set policy-options policy-statement BGP2Local term t1 then accept
set policy-options policy-statement NHS then next-hop self
set policy-options policy-statement viaISIS term t1 from route-filter 172.17.20.1/32 exact
set policy-options policy-statement viaISIS term t1 from route-filter 172.16.1.0/30 exact
set policy-options policy-statement viaISIS term t1 then accept
set policy-options policy-statement viaISIS term t2 then reject
set routing-instances JCust-1 instance-type vrf
set routing-instances JCust-1 interface ge-0/0/0.11
set routing-instances JCust-1 route-distinguisher 10.11.1.1:1
set routing-instances JCust-1 vrf-target import target:65111:1
set routing-instances JCust-1 vrf-target export target:65111:1
set routing-instances JCust-1 protocols ospf export BGP2Local
set routing-instances JCust-1 protocols ospf area 0.0.0.10 interface ge-0/0/0.11
set routing-instances JCust-2 instance-type vrf
set routing-instances JCust-2 interface ge-0/0/0.12
set routing-instances JCust-2 route-distinguisher 10.11.1.1:2
set routing-instances JCust-2 vrf-target import target:65111:2
set routing-instances JCust-2 vrf-target export target:65111:2
set routing-instances JCust-2 protocols ospf export BGP2Local
set routing-instances JCust-2 protocols ospf area 0.0.0.10 interface ge-0/0/0.12

set system host-name MX-PE2
set interfaces ge-0/0/0 vlan-tagging
set interfaces ge-0/0/0 unit 11 vlan-id 11
set interfaces ge-0/0/0 unit 11 family inet address 10.12.1.1/24
set interfaces ge-0/0/0 unit 12 vlan-id 12
set interfaces ge-0/0/0 unit 12 family inet address 10.12.2.1/24
set interfaces ge-0/0/1 unit 0 family inet address 192.168.2.1/30
set interfaces ge-0/0/1 unit 0 family mpls
set interfaces ge-0/0/2 unit 0 family inet address 172.16.2.1/30
set interfaces ge-0/0/2 unit 0 family iso
set interfaces ge-0/0/2 unit 0 family mpls
set interfaces lo0 unit 0 family inet address 172.17.20.2/32
set interfaces lo0 unit 0 family inet address 172.18.20.2/32
set interfaces lo0 unit 0 family iso address 49.4651.1720.1702.0002.00
set routing-options autonomous-system 65123
set protocols rsvp interface ge-0/0/2.0
set protocols rsvp interface ge-0/0/1.0
set protocols mpls interface ge-0/0/1.0
set protocols mpls interface ge-0/0/2.0
set protocols bgp group ToMX-RR type internal
set protocols bgp group ToMX-RR local-address 172.17.20.2
set protocols bgp group ToMX-RR family inet unicast
set protocols bgp group ToMX-RR family inet-vpn any
set protocols bgp group ToMX-RR neighbor 172.17.20.3
set protocols bgp group ToMX-PE1 type internal
set protocols bgp group ToMX-PE1 local-address 172.18.20.2
set protocols bgp group ToMX-PE1 family inet unicast
set protocols bgp group ToMX-PE1 family inet-vpn any
set protocols bgp group ToMX-PE1 neighbor 172.18.20.1
set protocols isis export viaISIS
set protocols isis level 1 disable
set protocols isis interface ge-0/0/1.0
set protocols isis interface ge-0/0/2.0
set protocols isis interface lo0.0
set protocols ospf area 0.0.0.10 interface ge-0/0/1.0
set protocols ospf area 0.0.0.10 interface 172.18.20.2
set protocols ldp interface ge-0/0/1.0
set protocols ldp interface ge-0/0/2.0
set policy-options policy-statement BGP2Local term t1 from protocol bgp
set policy-options policy-statement BGP2Local term t1 then accept
set policy-options policy-statement viaISIS term t1 from route-filter 172.17.20.2/32 exact
set policy-options policy-statement viaISIS term t1 from route-filter 172.16.2.0/30 exact
set policy-options policy-statement viaISIS term t1 then accept
set policy-options policy-statement viaISIS term t2 then reject
set routing-instances JCust-1 instance-type vrf
set routing-instances JCust-1 interface ge-0/0/0.11
set routing-instances JCust-1 route-distinguisher 10.12.1.1:1
set routing-instances JCust-1 vrf-target import target:65111:1
set routing-instances JCust-1 vrf-target export target:65111:1
set routing-instances JCust-1 protocols rip group rip-group export BGP2Local
set routing-instances JCust-1 protocols rip group rip-group neighbor ge-0/0/0.11
set routing-instances JCust-2 instance-type vrf
set routing-instances JCust-2 interface ge-0/0/0.12
set routing-instances JCust-2 route-distinguisher 10.12.1.1:2
set routing-instances JCust-2 vrf-target import target:65111:2
set routing-instances JCust-2 vrf-target export target:65111:2
set routing-instances JCust-2 protocols ospf export BGP2Local
set routing-instances JCust-2 protocols ospf area 0.0.0.10 interface ge-0/0/0.12

set system host-name MX-RR
set interfaces ge-0/0/1 unit 0 family inet address 172.16.1.2/30
set interfaces ge-0/0/1 unit 0 family iso
set interfaces ge-0/0/1 unit 0 family mpls
set interfaces ge-0/0/2 unit 0 family inet address 172.16.2.2/30
set interfaces ge-0/0/2 unit 0 family iso
set interfaces ge-0/0/2 unit 0 family mpls
set interfaces lo0 unit 0 family inet address 172.17.20.3/32
set interfaces lo0 unit 0 family iso address 49.4651.1720.1702.0003.00
set routing-options autonomous-system 65123
set protocols rsvp interface ge-0/0/2.0
set protocols rsvp interface ge-0/0/1.0
set protocols mpls interface ge-0/0/2.0
set protocols mpls interface ge-0/0/1.0
set protocols bgp group ToMXPE type internal
set protocols bgp group ToMXPE local-address 172.17.20.3
set protocols bgp group ToMXPE family inet unicast
set protocols bgp group ToMXPE family inet-vpn any
set protocols bgp group ToMXPE export NHS
set protocols bgp group ToMXPE cluster 172.17.20.3
set protocols bgp group ToMXPE neighbor 172.17.20.1
set protocols bgp group ToMXPE neighbor 172.17.20.2
set protocols isis interface ge-0/0/1.0
set protocols isis interface ge-0/0/2.0
set protocols isis interface lo0.0
set protocols ldp interface ge-0/0/1.0
set protocols ldp interface ge-0/0/2.0
set policy-options policy-statement NHS then next-hop self

set system host-name MX-P
set interfaces ge-0/0/1 unit 0 family inet address 192.168.2.2/30
set interfaces ge-0/0/1 unit 0 family mpls
set interfaces ge-0/0/3 unit 0 family inet address 192.168.1.2/30
set interfaces ge-0/0/3 unit 0 family mpls
set interfaces lo0 unit 0 family inet address 172.17.20.4/32
set protocols rsvp interface all
set protocols mpls interface ge-0/0/3.0
set protocols mpls interface ge-0/0/1.0
set protocols ospf area 0.0.0.10 interface lo0.0 passive
set protocols ospf area 0.0.0.10 interface ge-0/0/1.0
set protocols ospf area 0.0.0.10 interface ge-0/0/3.0
set protocols ldp interface ge-0/0/1.0
set protocols ldp interface ge-0/0/3.0



 

 

6 REPLIES 6
Highlighted
Routing

Re: Multiple paths for VRFs questions ?

‎03-12-2020 07:42 AM

Hello,

Route is complainaing protocol next-hop 172.18.20.1.

Can you please share "show route 172.18.20.1 table inet.3" output in problem condition. We can if there is something wrong.

Thanks

Vishal

 

Highlighted
Routing

Re: Multiple paths for VRFs questions ?

‎03-12-2020 08:27 AM

I just found something related to inet.3

when lo0 interfaces are configured as follows on both PE routers

root@MX-PE1# show interfaces lo0 | display set
set interfaces lo0 unit 0 family inet address 172.17.20.1/32
set interfaces lo0 unit 0 family inet address 172.18.20.1/32

root@MX-PE2# show interfaces lo0 | display set
set interfaces lo0 unit 0 family inet address 172.17.20.2/32
set interfaces lo0 unit 0 family inet address 172.18.20.2/32

the inet.3 on PE2 as follows:

root@MX-PE2# run show route | find inet.3

inet.3: 2 destinations, 2 routes (2 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

172.17.20.1/32     *[LDP/9] 00:04:10, metric 1
                    > to 172.16.2.2 via ge-0/0/2.0, Push 300368
172.17.20.3/32     *[LDP/9] 00:04:24, metric 1
                    > to 172.16.2.2 via ge-0/0/2.0

root@MX-PE2# run show route table JCust-1.inet.0 hidden

JCust-1.inet.0: 6 destinations, 8 routes (6 active, 0 holddown, 2 hidden)
+ = Active Route, - = Last Active, * = Both

1.1.1.1/32          [BGP/170] 00:05:28, MED 1, localpref 100, from 172.18.20.1
                      AS path: I, validation-state: unverified
                      Unusable
10.11.1.0/24        [BGP/170] 00:05:28, localpref 100, from 172.18.20.1
                      AS path: I, validation-state: unverified
                      Unusable

when lo0 interfaces are configured as follows:

root@MX-PE1# show interfaces lo0 | display set
set interfaces lo0 unit 0 family inet address 172.17.20.1/32
set interfaces lo0 unit 0 family inet address 172.18.20.1/32 primary
root@MX-PE2# show interfaces lo0 | display set
set interfaces lo0 unit 0 family inet address 172.17.20.2/32
set interfaces lo0 unit 0 family inet address 172.18.20.2/32 primary

The path fails over

root@MX-PE2# run show route | find inet.3

inet.3: 2 destinations, 2 routes (2 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

172.17.20.4/32     *[LDP/9] 00:01:15, metric 1
                    > to 192.168.2.2 via ge-0/0/1.0
172.18.20.1/32     *[LDP/9] 00:01:10, metric 1
                    > to 192.168.2.2 via ge-0/0/1.0, Push 300000

All the VRF routes fail over.

Anyway to make inet.3 has both paths ?

thanks !

Highlighted
Routing

Re: Multiple paths for VRFs questions ?

‎03-13-2020 07:22 AM

found docs from Juniper for this:

Prior to Junos OS Release 19.1R1, LDP provided support only for router-ID or the interface address as the transport address on any LDP interface. The adjacencies formed on that interface used one of the IP addresses assigned to the interface or the router-ID. In case of targeted adjacency, the interface is the loopback interface. When multiple loopback addresses were configured on the device, the transport address could not be derived for the interface, and as a result, the LDP session could not be established.

Highlighted
Routing

Re: Multiple paths for VRFs questions ?

‎03-13-2020 09:04 AM

Hi

 

You have made a huge mistake to configure NHS on the RR


Mengzhe Hu
JNCIE x 3 (SP DC ENT)
Highlighted
Routing

Re: Multiple paths for VRFs questions ?

‎03-13-2020 09:15 AM

I see the confusion is caused by multiple IP addresses on lo0. My personal opinion is not to use that like. We have rarely seen any ISP implemention like this.

 

You can try to use RSVP and create multiple LSPs, each to a seperate lo0 IP. If the LSP comes up, your routes should also be resolved 


Mengzhe Hu
JNCIE x 3 (SP DC ENT)
Highlighted
Routing

Re: Multiple paths for VRFs questions ?

‎03-13-2020 09:28 AM

Hello,

 

So as I mentioned in previosu respobse issue was related with next-hop resolution as route was not present in the inet.3 table.

By default on Junos, a router will only map a Label for its primary loopback Interface. It means that an LDP router is by default Egress PE only for its /32 primary lo0 address.

 

When you have more than one IP address its recommended to configure a primary/preffered loopback address which you are using as local-address in BGP.

 

If you want to use secondary IP address for MPLS, you can achieve that by LDP export policy. But, frankly I haven't seen many such implemtnation.

 

This is a really good blog on LDP operation, you can also learn how to use secondary loopback for MPLS: https://www.inetzero.com/no-more-doubt-about-ldp/

 

PS: Please mark my response as solution if it answers your query, kudos are appreciated too!

 

Thanks

Vishal

Feedback