Routing

Hi Guys,

Just while doing some labbing I noticed that when 'auto-export' feature is used to leak VPN routes from one VRF to another(which seems to work just fine), these route are not advertised to remote PE routers via MP-BGP??

This is effectively exactly same thing as shown here -> https://www.saidvandeklundert.nl/hub-and-spoke-vpn-on-mx-with-1-interface.php 

However in my example the routes that are leaked using auto-export feature are never redistributed to other PE routers. I need to say that I am using logical systems and logical interfaces but I have never run into any L3VPN limiations like that before so don't see if that could be some sort of limitation.

Why are the routes not redistributed, its super weird as when I configure sample static route in same VRF it is redistributed via MP-BGP instantly and visible on all relevant router in bgp.l3vpn + VRF tables, so that verifies RT must be correct.

See below for some more specific configuration regarding my setup, let me know if more is needed and I will add to the thread.

Main VRF that learns the routes from other SPOKES and CE-HUB router (default route advertised from CE-HUB):

r4@MX480_LAB_1_RE0:R4> show configuration routing-instances BGP-HUB-SPOKE-1-INT->HUB | display set
set logical-systems R4 routing-instances BGP-HUB-SPOKE-1-INT->HUB instance-type vrf
set logical-systems R4 routing-instances BGP-HUB-SPOKE-1-INT->HUB interface lt-0/0/10.6
set logical-systems R4 routing-instances BGP-HUB-SPOKE-1-INT->HUB vrf-import SPOKE-IN
set logical-systems R4 routing-instances BGP-HUB-SPOKE-1-INT->HUB vrf-export HUB-OUT
set logical-systems R4 routing-instances BGP-HUB-SPOKE-1-INT->HUB no-vrf-advertise
set logical-systems R4 routing-instances BGP-HUB-SPOKE-1-INT->HUB routing-options auto-export
set logical-systems R4 routing-instances BGP-HUB-SPOKE-1-INT->HUB protocols bgp group CE4 type external
set logical-systems R4 routing-instances BGP-HUB-SPOKE-1-INT->HUB protocols bgp group CE4 peer-as 65020
set logical-systems R4 routing-instances BGP-HUB-SPOKE-1-INT->HUB protocols bgp group CE4 as-override
set logical-systems R4 routing-instances BGP-HUB-SPOKE-1-INT->HUB protocols bgp group CE4 neighbor 172.16.0.22

VRF to which default route is leaked so it can be redistributed to other sites vie MP-BGP:

r4@MX480_LAB_1_RE0:R4> show configuration routing-instances BGP-HUB-SPOKE-1-INT->HUB-2 | display set
set logical-systems R4 routing-instances BGP-HUB-SPOKE-1-INT->HUB-2 instance-type vrf
set logical-systems R4 routing-instances BGP-HUB-SPOKE-1-INT->HUB-2 vrf-target target:65412:100
set logical-systems R4 routing-instances BGP-HUB-SPOKE-1-INT->HUB-2 vrf-table-label
set logical-systems R4 routing-instances BGP-HUB-SPOKE-1-INT->HUB-2 routing-options static route 1.1.1.1/32 next-table BGP-HUB-SPOKE-1-INT->HUB.inet.0 <- sample static route to test general MP-BGP + RT, this gets propagated wihtout any issues
set logical-systems R4 routing-instances BGP-HUB-SPOKE-1-INT->HUB-2 routing-options auto-export

r4@MX480_LAB_1_RE0:R4> show configuration policy-options policy-statement HUB-OUT | display set
set logical-systems R4 policy-options policy-statement HUB-OUT term 1 from protocol bgp
set logical-systems R4 policy-options policy-statement HUB-OUT term 1 from protocol direct
set logical-systems R4 policy-options policy-statement HUB-OUT term 1 from route-filter 0.0.0.0/0 exact
set logical-systems R4 policy-options policy-statement HUB-OUT term 1 from route-filter 172.16.0.20/30 exact
set logical-systems R4 policy-options policy-statement HUB-OUT term 1 then community add HUB
set logical-systems R4 policy-options policy-statement HUB-OUT term 1 then accept

r4@MX480_LAB_1_RE0:R4> show configuration policy-options policy-statement SPOKE-IN | display set
set logical-systems R4 policy-options policy-statement SPOKE-IN term 1 from protocol bgp
set logical-systems R4 policy-options policy-statement SPOKE-IN term 1 from community SPOKE
set logical-systems R4 policy-options policy-statement SPOKE-IN term 1 then accept

r4@MX480_LAB_1_RE0:R4> show configuration policy-options community HUB | display set
set logical-systems R4 policy-options community HUB members target:65412:100

r4@MX480_LAB_1_RE0:R4> show configuration policy-options community SPOKE | display set
set logical-systems R4 policy-options community SPOKE members target:65412:200

and view from inside VRF routing table that the routes are leaked into, from this table I would expect routes to be exported into bgp.l3vpn table and to other PE routers via MP-BGP, but somehow doesn't work for leaked prefiexes:

r4@MX480_LAB_1_RE0:R4> show route table BGP-HUB-SPOKE-1-INT->HUB-2 detail

BGP-HUB-SPOKE-1-INT->HUB-2.inet.0: 3 destinations, 3 routes (3 active, 0 holddown, 0 hidden)
0.0.0.0/0 (1 entry, 1 announced)
*BGP Preference: 170/-101
Next hop type: Router, Next hop index: 2178
Address: 0xa6017f4
Next-hop reference count: 4
Source: 172.16.0.22
Next hop: 172.16.0.22 via lt-0/0/10.6, selected
Session Id: 0x2f9
State: <Secondary Active Ext>
Peer AS: 65020
Age: 1:24:00
Validation State: unverified
Task: BGP_65020.172.16.0.22
Announcement bits (1): 0-KRT
AS path: 65020 I
Communities: target:65412:100
Accepted
Localpref: 100
Router ID: 172.16.0.22
Primary Routing Table BGP-HUB-SPOKE-1-INT->HUB.inet.0

1.1.1.1/32 (1 entry, 1 announced)
*Static Preference: 5
Next table: BGP-HUB-SPOKE-1-INT->HUB.inet.0
Next-hop index: 1957
Address: 0xa601ae8
Next-hop reference count: 5
State: <Active Int Ext>
Age: 55:39
Validation State: unverified
Task: RT
Announcement bits (2): 0-KRT 1-rt-export
AS path: I

172.16.0.20/30 (1 entry, 1 announced)
*Direct Preference: 0
Next hop type: Interface, Next hop index: 0
Address: 0xa5fe224
Next-hop reference count: 2
Next hop: via lt-0/0/10.6, selected
State: <Secondary Active Int>
Age: 1:24:04
Validation State: unverified
Task: IF
Announcement bits (1): 0-KRT
AS path: I
Communities: target:65412:100
Primary Routing Table BGP-HUB-SPOKE-1-INT->HUB.inet.0

Routes are also not present in the local bgp.l3vpn table(except for sample static route):

r4@MX480_LAB_1_RE0:R4> show route table bgp.l3vpn

bgp.l3vpn.0: 5 destinations, 5 routes (5 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

10.0.3.4:76:1.1.1.1/32
*[Static/5] 01:02:13
to table BGP-HUB-SPOKE-1-INT->HUB.inet.0

Hopefully someone has seen this before at some stage and can advice.

Thanks,
Arthur

Hello,

Of course it was seen before umpteen times and explained in this blog post from 2013:

https://forums.juniper.net/t5/Archive/Using-rib-groups-or-auto-export-for-route-leaking/ba-p/202349

Please have a good read and pay attention to this wording, it explains everything You see in Your lab

only the primary route is then redistributed via MP-BGP

Please start using rib-groups in Your lab to meet Your requirements.

HTH

Thx

Alex

Hi,

Thanks for the response and I am very aware of the blog you shared and have read it numerous times before posting. If I understand it correctly(see posts/comments that followed the main post on that thread) routes that are learnt by BGP(PE<>CE) are still eligible to be redistributed via MP-BGP(when in new VRF) even when auto-export was used to share them. Please correct me if im wrong here.

See below the part im talking about, for full post see other thread on the link you posted:

"...In your case, you had eBGP as PE-CE protocol, and it can then legitimately pick up this sibling path from the RIB to be exported to the CE. That was the reason in your case for the flag to appear with auto-export: the route was being selected by PE-CE eBGP for advertisement...."

Also same seems to be demonstrated on the reference link I shared initially, also eBGP is used as PE<>CE and it works using 'auto-export' -> https://www.saidvandeklundert.nl/hub-and-spoke-vpn-on-mx-with-1-interface.php

On another note I already tried to use RIB groups to copy the same routes between VRF tables and I achieved the same result. Routes are shared but not exported by MP-BGP. The output of the routing table is exactly the same as when auto-export is used. For whatever reasone they will just not be picked up???

This is why I created this thread. Let me know if you would like me to revert to RIB-groups config again and share the configuration + routing table output.

Thanks,

Arthur

Hello,

You won't be able to export STATIC route shared via auto-export, period. This is explained in the 2013 blog post.

Now, please share Your RIB-GROUP config where You "achieved the same result".

Thx

Alex

Hi,

The static route is only used in this example to confirm route-target is correct and that there is no issue with MP-BGP topology, it is not part of the scenario as such. If you look at my configs the static route was never shared between VRFs it was created in the destination VRF. Anyhow to avoid the confusion I have removed it now.

Please see same result below when RIB-group is used:

r4@MX480_LAB_1_RE0:R4# show routing-instances BGP-HUB-SPOKE-1-INT->HUB | display set

set logical-systems R4 routing-instances BGP-HUB-SPOKE-1-INT->HUB instance-type vrf
set logical-systems R4 routing-instances BGP-HUB-SPOKE-1-INT->HUB interface lt-0/0/10.6
set logical-systems R4 routing-instances BGP-HUB-SPOKE-1-INT->HUB vrf-import SPOKE-IN
set logical-systems R4 routing-instances BGP-HUB-SPOKE-1-INT->HUB vrf-export HUB-OUT
set logical-systems R4 routing-instances BGP-HUB-SPOKE-1-INT->HUB no-vrf-advertise
set logical-systems R4 routing-instances BGP-HUB-SPOKE-1-INT->HUB routing-options interface-routes rib-group inet BGP-2-to-BGP
set logical-systems R4 routing-instances BGP-HUB-SPOKE-1-INT->HUB protocols bgp group CE4 type external
set logical-systems R4 routing-instances BGP-HUB-SPOKE-1-INT->HUB protocols bgp group CE4 peer-as 65020
set logical-systems R4 routing-instances BGP-HUB-SPOKE-1-INT->HUB protocols bgp group CE4 as-override
set logical-systems R4 routing-instances BGP-HUB-SPOKE-1-INT->HUB protocols bgp group CE4 neighbor 172.16.0.22 family inet unicast rib-group BGP-2-to-BGP

r4@MX480_LAB_1_RE0:R4# show routing-instances BGP-HUB-SPOKE-1-INT->HUB-2 | display set
set logical-systems R4 routing-instances BGP-HUB-SPOKE-1-INT->HUB-2 instance-type vrf
set logical-systems R4 routing-instances BGP-HUB-SPOKE-1-INT->HUB-2 vrf-target target:65412:100
set logical-systems R4 routing-instances BGP-HUB-SPOKE-1-INT->HUB-2 vrf-table-label

r4@MX480_LAB_1_RE0:R4# show routing-options rib-groups BGP-2-to-BGP | display set
set logical-systems R4 routing-options rib-groups BGP-2-to-BGP import-rib BGP-HUB-SPOKE-1-INT->HUB.inet.0
set logical-systems R4 routing-options rib-groups BGP-2-to-BGP import-rib BGP-HUB-SPOKE-1-INT->HUB-2.inet.0
set logical-systems R4 routing-options rib-groups BGP-2-to-BGP import-policy LEAK-POLICY

r4@MX480_LAB_1_RE0:R4# show policy-options policy-statement LEAK-POLICY | display set
set logical-systems R4 policy-options policy-statement LEAK-POLICY term 1 from protocol bgp
set logical-systems R4 policy-options policy-statement LEAK-POLICY term 1 from protocol direct
set logical-systems R4 policy-options policy-statement LEAK-POLICY term 1 from route-filter 0.0.0.0/0 exact
set logical-systems R4 policy-options policy-statement LEAK-POLICY term 1 from route-filter 172.16.0.20/30 exact
set logical-systems R4 policy-options policy-statement LEAK-POLICY then accept

Below is routing table view of "BGP-HUB-SPOKE-1-INT->HUB-2" VRF, rib-group is working as expected but again default route is not exported via MP-BGP, this route is learnt via eBGP between CE<>PE in the primary VRF so in my understanding it should be eligible for MP-BGP redistribution after 'leaking' using both techniques -> 'auto-export' and 'rib-groups', bet let's try to figure this out using rib-groups for now.

r4@MX480_LAB_1_RE0:R4> show route table BGP-HUB-SPOKE-1-INT->HUB-2.inet.0 detail

BGP-HUB-SPOKE-1-INT->HUB-2.inet.0: 3 destinations, 3 routes (3 active, 0 holddown, 0 hidden)
0.0.0.0/0 (1 entry, 1 announced)
*BGP Preference: 170/-101
Next hop type: Router, Next hop index: 2178
Address: 0xa6017f4
Next-hop reference count: 4
Source: 172.16.0.22
Next hop: 172.16.0.22 via lt-0/0/10.6, selected
Session Id: 0x2f9
State: <Secondary Active Ext>
Peer AS: 65020
Age: 6:25
Validation State: unverified
Task: BGP_65020.172.16.0.22
Announcement bits (1): 0-KRT
AS path: 65020 I
Accepted
Localpref: 100
Router ID: 172.16.0.22
Primary Routing Table BGP-HUB-SPOKE-1-INT->HUB.inet.0

172.16.0.20/30 (1 entry, 1 announced)
*Direct Preference: 0
Next hop type: Interface, Next hop index: 0
Address: 0xa5fe224
Next-hop reference count: 2
Next hop: via lt-0/0/10.6, selected
State: <Secondary Active Int>
Age: 6:25
Validation State: unverified
Task: IF
Announcement bits (1): 0-KRT
AS path: I
Primary Routing Table BGP-HUB-SPOKE-1-INT->HUB.inet.0

172.16.0.21/32 (1 entry, 1 announced)
*Local Preference: 0
Next hop type: Local, Next hop index: 0
Address: 0x95ef0c4
Next-hop reference count: 35
Next hop:
Interface: lt-0/0/10.6
State: <Secondary Active NoReadvrt Int>
Age: 6:25
Validation State: unverified
Task: IF
Announcement bits (1): 0-KRT
AS path: I
Primary Routing Table BGP-HUB-SPOKE-1-INT->HUB.inet.0

Let me know what you think.

Thanks,

Arthur

Hello,

Thanks for clarifying Your scenario.

I labbed it up using separate VMX JUNOS 18.3R1 instances as opposed to Your LS and it works fine with Your RIB-GROUP config meaning the 0/0 is sento via MP-BGP to the tailend PE.

The only difference I see is that Your leaked routes have only KRT bit set but in my lab they also have 1-BGP_RT_Background bit set.

Announcement bits (2): 0-KRT 1-BGP_RT_Background 

The significance of this bit is explained in the 2013 blog post.

HTH

Thx

Alex

Hi Alex,

Thanks for great and fast feedback, and indeed I also started to suspect the missing bit could be the issue why the routes are never picked up by MP-BGP. I have not been able to find a way to change this in my setup no matter which way I try to set this up. What I also find interesting is the fact that this bit is not set in either primary VRF or the secondary VRF on eBGP locally learnt CE->PE routes, not exactly sure why this is or if there is a way to change that, but routes get picked up by MP-BGP from primary VRF without issue.

r4@MX480_LAB_1_RE0:R4> show route table BGP-HUB-SPOKE-1-INT->HUB.inet.0 exact 0/0 detail

BGP-HUB-SPOKE-1-INT->HUB.inet.0: 7 destinations, 7 routes (7 active, 0 holddown, 0 hidden)
0.0.0.0/0 (1 entry, 1 announced)
*BGP Preference: 170/-101
Next hop type: Router, Next hop index: 2178
Address: 0xa6017f4
Next-hop reference count: 4
Source: 172.16.0.22
Next hop: 172.16.0.22 via lt-0/0/10.6, selected
Session Id: 0x2f9
State: <Active Ext>
Peer AS: 65020
Age: 17:45:21
Validation State: unverified
Task: BGP_65020.172.16.0.22
Announcement bits (1): 0-KRT
AS path: 65020 I
Accepted
Localpref: 100
Router ID: 172.16.0.22
Secondary Tables: BGP-HUB-SPOKE-1-INT->HUB-2.inet.0

r4@MX480_LAB_1_RE0:R4> show route table BGP-HUB-SPOKE-1-INT->HUB-2.inet.0 exact 0/0 detail

BGP-HUB-SPOKE-1-INT->HUB-2.inet.0: 3 destinations, 3 routes (3 active, 0 holddown, 0 hidden)
0.0.0.0/0 (1 entry, 1 announced)
*BGP Preference: 170/-101
Next hop type: Router, Next hop index: 2178
Address: 0xa6017f4
Next-hop reference count: 4
Source: 172.16.0.22
Next hop: 172.16.0.22 via lt-0/0/10.6, selected
Session Id: 0x2f9
State: <Secondary Active Ext>
Peer AS: 65020
Age: 1:44:54
Validation State: unverified
Task: BGP_65020.172.16.0.22
Announcement bits (1): 0-KRT
AS path: 65020 I
Accepted
Localpref: 100
Router ID: 172.16.0.22
Primary Routing Table BGP-HUB-SPOKE-1-INT->HUB.inet.0

At least you were able to confirm the config is correct and it should work. So my guess is that it is the cobmination of JunosVersion/LogicalSystmes/LogicalTunnels that introduces this. Would you agree?

I'm running following code/hardware if anyone is curious -> 15.1F6-S8.1 on a real MX480 hardware.

Thanks,

Arthur

Hello,

By default, JUNOS checks for AS_PATH loops for all ASNs locally-configured inside Logical Systems/VRs/VRFs.

This may be contributing to Your issue.

You may get around it if You configure "independent-domain" or "loops" but I suggest You try with separate MX/VMX nodes to make sure. And this code is ancient, it just went End of Support on 5 Dec 2018 (15.1F6-S10 is still supported)

https://support.juniper.net/support/eol/software/junos/

HTH

Thx

Alex

Hi Alex,

Fair enough, but these features should really work on any version of code from 15.x family regardless.

Most important fact is that the config itself is good. If I get a chance I will spin up few vMX routers and give it a test.

BTW this router already has 'as-loops' enabled so that it would be capable of simulating other scenarios, but as you say there is no point trying to make it work on old version of code using complex setup of logical-systems. We already know that it should and does work when using separate vMX boxes 🙂 I just wanted to clarify if I was missing something silly here that was breaking the setup.

Thanks a lot for your help.

Thanks,

Arthur