Routing

Noticing some strange behaviour on some VQFXs acting as a PE routers to a dual homed CE.

My question is:

In a multihomed setup - should each PE advertise type2 routes to the VTEP?

I have a pretty basic setup. 

There is two PE devices (tor1 & tor2) with a CE device connected to both

The PEs then IBGP peer with a MX for VXLAN routing (this is all done via loopback addressing, underlay is using EBGP)

The MX is acting as a route reflector for each PE

This is what i'm seeing, sometimes both PEs will advertise the type2 routes and sometimes they won't. 

For example:

root@tor1> show route advertising-protocol bgp 192.168.0.1    

default-switch.evpn.0: 42 destinations, 55 routes (42 active, 0 holddown, 0 hidden)
  Prefix                  Nexthop              MED     Lclpref    AS path
  1:192.168.0.5:1::111111111111111111::0/192 AD/EVI            
*                         Self                         100        I
  2:192.168.0.5:1::330::4c:96:14:1e:47:80/304 MAC/IP            
*                         Self                         100        I
  3:192.168.0.5:1::110::192.168.0.5/248 IM                
*                         Self                         100        I
  3:192.168.0.5:1::220::192.168.0.5/248 IM                
*                         Self                         100        I
  3:192.168.0.5:1::330::192.168.0.5/248 IM                
*                         Self                         100        I

__default_evpn__.evpn.0: 3 destinations, 4 routes (3 active, 0 holddown, 0 hidden)
  Prefix                  Nexthop              MED     Lclpref    AS path
  1:192.168.0.5:0::111111111111111111::FFFF:FFFF/192 AD/ESI            
*                         Self                         100        I
  4:192.168.0.5:0::111111111111111111:192.168.0.5/296 ES                
*                         Self                         100        I

root@tor2> show route advertising-protocol bgp 192.168.0.1 

default-switch.evpn.0: 42 destinations, 55 routes (42 active, 0 holddown, 0 hidden)
  Prefix                  Nexthop              MED     Lclpref    AS path
  1:192.168.0.6:1::111111111111111111::0/192 AD/EVI            
*                         Self                         100        I
  2:192.168.0.6:1::330::4c:96:14:1e:47:80/304 MAC/IP            
*                         Self                         100        I
  3:192.168.0.6:1::110::192.168.0.6/248 IM                
*                         Self                         100        I
  3:192.168.0.6:1::220::192.168.0.6/248 IM                
*                         Self                         100        I
  3:192.168.0.6:1::330::192.168.0.6/248 IM                
*                         Self                         100        I

__default_evpn__.evpn.0: 3 destinations, 4 routes (3 active, 0 holddown, 0 hidden)
  Prefix                  Nexthop              MED     Lclpref    AS path
  1:192.168.0.6:0::111111111111111111::FFFF:FFFF/192 AD/ESI            
*                         Self                         100        I
  4:192.168.0.6:0::111111111111111111:192.168.0.6/296 ES                
*                         Self                         100        I

So you can see that both PEs (tor1 and tor2) are advertising the same type2 route for MAC 4c:96:14:1e:47:80.

However, if i simulate a failure or by shutting down one PE > CE link the the type2 route will only be advertised by one PE, even after the link has been restored:

(no type2 route advertised)

root@tor1> show route advertising-protocol bgp 192.168.0.1    

default-switch.evpn.0: 42 destinations, 56 routes (42 active, 0 holddown, 0 hidden)
  Prefix                  Nexthop              MED     Lclpref    AS path
  1:192.168.0.5:1::111111111111111111::0/192 AD/EVI            
*                         Self                         100        I
  3:192.168.0.5:1::110::192.168.0.5/248 IM                
*                         Self                         100        I
  3:192.168.0.5:1::220::192.168.0.5/248 IM                
*                         Self                         100        I
  3:192.168.0.5:1::330::192.168.0.5/248 IM                
*                         Self                         100        I

__default_evpn__.evpn.0: 3 destinations, 4 routes (3 active, 0 holddown, 0 hidden)
  Prefix                  Nexthop              MED     Lclpref    AS path
  1:192.168.0.5:0::111111111111111111::FFFF:FFFF/192 AD/ESI            
*                         Self                         100        I
  4:192.168.0.5:0::111111111111111111:192.168.0.5/296 ES                
*                         Self                         100        I

{master:0}
root@tor1> show route evpn-mac-address 4c:96:14:1e:47:80 

<snip>

bgp.evpn.0: 39 destinations, 54 routes (39 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

2:192.168.0.6:1::330::4c:96:14:1e:47:80/304 MAC/IP        
                   *[BGP/170] 00:12:11, localpref 100, from 192.168.0.1
                      AS path: I, validation-state: unverified
                    > to 192.168.1.16 via xe-0/0/1.0

default-switch.evpn.0: 42 destinations, 56 routes (42 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

2:192.168.0.6:1::330::4c:96:14:1e:47:80/304 MAC/IP        
                   *[BGP/170] 00:12:11, localpref 100, from 192.168.0.1
                      AS path: I, validation-state: unverified
                    > to 192.168.1.16 via xe-0/0/1.0

__default_evpn__.evpn.0: 3 destinations, 4 routes (3 active, 0 holddown, 0 hidden)



root@tor1> show ethernet-switching table interface ae2 

<snip>

Ethernet switching table : 1 entries, 1 learned
Routing instance : default-switch
   Vlan                MAC                 MAC      Logical                Active
   name                address             flags    interface              source
   CUST-3              4c:96:14:1e:47:80   DR       ae2.0                

Other PE, advertising type2

root@tor2> show route advertising-protocol bgp 192.168.0.1    

default-switch.evpn.0: 42 destinations, 55 routes (42 active, 0 holddown, 0 hidden)
  Prefix                  Nexthop              MED     Lclpref    AS path
  1:192.168.0.6:1::111111111111111111::0/192 AD/EVI            
*                         Self                         100        I
  2:192.168.0.6:1::330::4c:96:14:1e:47:80/304 MAC/IP            
*                         Self                         100        I
  3:192.168.0.6:1::110::192.168.0.6/248 IM                
*                         Self                         100        I
  3:192.168.0.6:1::220::192.168.0.6/248 IM                
*                         Self                         100        I
  3:192.168.0.6:1::330::192.168.0.6/248 IM                
*                         Self                         100        I

__default_evpn__.evpn.0: 3 destinations, 4 routes (3 active, 0 holddown, 0 hidden)
  Prefix                  Nexthop              MED     Lclpref    AS path
  1:192.168.0.6:0::111111111111111111::FFFF:FFFF/192 AD/ESI            
*                         Self                         100        I
  4:192.168.0.6:0::111111111111111111:192.168.0.6/296 ES                
*                         Self                         100        I

{master:0}
root@tor2> show route evpn-mac-address 4c:96:14:1e:47:80 

<snip>

default-switch.evpn.0: 42 destinations, 55 routes (42 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

2:192.168.0.6:1::330::4c:96:14:1e:47:80/304 MAC/IP        
                   *[EVPN/170] 00:13:57
                      Indirect

__default_evpn__.evpn.0: 3 destinations, 4 routes (3 active, 0 holddown, 0 hidden)

{master:0}
root@tor2> show ethernet-switching table interface ae2 

<snip>


Ethernet switching table : 1 entries, 1 learned
Routing instance : default-switch
   Vlan                MAC                 MAC      Logical                Active
   name                address             flags    interface              source
   CUST-3              4c:96:14:1e:47:80   DL       ae2.0                

I think that once everything is stable both PEs should be advertising the type2 since this is active-active. Is this behaviour expected or is something wrong perhaps?

Hello,

Your tor1 PE does not advertise type2 route for MAC  4c:96:14:1e:47:80 because it is not locally learned:

---

@Luca wrote:

 

 

(no type2 route advertised)

 

root@tor1> show ethernet-switching table interface ae2 

<snip>

Ethernet switching table : 1 entries, 1 learned
Routing instance : default-switch
   Vlan                MAC                 MAC      Logical                Active
   name                address             flags    interface              source
   CUST-3              4c:96:14:1e:47:80   DR       ae2.0                

 

---

DR means "dynamic, coming from remote PE".

Once tor1 PE learns this MAC over ae2, it will advertise it.

There is nothing tor1 PE can do about this (or any other) particular MAC - until Your CE decides to load-balance a frame with this src MAC to the tor1-facing link. Or until You decide to configure it as static MAC.

HTH

Thx
Alex

also a query

if this is multihoming and you are using an ESI, should the MAC address not be learnt via the ESI, then you need to dig into the ESI itself to see associated VTEPs?

Hello,

Please have a look into this whitepaper, it may answer many of Your questions

https://www.juniper.net/assets/uk/en/local/pdf/whitepapers/2000606-en.pdf

HTH

Thx
Alex

Ah yes you're right!  In my lab traffic was only being hased out one link, so the CE MAC was only being learned on the one link.

I generated some more traffic to different destinations and now see the CE MAC learned locally on each individual PE switch:

root@tor1> show ethernet-switching table interface ae2                      
<snip>
Ethernet switching table : 1 entries, 1 learned
Routing instance : default-switch
   Vlan                MAC                 MAC      Logical                Active
   name                address             flags    interface              source
   CUST-3              4c:96:14:1e:47:80   DLR      ae2.0                

root@tor2> show ethernet-switching table interface ae2                      
<snip>
Ethernet switching table : 1 entries, 1 learned
Routing instance : default-switch
   Vlan                MAC                 MAC      Logical                Active
   name                address             flags    interface              source
   CUST-3              4c:96:14:1e:47:80   DLR      ae2.0                

Now i see both PE switches advertising the type2 route to the MX:

root@tor1> show route advertising protocol bgp 192.168.0.1 evpn-mac-address 4c:96:14:1e:47:80   

default-switch.evpn.0: 39 destinations, 39 routes (39 active, 0 holddown, 0 hidden)
  Prefix                  Nexthop              MED     Lclpref    AS path
  2:192.168.0.5:1::330::4c:96:14:1e:47:80/304 MAC/IP            
*                         Self                         100        I

root@tor2> show route advertising protocol bgp 192.168.0.1 evpn-mac-address 4c:96:14:1e:47:80     

default-switch.evpn.0: 39 destinations, 39 routes (39 active, 0 holddown, 0 hidden)
  Prefix                  Nexthop              MED     Lclpref    AS path
  2:192.168.0.6:1::330::4c:96:14:1e:47:80/304 MAC/IP            
*                         Self                         100        I

And the MX has them both:

root@vtep1> show route evpn-mac-address 4c:96:14:1e:47:80 

<snip>
bgp.evpn.0: 44 destinations, 44 routes (44 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

2:192.168.0.5:1::330::4c:96:14:1e:47:80/304 MAC/IP        
                   *[BGP/170] 00:39:49, localpref 100, from 192.168.0.5
                      AS path: I, validation-state: unverified
                    > to 192.168.1.1 via ge-0/0/0.0
2:192.168.0.6:1::330::4c:96:14:1e:47:80/304 MAC/IP        
                   *[BGP/170] 00:06:44, localpref 100, from 192.168.0.6
                      AS path: I, validation-state: unverified
                    > to 192.168.1.3 via ge-0/0/1.0
                                        
VLAN10.evpn.0: 16 destinations, 16 routes (16 active, 0 holddown, 0 hidden)

VLAN20.evpn.0: 16 destinations, 16 routes (16 active, 0 holddown, 0 hidden)

VLAN30.evpn.0: 21 destinations, 21 routes (21 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

2:192.168.0.5:1::330::4c:96:14:1e:47:80/304 MAC/IP        
                   *[BGP/170] 00:39:49, localpref 100, from 192.168.0.5
                      AS path: I, validation-state: unverified
                    > to 192.168.1.1 via ge-0/0/0.0
2:192.168.0.6:1::330::4c:96:14:1e:47:80/304 MAC/IP        
                   *[BGP/170] 00:06:44, localpref 100, from 192.168.0.6
                      AS path: I, validation-state: unverified
                    > to 192.168.1.3 via ge-0/0/1.0

Thanks!