Ethernet Switching
Highlighted
Ethernet Switching

MCLAG and non balanced traffic load

[ Edited ]
‎06-17-2018 03:54 AM

Hi

I have couple of QFX-10Ks which connects two QFX5K in mclag.

on the 5K's there are servers in 4 Vlans - two of them are active on QFX10K1 and two are active on QFX10K2 (VRRP).

it seems that traffic that gets from servers to the 5Ks layer is balanced, but, as both 10K's are connected to two wan interfaces (2x100G) with isis and iBGP, traffic arrives to 10K-2, will pass on to 10K-1 via the ICCP link, and will exit the wan links in 10K-1.

does anyone have an idea why it is happeneing?

 large.png

8 REPLIES 8
Highlighted
Ethernet Switching

Re: MCLAG and non balanced traffic load

‎06-18-2018 07:04 AM

Hello,

Are You using MCLAG with VRRP on QFX10K?

Is QFX10K-1 the only VRRP master for all servers?

If yes+yes, then traffic from QFX10K-2 to QFX10K-1 across ICL is expected.

HTH

Thx

Alex

_____________________________________________________________________

Please ask Your Juniper account team about Juniper Professional Services offerings.
Juniper PS can design, test & build the network/part of the network as per Your requirements

+++++++++++++++++++++++++++++++++++++++++++++

Accept as Solution = cool !
Accept as Solution+Kudo = You are a Star !
Highlighted
Ethernet Switching

Re: MCLAG and non balanced traffic load

‎06-18-2018 07:51 AM

Hi

4 Vlans, 4 VRRP groups, 2 are active on each QFX10K and 2 are standby.

It's obvious that if QFX10K1 was active for all 4 vlans, traffic will be forwarded through the ICCP link, but this is not the case.

 

each server sends same amount of traffic, meaning each vlan generates same traffic rate, each 10K gets more or less same traffic rate. but 3/4 of the traffic get passed the ICCP link to be routed.

 

 1.png

Highlighted
Ethernet Switching

Re: MCLAG and non balanced traffic load

‎06-18-2018 10:58 AM

Hello,


@dudster83 wrote:

Hi

4 Vlans, 4 VRRP groups, 2 are active on each QFX10K and 2 are standby.

 

 

 1.png


Are those 4 VLANs U-shaped? I mean are they extended from QFX10Ks to WAN routers as well?

If yes do You have ISIS running across them?

Are Your WAN routers sending only 0/0 to QFX10Ks?

If yes what is the protocol nexthop for these 0/0s as seen by QFX10Ks?

HTH

Thx

Alex

_____________________________________________________________________

Please ask Your Juniper account team about Juniper Professional Services offerings.
Juniper PS can design, test & build the network/part of the network as per Your requirements

+++++++++++++++++++++++++++++++++++++++++++++

Accept as Solution = cool !
Accept as Solution+Kudo = You are a Star !
Highlighted
Ethernet Switching

Re: MCLAG and non balanced traffic load

‎06-18-2018 12:03 PM

Hi Alex, Thanks for your reply and intrest.

switching ends in the QFX10K, and not extended to the wan routers.

Layer3 for those vlans is configured on the 10K's, those networks are redistributed from connected to BGP towards the wan routers.

P routers (wan routers) sends 0.0.0.0/0 and all IGP table + 0.0.0.0/0 in BGP towards the PEs (10K's)

0.0.0.0 is preferred via isis through the locally connected links to P routers [which I don't understand why it won't go out directly from 10K-2 to the wan, and passes the ICCP link]

 

hopefully I provided all desired info

Thanks again.

Highlighted
Ethernet Switching

Re: MCLAG and non balanced traffic load

‎06-18-2018 01:37 PM

Hello,

If Your VLANs are not extended to WAN routers, and You don't run ISIS on these VLANs, then it is not a routed traffic.

Likely flood traffic - I reckon at least 0.9Gbps out of 4.9Gbps is flood.

My math is 4.9-(3.5+3.2-1.2-1.5))=0.9 

And 4.9-0.9=4 is switched traffic that takes suboptimal path.

Things to check:

1/ are VRRP masters/backups in their proper places? Could it be that QFX10K-1 is actual VRRP master for more than 2 VLANs?

2/ MAC table timeout on QFX5K and QFX10K - must be the same as ARP timeout on Your servers

HTH

Thx

Alex

_____________________________________________________________________

Please ask Your Juniper account team about Juniper Professional Services offerings.
Juniper PS can design, test & build the network/part of the network as per Your requirements

+++++++++++++++++++++++++++++++++++++++++++++

Accept as Solution = cool !
Accept as Solution+Kudo = You are a Star !
Highlighted
Ethernet Switching

Re: MCLAG and non balanced traffic load

[ Edited ]
‎06-19-2018 03:27 AM

Hi

VRRP is active and backup on proper places

10K1.RE0> show vrrp brief
Interface State Group VR state VR Mode Timer Type Address
irb.10 up 10 master Active A 0.036 lcl x.x.x.2
vip x.x.x.1
irb.20 up 20 backup Active D 0.273 lcl x.x.x.34
vip x.x.x.33
mas x.x.x.35
irb.30 up 30 master Active A 0.006 lcl x.x.x.66
vip x.x.x.65
irb.40 up 40 backup Active D 0.318 lcl x.x.x.98
vip x.x.x.99
mas x.x.x.97

 

10K2.RE0> show vrrp brief
Interface State Group VR state VR Mode Timer Type Address
irb.10 up 10 backup Active D 0.326 lcl x.x.x.3
vip x.x.x.1
mas x.x.x.2
irb.20 up 20 master Active A 0.034 lcl x.x.x.35
vip x.x.x.33
irb.30 up 30 backup Active D 0.281 lcl x.x.x.67
vip x.x.x.65
mas x.x.x.66
irb.40 up 40 master Active A 0.063 lcl x.x.x.97
vip x.x.x.99

 

 

regarding the 2nd questions, all 4 switches are configured as MAC aging interval of 300, and LE aging time/LE VLAN aging time of 1200

Highlighted
Ethernet Switching

Re: MCLAG and non balanced traffic load

‎06-19-2018 11:56 AM

Hello,

And what is the ARP timeout in servers? And in QFXes?

Also, do You run server-to-server database replication through QFXes, by any chance?

HTH

Thx
Alex

_____________________________________________________________________

Please ask Your Juniper account team about Juniper Professional Services offerings.
Juniper PS can design, test & build the network/part of the network as per Your requirements

+++++++++++++++++++++++++++++++++++++++++++++

Accept as Solution = cool !
Accept as Solution+Kudo = You are a Star !
Highlighted
Ethernet Switching

Re: MCLAG and non balanced traffic load

‎06-19-2018 12:51 PM

Hi

i do not run server-to-server database replication 

the servers doesn't speak with one another, all traffic is served for a lot of end hosts, just like a cdn would.

i do not have any special aging time out for arp configured on any interface nor on system level, so i guess they're all in default configurations.

regarding the server configs, i can't tell, those are closed boxes i don't have access to.

 

i guess it has something to do with switches configs, same servers with same vrrp solution (active/standby) with other switches vendors equally load balance, so i don't see how servers will age out their arp entries and perform this amound of flood & learn.