Switching

last person joined: 3 days ago 

Ask questions and share experiences about EX and QFX portfolios and all switching solutions across your data center, campus, and branch locations.
  • 1.  MTU issue with vlans

    Posted 10-11-2019 03:20
      |   view attached

    We have incomprehensible problems: MTU issue after make a vlan setup.

     

    Config at MX480 side:

    show configuration interfaces ae1

    vlan-tagging;
    mtu 1518;
    aggregated-ether-options {
    minimum-links 1;
    link-speed 10g;
    lacp {
    active;
    periodic fast;
    }
    }
    unit 300 {
    description localnet;
    vlan-id 300;
    family inet {
    rpf-check {
    mode loose;
    }
    address x.x.x.x {}
    ...
    }
    unit 350 {
    ...
    }
    ...
    }

     

     

    Config at EX4550-VC

     

    admin# show interfaces ae1 
    description "to MX480";
    aggregated-ether-options {
        minimum-links 1;
        link-speed 10g;
        lacp {
            active;
            periodic fast;
        }
    }
    unit 0 {
        family ethernet-switching {
            port-mode trunk;
            vlan {
                members [ vlan_300 vlan_350 vlan_400 vlan_450 vlan_500 vlan_550 vlan_800 ];
            }
            native-vlan-id 1;
            filter {
                input udpRate;
            }
        }
    }

    I try add/remove mtu 1518; at MX480 config.

    MTU size:

    MX480:

    show interfaces ae1 detail | grep MTU 
      Link-level type: Ethernet, MTU: 1518, Speed: 80Gbps, BPDU Error: None, MAC-REWRITE Error: None, Loopback: Disabled, Source filtering: Disabled, Flow control: Disabled
        Protocol inet, MTU: 1500

    EX4550:

    admin# run show interfaces ae1 | grep MTU 
      Link-level type: Ethernet, MTU: 1514, Speed: 80Gbps, BPDU Error: None,

    Server connected to port of 4550:

    admin# run show interfaces xe-2/0/13 
    Physical interface: xe-2/0/13, Enabled, Physical link is Up
      Interface index: 152, SNMP ifIndex: 675
    Link-level type: Ethernet, MTU: 1514, Speed: 10Gbps, Duplex: Full-Duplex,

    MTU issue at server:

    wget -O /dev/null https://mirror.leaseweb.com/speedtest/10000mb.bin
    --2019-10-11 04:08:51--  https://mirror.leaseweb.com/speedtest/10000mb.bin
    Resolving mirror.leaseweb.com (mirror.leaseweb.com)... 37.58.58.140, 2a00:c98:2030:a034::21
    Connecting to mirror.leaseweb.com (mirror.leaseweb.com)|37.58.58.140|:443... connected.
    -- stuck --
    
    root@ubuntu:~# ifconfig enp13s0f1 mtu 1496
    root@ubuntu:~# wget -O /dev/null https://mirror.leaseweb.com/speedtest/10000mb.bin
    --2019-10-11 05:16:09--  https://mirror.leaseweb.com/speedtest/10000mb.bin
    Resolving mirror.leaseweb.com (mirror.leaseweb.com)... 37.58.58.140, 2a00:c98:2030:a034::21
    Connecting to mirror.leaseweb.com (mirror.leaseweb.com)|37.58.58.140|:443... connected.
    HTTP request sent, awaiting response... 200 OK
    Length: 10000000000 (9.3G) [application/octet-stream]
    Saving to: ‘/dev/null’
    
    /dev/null                                                    5%[======>                                                                                                                                  ] 514.04M   284MB/s               ^

    Any ideas - how to fix it???

    P.S.

    admin# run show version
    Model: ex4550-32f
    JUNOS Base OS boot [12.3R12-S14]

    --

    show version
    Model: mx480
    Junos: 17.3R3-S4.2

     

     

     

     



  • 2.  RE: MTU issue with vlans

     
    Posted 10-11-2019 03:29

    Hello,

     

    Can you try to configure MTU 1518 on both sides of your link between MX and EX? You had MTU mismatch, which may very well be the root cause of your problem.

     

    Best regards,

    Sergii

    -------------------------------------------------------------------

    Please accept the solution if your problem is resolved Smiley Happy

    -------------------------------------------------------------------



  • 3.  RE: MTU issue with vlans

    Posted 10-11-2019 04:02

    I have already read a ton of literature about "size MTU at EX switches".

    Like:

    https://lists.gt.net/nsp/juniper/25466

    https://packet-expert.org/2016/09/23/junos-mtu-handling-on-access-trunk-ports/

    In all source say:

    "

    Obviously its creating confusion, if trunk  interface is showing MTU value of 1514 then how it will receive packet with 1500 bytes payload + 18 bytes header . But the matter of the fact is , this interface will receive payload size of 1500 bytes and header size of 18 bytes even with MTU value displayed in CLI as 1514 .

    Conclusion

    • Trunk ports- Even though MTU size displayed in CLI is 1514 bytes but at hardware level 1518 bytes are handled for 802.1Q packets.

    "

    On the other hand, I have a other connection from this MX480 to the switch QFX1002:

    admin@QFX10002-nl-1> show interfaces ae1
    Physical interface: ae1 (MC-AE-1, active), Enabled, Physical link is Up
    Interface index: 149, SNMP ifIndex: 531
    Description: uplink to MX480
    Link-level type: Ethernet, MTU: 1514, Speed: 100Gbps, BPDU Error: None, Ethernet-Switching Error: None, MAC-REWRITE Error: None, Loopback: Disabled, Source filtering: Disabled, Flow control: Disabled, Minimum links needed: 1,
    Minimum bandwidth needed: 1bps
    Device flags : Present Running

    No issue.

    I have other POP. Other MX480 connected to QFX5100-VC. No issue.

    admin@P23> show interfaces ae0    
    Physical interface: ae0, Enabled, Physical link is Up
      Interface index: 695, SNMP ifIndex: 607
      Description: to MX480
      Link-level type: Ethernet, MTU: 1514, Speed: 180Gbps, BPDU Error: None, MAC-REWRITE Error: None, Loopback: Disabled, Source filtering: Disabled, Flow control: Disabled, Minimum links needed: 1, Minimum bandwidth needed: 1bps
      Device flags   : Present Running

    P.S. Of course I try at EX4550:

    admin# set interfaces ae1 mtu 1518 
    
    {master:2}[edit]
    admin# commit synchronize 
    fpc2: 
    configuration check succeeds
    fpc3: 
    commit complete
    fpc2: 
    commit complete
    
    {master:2}[edit]
    

    Not solved.

    root@ubuntu:~# wget -O /dev/null https://mirror.leaseweb.com/speedtest/10000mb.bin
    --2019-10-11 06:01:46--  https://mirror.leaseweb.com/speedtest/10000mb.bin
    Resolving mirror.leaseweb.com (mirror.leaseweb.com)... 37.58.58.140, 2a00:c98:2030:a034::21
    Connecting to mirror.leaseweb.com (mirror.leaseweb.com)|37.58.58.140|:443... connected.
    

    P.P.S.

    admin# run show ethernet-switching interfaces ae1 detail 
    Interface: ae1.0, Index: 126, State: up, Port mode: Trunk
    Native vlan: native
    Ether type for the interface: 0x8100
    VLAN membership:
        native, 802.1Q Tag: 1, untagged, msti-id: 0, unblocked
        vlan_300, 802.1Q Tag: 300, tagged, msti-id: 0, unblocked
        vlan_350, 802.1Q Tag: 350, tagged, msti-id: 0, unblocked
        vlan_400, 802.1Q Tag: 400, tagged, msti-id: 0, unblocked
        vlan_450, 802.1Q Tag: 450, tagged, msti-id: 0, unblocked
        vlan_500, 802.1Q Tag: 500, tagged, msti-id: 0, unblocked
        vlan_550, 802.1Q Tag: 550, tagged, msti-id: 0, unblocked
        vlan_800, 802.1Q Tag: 800, tagged, msti-id: 0, unblocked
    Number of MACs learned on IFL: 24
    
    {master:2}[edit]
    
    admin# show interfaces xe-2/0/13 
    unit 0 {
        family ethernet-switching {
            vlan {
                members vlan_550;
            }
            filter {
                input 2g;
            }
        }
    }


  • 4.  RE: MTU issue with vlans

     
    Posted 10-11-2019 07:10

    In my opinion, your configuration on both MX and EX should include properly calculated MTU, regardless of the behavior you described. More detailed description of this behavior can be found here:

    https://kb.juniper.net/InfoCenter/index?page=content&id=KB23906&smlogin=true 

     

    It boils down to the fact that MRU is always bigger then MTU on EX switches. However, even if MRU allows you to receive slightly bigger (e.g. tagged packets), MTU configuration should control whether such frames can be sent out of the interface.

     

    When you say there is a QFX connected to the same MX, and "there is no issue" - did you test using the same steps (wget on the directly connected to QFX server)?

     

    Is MX in your topology the default gateway, and EX VC is a pure L2 switch? If yes, you should be able to use ping with DF bit set to identify the actual maximum MTU between MX and the server.

     

    Best regards,

    Sergii

    -------------------------------------------------------------------

    Please accept the solution if your problem is resolved

    -------------------------------------------------------------------



  • 5.  RE: MTU issue with vlans

    Posted 10-11-2019 08:04

    1) yes .. same test

    QFX: always success

    root@ubuntu:~# ifconfig p6p1 | grep MTU
              UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
    root@ubuntu:~# wget -O /dev/null https://mirror.leaseweb.com/speedtest/10000mb.bin
    --2019-10-11 16:51:54--  https://mirror.leaseweb.com/speedtest/10000mb.bin
    Resolving mirror.leaseweb.com (mirror.leaseweb.com)... 5.79.108.33, 2001:1af8:4700:b210::33
    Connecting to mirror.leaseweb.com (mirror.leaseweb.com)|5.79.108.33|:443... connected.
    HTTP request sent, awaiting response... 200 OK
    Length: 10000000000 (9.3G) [application/octet-stream]
    Saving to: ‘/dev/null’
    
     5% [=========>                                                                                                                                                                                         ] 558,767,760  381MB/s             ^C
    root@ubuntu:~# wget -O /dev/null https://mirror.leaseweb.com/speedtest/10000mb.bin
    --2019-10-11 16:51:58--  https://mirror.leaseweb.com/speedtest/10000mb.bin
    Resolving mirror.leaseweb.com (mirror.leaseweb.com)... 5.79.108.33, 2001:1af8:4700:b210::33
    Connecting to mirror.leaseweb.com (mirror.leaseweb.com)|5.79.108.33|:443... connected.
    HTTP request sent, awaiting response... 200 OK
    Length: 10000000000 (9.3G) [application/octet-stream]
    Saving to: ‘/dev/null’
    
     3% [=====>                                                                                                                                                                                             ] 322,117,264  307MB/s             ^C
    root@ubuntu:~# wget -O /dev/null https://mirror.leaseweb.com/speedtest/10000mb.bin
    --2019-10-11 16:52:01--  https://mirror.leaseweb.com/speedtest/10000mb.bin
    Resolving mirror.leaseweb.com (mirror.leaseweb.com)... 5.79.108.33, 2001:1af8:4700:b210::33
    Connecting to mirror.leaseweb.com (mirror.leaseweb.com)|5.79.108.33|:443... connected.
    HTTP request sent, awaiting response... 200 OK
    Length: 10000000000 (9.3G) [application/octet-stream]
    Saving to: ‘/dev/null’
    
     4% [=======>   

    2) 

    Correct. L3 border - MX480. EX/QFX - L2.

    To server behind (after) EX:

    ping -s $((1500 - 28)) -D xx.220.40.214  -c 1               
    PING xx.220.40.214 (xx.220.40.214) 1472(1500) bytes of data.
    ^C
    --- xx.220.40.214 ping statistics ---
    1 packets transmitted, 0 received, 100% packet loss, time 0ms
    

    To server behind (after) QFX:

    ping -s $((1500 - 28)) -D xx.191.126.216 -c 1
    PING xx.191.126.216 (xx.191.126.216) 1472(1500) bytes of data.
    [1570805684.140263] 1480 bytes from xx.191.126.216: icmp_req=1 ttl=56 time=49.2 ms
    
    --- xx.191.126.216 ping statistics ---
    1 packets transmitted, 1 received, 0% packet loss, time 0ms
    rtt min/avg/max/mdev = 49.267/49.267/49.267/0.000 ms

    From internet. From one place. I checked the network path matches, only different interfaces on the last inch (Mx480 to EX subnet or MX480 to QFX subnet). The path from the Internet to the router is the same.

    P.S.

    EX server
    root@ubuntu:~# ifconfig | grep mtu
    enp13s0f1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
    lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
    lxcbr0: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500
    MTU
    ping -s $((1500 - 32)) -D xx.220.40.214  -c 1 
    PING xx.220.40.214 (xx.220.40.214) 1468(1496) bytes of data.
    [1570806644.226991] 1476 bytes from xx.220.40.214: icmp_req=1 ttl=56 time=51.5 ms
    
    --- xx.220.40.214 ping statistics ---
    1 packets transmitted, 1 received, 0% packet loss, time 0ms

     



  • 6.  RE: MTU issue with vlans
    Best Answer

    Posted 10-11-2019 09:09

    It looks like the problem is solved.

    The problem was at the level of L1/L2 between EX switch and MX480. 

    it turned out there was not a dark fiber, but some interfaces went through the switch of service provider ..

    Thanks for the discussion - experiments with ping prompted the right thought.