Ethernet Switching
Ethernet Switching

Interface errors on AE, not physical ports

a week ago

So i've been investigating issues on an interface where I see the following on the ae when I run "show interfaces extensive ae9" 

Input errors: Errors: 28182975, Drops: 0, Framing errors: 28156078, Runts: 0, Giants: 26897, Policed discards: 0, Resource errors: 0

 

I don't see these errors on the physical ports that make up the ae when I run the same command on them.  Is there a way to diagnose which link is responsible other than just checking every individual cable and component?

7 REPLIES 7
Ethernet Switching

Re: Interface errors on AE, not physical ports

a week ago

What platform is it? If there's no physical port reporting framing error, but only ae reporting, this is something interesting. May not be related with L1. What is remote side?

 

Please get following:

1. config on ae

2. show interface xx extensive (for both ae and member link)

3. More information on the remote / transmission devices 


Mengzhe Hu
JNCIE x 3 (SP DC ENT)
Ethernet Switching

Re: Interface errors on AE, not physical ports

a week ago

another way to check. Remove all the interfaces from ae bundle. If this is physical issue, you should see framing error on one or several interfaces. This way, you don't need to go on site and switching cable optics


Mengzhe Hu
JNCIE x 3 (SP DC ENT)
Ethernet Switching

Re: Interface errors on AE, not physical ports

Wednesday

Hi

 

This does not look like an issue with physical interface. 

What you can do is find out if these errors are incrementing due to any single port or if it is being incremented by all the ports. So if this is incrementing cumulatively for all the ports then it is layer 2 framing errors which could be caused by some of the frames received on this ae bundle from the remote side which are being classified as framing errors for those packets. 

 

Thanks

Ethernet Switching

Re: Interface errors on AE, not physical ports

Thursday

Framing errors are typically related to physical issues on the cable or SFPs. See https://kb.juniper.net/InfoCenter/index?page=content&id=KB27597&cat=JUNOS&actp=LIST

 

 As per your output, the port is also reporting 26897 giant packets, which are packets over the size the interface supports. You may want to check the MTU settings on both peers as well.

 

 

Ethernet Switching

Re: Interface errors on AE, not physical ports

Monday

Sorry for the delay in response.  This is a QFX-5100 VC.  The remote side is a Tintri VMstore T-850.  I have them cabled using DAC cables from Juniper and I've got another identical Tintri unit on this switch cabled the same way and no errors so it can't be an incompatibility.  Here's the output requested

 

 

%> show configuration | display set | match ae9
set interfaces xe-0/0/11 ether-options 802.3ad ae9
set interfaces xe-1/0/11 ether-options 802.3ad ae9
set interfaces ae9 description ********
set interfaces ae9 aggregated-ether-options minimum-links 1
set interfaces ae9 aggregated-ether-options lacp active
set interfaces ae9 aggregated-ether-options lacp periodic slow
set interfaces ae9 unit 0 family ethernet-switching interface-mode trunk
set interfaces ae9 unit 0 family ethernet-switching vlan members 504-505

{master:1}
%> show interfaces ae9 extensive
Physical interface: ae9, Enabled, Physical link is Up
  Interface index: 649, SNMP ifIndex: 563, Generation: 217
  Description: SP: AE, Mall-Tintri-2:Data B
  Link-level type: Ethernet, MTU: 1514, Speed: 20Gbps, BPDU Error: None, MAC-REWRITE Error: None, Loopback: Disabled, Source filtering: Disabled, Flow control: Disabled, Minimum links needed: 1,
  Minimum bandwidth needed: 0
  Device flags   : Present Running
  Interface flags: SNMP-Traps Internal: 0x4000
  Current address: 10:0e:7e:b3:4b:49, Hardware address: 10:0e:7e:b2:98:89
  Last flapped   : 2019-07-31 11:16:40 EDT (6w4d 20:53 ago)
  Statistics last cleared: Never
  Traffic statistics:
   Input  bytes  :      353521069280834                    0 bps
   Output bytes  :       74239734949079                 1128 bps
   Input  packets:         241388622839                    0 pps
   Output packets:          67732251873                    2 pps
   IPv6 transit statistics:
    Input  bytes  :                   0
    Output bytes  :                   0
    Input  packets:                   0
    Output packets:                   0
  Input errors:
    Errors: 28183181, Drops: 0, Framing errors: 28156284, Runts: 0, Giants: 26897, Policed discards: 0, Resource errors: 0
  Output errors:
    Carrier transitions: 62, Errors: 0, Drops: 0, MTU errors: 0, Resource errors: 0
  Egress queues: 12 supported, 5 in use
  Queue counters:       Queued packets  Transmitted packets      Dropped packets
    0 best-effort                    0          67473706689                    0
    3 fcoe                           0                    0                    0
    4 no-loss                        0                    0                    0
    7 network-cont                   0              9718718                    0
    8 mcast                          0            248826472                    0
  Queue number:         Mapped forwarding classes
    0                   best-effort
    3                   fcoe
    4                   no-loss
    7                   network-control
    8                   mcast

  Logical interface ae9.0 (Index 627) (SNMP ifIndex 628) (Generation 355)
    Flags: SNMP-Traps 0x20024000 Encapsulation: Ethernet-Bridge
    Statistics        Packets        pps         Bytes          bps
    Bundle:
        Input : 4918700452339422880          0 8079587992859            0
        Output: 8164732829696          0 1776827211271309528            0
    Link:
      xe-0/0/11.0
        Input :             0          0             0            0
        Output:       2505316          0     856817813            0
      xe-1/0/11.0
        Input :             0          0             0            0
        Output:       2513073          0     859470966            0
    LACP info:        Role     System             System      Port    Port  Port
                             priority          identifier  priority  number   key
      xe-0/0/11.0    Actor        127  10:0e:7e:b3:4a:80       127      32    10
      xe-0/0/11.0  Partner      65535  90:e2:ba:a5:9b:ec       255       2    33
      xe-1/0/11.0    Actor        127  10:0e:7e:b3:4a:80       127      34    10
      xe-1/0/11.0  Partner      65535  90:e2:ba:a5:9b:ec       255       1    33
    LACP Statistics:       LACP Rx     LACP Tx   Unknown Rx   Illegal Rx
      xe-0/0/11.0          2301125     2318030            0            0
      xe-1/0/11.0          2320450     2325221            0            0
    Marker Statistics:   Marker Rx     Resp Tx   Unknown Rx   Illegal Rx
      xe-0/0/11.0                0           0            0            0
      xe-1/0/11.0                0           0            0            0
    Protocol eth-switch, MTU: 1514, Generation: 375, Route table: 6
      Flags: Trunk-Mode

{master:1}
%> show interfaces xe-0/0/11 extensive
Physical interface: xe-0/0/11, Enabled, Physical link is Up
  Interface index: 731, SNMP ifIndex: 537, Generation: 307
  Description: SP: DAC, Mall-Tintri-2:Data B:2a
  Link-level type: Ethernet, MTU: 1514, Speed: 10Gbps, BPDU Error: None, MAC-REWRITE Error: None, Loopback: Disabled, Source filtering: Disabled, Flow control: Disabled, Media type: Fiber
  Device flags   : Present Running
  Interface flags: SNMP-Traps Internal: 0x4000
  Link flags     : None
  CoS queues     : 12 supported, 12 maximum usable queues
  Hold-times     : Up 0 ms, Down 0 ms
  Current address: 10:0e:7e:b3:4b:49, Hardware address: 10:0e:7e:b2:97:ce
  Last flapped   : 2019-07-31 11:16:38 EDT (6w4d 20:53 ago)
  Statistics last cleared: Never
  Traffic statistics:
   Input  bytes  :       58609636476750                    0 bps
   Output bytes  :       35280860920615                 1072 bps
   Input  packets:          42055024966                    0 pps
   Output packets:          35504738199                    1 pps
   IPv6 transit statistics:
    Input  bytes  :                   0
    Output bytes  :                   0
    Input  packets:                   0
    Output packets:                   0
  Input errors:
    Errors: 28183181, Drops: 0, Framing errors: 28156284, Runts: 0, Policed discards: 0, L3 incompletes: 0, L2 channel errors: 0, L2 mismatch timeouts: 0, FIFO errors: 0, Resource errors: 0
  Output errors:
    Carrier transitions: 31, Errors: 0, Drops: 0, Collisions: 0, Aged packets: 0, FIFO errors: 0, HS link CRC errors: 0, MTU errors: 0, Resource errors: 0
  Egress queues: 12 supported, 5 in use
  Queue counters:       Queued packets  Transmitted packets      Dropped packets
    0 best-effort                    0          35341674250                    0
    3 fcoe                           0                    0                    0
    4 no-loss                        0                    0                    0
    7 network-cont                   0              4823350                    0
    8 mcast                          0            158240600                    0
  Queue number:         Mapped forwarding classes
    0                   best-effort
    3                   fcoe
    4                   no-loss
    7                   network-control
    8                   mcast
  Active alarms  : None
  Active defects : None
  MAC statistics:                      Receive         Transmit
    Total octets                58609636476750   35280860920615
    Total packets                  42055024966      35504738199
    Unicast packets                42041774992      35343375621
    Broadcast packets                 10949169        156539071
    Multicast packets                  2300805          4823507
    CRC/Align errors                  28156284                0
    FIFO errors                              0                0
    MAC control frames                       0                0
    MAC pause frames                       302                0
    Oversized frames                     26897
    Jabber frames                        81940
    Fragment frames                      39786
    VLAN tagged frames             42052724161
    Code violations                          0
  MAC Priority Flow Control Statistics:
    Priority :  0                             0                0
    Priority :  1                             0                0
    Priority :  2                             0                0
    Priority :  3                             0                0
    Priority :  4                             0                0
    Priority :  5                             0                0
    Priority :  6                             0                0
    Priority :  7                             0                0
  Filter statistics:
    Input packet count                       0
    Input packet rejects                     0
    Input DA rejects                         0
    Input SA rejects                         0
    Output packet count                                       0
    Output packet pad count                                   0
    Output packet error count                                 0
    CAM destination filters: 1, CAM source filters: 0
  Packet Forwarding Engine configuration:
    Destination slot: 0 (0x00)
  CoS information:
    Direction : Output
    CoS transmit queue               Bandwidth               Buffer Priority   Limit
                              %            bps     %           usec
    0 best-effort             5      500000000     5              0      low    none
    3 fcoe                   35     3500000000    35              0      low    none
    4 no-loss                35     3500000000    35              0      low    none
    7 network-control         5      500000000     5              0      low    none
    8 mcast                  20     2000000000    20              0      low    none
  Interface transmit statistics: Disabled

  Logical interface xe-0/0/11.0 (Index 595) (SNMP ifIndex 647) (Generation 404)
    Flags: SNMP-Traps 0x24024000 Encapsulation: Ethernet-Bridge
    Traffic statistics:
     Input  bytes  :                39928
     Output bytes  :            856818155
     Input  packets:                  322
     Output packets:              2505317
    Local statistics:
     Input  bytes  :                39928
     Output bytes  :            856818155
     Input  packets:                  322
     Output packets:              2505317
    Transit statistics:
     Input  bytes  :                    0                    0 bps
     Output bytes  :                    0                    0 bps
     Input  packets:                    0                    0 pps
     Output packets:                    0                    0 pps
    Protocol aenet, AE bundle: ae9.0, Generation: 436, Route table: 0

{master:1}
%> show interfaces xe-1/0/11 extensive
Physical interface: xe-1/0/11, Enabled, Physical link is Up
  Interface index: 720, SNMP ifIndex: 648, Generation: 294
  Description: SP: DAC, Mall-Tintri-2:Data B:2b
  Link-level type: Ethernet, MTU: 1514, Speed: 10Gbps, BPDU Error: None, MAC-REWRITE Error: None, Loopback: Disabled, Source filtering: Disabled, Flow control: Disabled, Media type: Fiber
  Device flags   : Present Running
  Interface flags: SNMP-Traps Internal: 0x4000
  Link flags     : None
  CoS queues     : 12 supported, 12 maximum usable queues
  Hold-times     : Up 0 ms, Down 0 ms
  Current address: 10:0e:7e:b2:98:89, Hardware address: 10:0e:7e:b3:4a:8e
  Last flapped   : 2019-07-31 11:16:38 EDT (6w4d 20:53 ago)
  Statistics last cleared: Never
  Traffic statistics:
   Input  bytes  :      294911432805108                    0 bps
   Output bytes  :       38958874043758                  616 bps
   Input  packets:         199333597887                    0 pps
   Output packets:          32227513897                    1 pps
   IPv6 transit statistics:
    Input  bytes  :                   0
    Output bytes  :                   0
    Input  packets:                   0
    Output packets:                   0
  Input errors:
    Errors: 0, Drops: 0, Framing errors: 0, Runts: 0, Policed discards: 0, L3 incompletes: 0, L2 channel errors: 0, L2 mismatch timeouts: 0, FIFO errors: 0, Resource errors: 0
  Output errors:
    Carrier transitions: 31, Errors: 0, Drops: 0, Collisions: 0, Aged packets: 0, FIFO errors: 0, HS link CRC errors: 0, MTU errors: 0, Resource errors: 0
  Egress queues: 12 supported, 5 in use
  Queue counters:       Queued packets  Transmitted packets      Dropped packets
    0 best-effort                    0          32132032451                    0
    3 fcoe                           0                    0                    0
    4 no-loss                        0                    0                    0
    7 network-cont                   0              4895374                    0
    8 mcast                          0             90586072                    0
  Queue number:         Mapped forwarding classes
    0                   best-effort
    3                   fcoe
    4                   no-loss
    7                   network-control
    8                   mcast
  Active alarms  : None
  Active defects : None
  MAC statistics:                      Receive         Transmit
    Total octets               294911432805108   38958874043758
    Total packets                 199333597887      32227513897
    Unicast packets               199331228092      32133188619
    Broadcast packets                    21972         89429742
    Multicast packets                  2347823          4895536
    CRC/Align errors                         0                0
    FIFO errors                              0                0
    MAC control frames                       0                0
    MAC pause frames                       292                0
    Oversized frames                         0
    Jabber frames                            0
    Fragment frames                          0
    VLAN tagged frames            199331250064
    Code violations                          0
  MAC Priority Flow Control Statistics:
    Priority :  0                             0                0
    Priority :  1                             0                0
    Priority :  2                             0                0
    Priority :  3                             0                0
    Priority :  4                             0                0
    Priority :  5                             0                0
    Priority :  6                             0                0
    Priority :  7                             0                0
  Filter statistics:
    Input packet count                       0
    Input packet rejects                     0
    Input DA rejects                         0
    Input SA rejects                         0
    Output packet count                                       0
    Output packet pad count                                   0
    Output packet error count                                 0
    CAM destination filters: 1, CAM source filters: 0
  Packet Forwarding Engine configuration:
    Destination slot: 0 (0x00)
  CoS information:
    Direction : Output
    CoS transmit queue               Bandwidth               Buffer Priority   Limit
                              %            bps     %           usec
    0 best-effort             5      500000000     5              0      low    none
    3 fcoe                   35     3500000000    35              0      low    none
    4 no-loss                35     3500000000    35              0      low    none
    7 network-control         5      500000000     5              0      low    none
    8 mcast                  20     2000000000    20              0      low    none
  Interface transmit statistics: Disabled

  Logical interface xe-1/0/11.0 (Index 635) (SNMP ifIndex 663) (Generation 391)
    Flags: SNMP-Traps 0x20020000 Encapsulation: Ethernet-Bridge
    Traffic statistics:
     Input  bytes  :                 1612
     Output bytes  :            859471650
     Input  packets:                   13
     Output packets:              2513075
    Local statistics:
     Input  bytes  :                 1612
     Output bytes  :            859471650
     Input  packets:                   13
     Output packets:              2513075
    Transit statistics:
     Input  bytes  :                    0                    0 bps
     Output bytes  :                    0                    0 bps
     Input  packets:                    0                    0 pps
     Output packets:                    0                    0 pps
    Protocol aenet, AE bundle: ae9.0, Generation: 420, Route table: 0

{master:1}

 

 

 

 

 

Ethernet Switching
Solution
Accepted by topic author lcroce
Monday

Re: Interface errors on AE, not physical ports

[ Edited ]
Monday

@lcroce wrote:

I don't see these errors on the physical ports that make up the ae when I run the same command on them.  Is there a way to diagnose which link is responsible other than just checking every individual cable and component?


 

Unless I'm misunderstanding the question, member xe-0/0/11 has the exact same error count as the ae. 

 

  Input errors:
    Errors: 28183181, Drops: 0, Framing errors: 28156284, Runts: 0, Policed discards: 0, L3 incompletes: 0, L2 channel errors: 0, L2 mismatch timeouts: 0, FIFO errors: 0, Resource errors: 0

 

Ethernet Switching

Re: Interface errors on AE, not physical ports

Monday

You're right.  I somehow missed that when I was initially looking over this.  Thanks.