Junos OS

Hello,

I have some issue on my network.

topology is like that:

2 QFX- 5100 ROLE: root and backup

16 Line cards (ex-4300 & qfx 5100)

The network is running video streaming, start from Source which is video sender, and reciver is Loader.

Each loader register to multicast group on IRB 71

Each Source send to multicast group on IRB 70

 

When the system reaches around 23 g/bps Drop & Discards start to apear heavily.

just to mention the ports are connected with 40 gb cables.

No cos/qos implention .

Recommended version is running on each Switch.

 

does any one knows where to troubleshoot this? is this normal?

 

root> show interfaces ae14 extensive
Physical interface: ae14, Enabled, Physical link is Up
Interface index: 654, SNMP ifIndex: 671, Generation: 169
Description: LOADER - 10.71.60.102
Link-level type: Ethernet, MTU: 1514, Speed: 20Gbps, BPDU Error: None, MAC-REWRITE Error: None, Loopback: Disabled,
Source filtering: Disabled, Flow control: Disabled, Minimum links needed: 1, Minimum bandwidth needed: 1bps
Device flags : Present Running
Interface flags: SNMP-Traps Internal: 0x4000
Current address: d4:04:ff:94:f6:fe, Hardware address: d4:04:ff:94:f6:fe
Last flapped : 2018-06-13 10:48:33 IDT (7w1d 01:36 ago)
Statistics last cleared: 2018-07-29 18:15:39 IDT (3d 18:08 ago)
Traffic statistics:
Input bytes : 1515850564345 77838320 bps
Output bytes : 358613423879547 16235213648 bps
Input packets: 21889643076 144225 pps
Output packets: 241592637925 1349066 pps
IPv6 transit statistics:
Input bytes : 0
Output bytes : 0
Input packets: 0
Output packets: 0
Input errors:
Errors: 0, Drops: 0, Framing errors: 0, Runts: 0, Giants: 0, Bucket drops: 0, Policed discards: 0, Resource errors: 0
Output errors:
Carrier transitions: 0, Errors: 0, Drops: 1124985593, MTU errors: 0, Resource errors: 0, Bucket drops: 0
Egress queues: 12 supported, 5 in use
Queue counters: Queued packets Transmitted packets Dropped packets
0 0 241591445231 1124985593
3 0 0 0
4 0 0 0
7 0 215177 0
8 0 97541 0
Queue number: Mapped forwarding classes
0 best-effort
3 fcoe
4 no-loss
7 network-control
8 mcast

Logical interface ae14.0 (Index 558) (SNMP ifIndex 1676) (Generation 167)
Flags: Up SNMP-Traps 0x24024000 Encapsulation: Ethernet-Bridge
Statistics Packets pps Bytes bps
Bundle:
Input : 0 0 0 0
Output: 0 0 0 0
Adaptive Statistics:
Adaptive Adjusts: 0
Adaptive Scans : 0
Adaptive Updates: 0
Link:
xe-6/0/10.0
Input : 0 0 0 0
Output: 11697 0 3824919 0
xe-6/0/11.0
Input : 0 0 0 0
Output: 11692 0 3823284 0
xe-6/0/11.0
Input : 0 0 0 0
Output: 11692 0 3823284 0
LACP info: Role System System Port Port Port
priority identifier priority number key
xe-6/0/10.0 Actor 127 d4:04:ff:94:f6:00 127 34 15
xe-6/0/10.0 Partner 65535 00:0a:f7:6e:bd:40 255 1 11
xe-6/0/11.0 Actor 127 d4:04:ff:94:f6:00 127 35 15
xe-6/0/11.0 Partner 65535 00:0a:f7:6e:bd:40 255 2 11
xe-6/0/11.0 Actor 127 d4:04:ff:94:f6:00 127 35 15
xe-6/0/11.0 Partner 65535 00:0a:f7:6e:bd:40 255 2 11
LACP Statistics: LACP Rx LACP Tx Unknown Rx Illegal Rx
xe-6/0/10.0 324520 10816 0 0
xe-6/0/11.0 324519 10816 0 0
xe-6/0/11.0 324519 10816 0 0
Marker Statistics: Marker Rx Resp Tx Unknown Rx Illegal Rx
xe-6/0/10.0 0 0 0 0
xe-6/0/11.0 0 0 0 0
xe-6/0/11.0 0 0 0 0
Protocol eth-switch, MTU: 1514, Generation: 186, Route table: 4

Hi,

 

i think your problem is scaling of the virtual chassis. This technology officially only supports up to 10 members per virtual chassis. You are on 18 members which is way over qualified limits.

 

I would recommend you to split the setup into two seperate virtual chassis and then interconnect them with at least 2 x 40G bundled in an ae-interface.

 

From the QFX5100 datasheet:

Virtual Chassis Technology
The QFX5100 supports Juniper Networks’ unique Virtual Chassis 
technology, which enables up to 10 interconnected switches to 
be managed and operated as a single, logical device with a single 
IP address.

 

Ty for your comments.
If it said the virtual chassis can support up to 10 manged switches.. why I can add more then 10?

Well, Juniper and Junos have many scenarios where you can configure over the limits of supported numbers.

Supported means a design/configuration which has been through proper testing and validation.

 

BUT: In this case I was wrong.  After more detailed review of your configuration I see that you have a  "Virtual chassis fabric" (VCF) which is a spine/leaf design compared to virtual chassis which usually is a ring topology. VCF supports up to 20 switches so you are within limits.

 

MDT is also configured on both spines... to be honest I don't have any firm ideas about the issue. I would recommend to get in touch with JTAC for futher debugging.

 

Note: Only 9 out of your 18 switches are under active support. Your EX4300-48T's and QFX5100-24Q (spines) haven't had their support renewed. My guess is that JTAC won't handle the case before support is in place on all devices in the VCF.

 

 

---

@SaharH wrote:

No cos/qos implention .

 

In fact you do:

class-of-service {
    classifiers {
        ieee-802.1 802_1_P_CLASSIFIER {
            forwarding-class best-effort {
                loss-priority low code-points [ 000 001 010 011 100 101 ];
            }
            forwarding-class network-control {
                loss-priority low code-points [ 110 111 ];
            }
        }
    }
.
.
.
}

 

All of your drops are on the BE queue: 

Output errors:
Carrier transitions: 0, Errors: 0, Drops: 1124985593, MTU errors: 0, Resource errors: 0, Bucket drops: 0
Egress queues: 12 supported, 5 in use
Queue counters: Queued packets Transmitted packets Dropped packets
0                            0        241591445231      1124985593
3                            0                   0               0
4                            0                   0               0
7                            0              215177               0
8                            0               97541               0

 

And you're pushing 16Gb/s on a 20Gb link

Traffic statistics:
Input bytes : 1515850564345 77838320 bps
Output bytes : 358613423879547 16235213648 bps

 

Is it not likely that you're simply bursting over 20Gb from time to time? Or microbursting?