SRX Services Gateway
Highlighted
SRX Services Gateway

CoS on SRX240 dropping VoIP / EF packets despite moderate load

‎11-02-2015 04:58 AM

We have a SRX240H2 In an office environment with about 50 users behind it. WAN is 1GE WDM connecting to a data center. On the LAN side of the SRX we have a couple of different security zones, amongst them "guest", "office" and "tech" for a higher access level to the production network in the data center. Traffic through the SRX is usually low (well below 100MBit/s) but somethimes around double that due to Time Machine backups from users in the "tech" zone backing up against a NAS in the "office" zone.

 

Seeing packet loss on voice traffic through the SRX during these times, a couple of days ago we setup CoS in an effort to address this. I think we've managed to do it good enough (queus show traffic at expected rates) but we still see PL. So a couple of questions regarding how to best move forward and in trying to understand how to think of CoS in this scenario:

 

  1. The SRX240H2 has a stated max throughput of about 700mbit/s in a scenario close to mine, we're not close to that but still see PL in times of high use, with and without CoS. How do I pinpoint what is causing this? CPU (on RE and SPC) is never above 20%.
  2. The scheduler-map revolves around percentages of available bendwidth on the egress interface. In this case, we are unlikely to be able to push a full 1GE link through the SRX, ie. some other factor(s?) than the CoS configuration are likely to dictate what gets forwarded through the device and at what rate. This question should apply to all SRX branch models below the 550 since they are rarely able to make full use of the interface bandwidth, yet I havn't found any explanations on what to expect.
5 REPLIES 5
Highlighted
SRX Services Gateway

Re: CoS on SRX240 dropping VoIP / EF packets despite moderate load

‎11-02-2015 07:45 AM

Hi,

 

Not sure how helpful this is but we have had some issues with quality for real time applications with a cluster of SRX240. We also did encryption, so the performance is far from 1 Gbps. We used Ixia to measure QoS in different classes.

 

We found that it is important to give your real time / ef class strict priority in the schedulers, but also to assign it with a larger part of the buffer than we had originally configured.

 

Also, note that the throughput stated in the data sheets often are "half duplex", so if you have symmetric trafiic in your high load scenario, the capacity in one direction is lower.

 

Can you post parts of your CoS configuration?

 

regards

Karl

Highlighted
SRX Services Gateway

Re: CoS on SRX240 dropping VoIP / EF packets despite moderate load

‎11-02-2015 11:40 AM

Hi Karl,

 

Appreciate your input. I'm attaching the relevant parts of the CoS below. In addition to this i have a couple of firewall filters setup for marking traffic not correctly marked (BA) by the endpoints. As stated, I'm pretty confident we mark the correct traffic judging by what amounts of traffic is expected in the different classes and what really ends up there.. As you can see, we don't set the EF class to strict. Maybe I'll try that out.

A thing that struck me when looking at the config now is that both classification and scheduling takes place on vlan interfaces and I'm not sure how that works. When bandwidth is set in relative terms, is the real (physical) egress interface that the packet is send on used to calculate the available bandwidth and then the transmit rate is derived from that?

 

gustav@mmo2-fw01# show class-of-service
classifiers {
    dscp xyz-branch {
        import default;
        forwarding-class assured-forwarding {
            loss-priority low code-points 011010;
        }
    }
}
drop-profiles {
    be_low_plp {
        fill-level 50 drop-probability 5;
        fill-level 80 drop-probability 50;
    }
    be_high_plp {
        fill-level 50 drop-probability 50;
        fill-level 80 drop-probability 70;
    }
}
forwarding-classes {
    queue 0 best-effort;
    queue 1 expedited-forwarding;
    queue 2 assured-forwarding;
    queue 3 network-control;
}
interfaces {
    vlan {
        unit 2 {
            scheduler-map xyz_branch_scheduler;
            classifiers {
                dscp xyz-branch;
            }
        }
        unit 3 {
            scheduler-map xyz_branch_scheduler;
            classifiers {
                dscp xyz-branch;
            }
        }
        unit 4 {
            scheduler-map xyz_branch_scheduler;
            classifiers {
                dscp xyz-branch;
            }
        }
    }
}
scheduler-maps {
    xyz_branch_scheduler {
        forwarding-class assured-forwarding scheduler af_sched;
        forwarding-class best-effort scheduler be_sched;
        forwarding-class expedited-forwarding scheduler ef_sched;
        forwarding-class network-control scheduler nc_sched;
    }
}
schedulers {
    be_sched {
        transmit-rate percent 50;
        priority low;
        drop-profile-map loss-priority low protocol any drop-profile be_low_plp;
        drop-profile-map loss-priority high protocol any drop-profile be_high_plp;
    }
    ef_sched {
        transmit-rate {
            percent 10;
            exact;
        }
        priority high;
    }
    nc_sched {
        transmit-rate percent 5;
        shaping-rate percent 20;
        priority high;
    }
    af_sched {
        transmit-rate {
            percent 5;
            exact;
        }
        priority medium-high;
    }
}
Highlighted
SRX Services Gateway

Re: CoS on SRX240 dropping VoIP / EF packets despite moderate load

‎11-03-2015 01:15 AM

Hi,

 

I have not used CoS on vlans, but i guess that should work. Is the traffic load higher on upstream or downstream when you see the issues? Can you confirm the packet loss occurs in the device as dropped packets in you egress queues?

 

I see that you import defaults in your classifier. Do you know if that will override the classification you do on yout firewall filters?

 

I would try using a strict priority for ef and try configure the buffers allocations. Also, maybe consider setting transmit-rate (and buffer size?) for be to "remainder".

 

regards

Karl

Highlighted
SRX Services Gateway

Re: CoS on SRX240 dropping VoIP / EF packets despite moderate load

‎11-04-2015 08:12 AM

Hi Karl,

 

Thanks for your input, much appreciated! From your comment I realized some of my BA classifiers were overridden by MF classifiers (explained here http://www.juniper.net/documentation/en_US/junos15.1/topics/concept/cos-ex-series-classifiers-unders... ) so I cleaned them up a bit by using either BA or MF classifiers on a single interface, but not both at the same time. That, in conjunction with setting transmit-rate at "200m exact" for the be-class seem to have done things markedly better. I now see a clear difference in PL/jitter on packets sent from ef queue compared to packets served in the be queue. Currently schedulers is:

gustav@mmo2-fw01# show class-of-service schedulers
be_sched {
    transmit-rate {
        250m;
        exact;
    }
    buffer-size {
        percent 50;
        exact;
    }
    priority low;
    drop-profile-map loss-priority low protocol any drop-profile be_low_plp;
    drop-profile-map loss-priority high protocol any drop-profile be_high_plp;
}
ef_sched {
    transmit-rate 10m;
    buffer-size percent 20;
    priority strict-high;
}
nc_sched {
    transmit-rate 2m;
    priority high;
}
af_sched {
    transmit-rate {
        5m;
        exact;
    }
    priority medium-high;
}

as you can see I also added RED drop profiles:

gustav@mmo2-fw01# show class-of-service drop-profiles
be_low_plp {
    fill-level 50 drop-probability 5;
    fill-level 80 drop-probability 50;
}
be_high_plp {
    fill-level 50 drop-probability 50;
    fill-level 80 drop-probability 70;
}

When setting transmit-rate rather low (200m) and "exact", I clearly see that the RED profiles are hit and packets are dropped. You asked wheterher I can see the dropped EF-packets as drops in the queues; no, I can't and that bothers me and makes me think they are dropped somewhere else, somehow CPU-related but I'm not sure how to go about trying to figure that out.

 

I will try your setting tomorrow (when we have live traffic) assigning "remainder" to be-queue and see what gives. Any hints on the size of the buffers? I feel larger buffers on the EF is not necessarily good (since it's voice) it will mostly lead to jitter.

Highlighted
SRX Services Gateway

Re: CoS on SRX240 dropping VoIP / EF packets despite moderate load

‎11-05-2015 08:00 AM

I agree that the whole part about assigning buffer to the EF is not very logical. For real time you typically want low latency and jitter rather than low loss.

 

Our scenario was different as we had AES256 encryption, which drastically reduces throughput, and our testing was not very academic. Also we generated traffic, rather than working with actual traffic and experiences.

 

We started off with a low buffer allocation to EF. For the test we ran Ixia with a number of TCP throughput scripts in AF and BE forwarding classes. For EF we simulated a number of voip calls. We saw an obvious improvment in the measured and calculated MOS-values when we increased the buffers.

 

The whole problem with the CPU choking long before the interfaces causes some though problems. All our traffic loads were TCP. For UDP i guess you would have to implement ingress policers to "protect" the CPU.

 

regards

Karl

Feedback