SRX

Hi all,

I've been racking my brain, but I can't seem to figure this one out.  Hopefully someone here can help.

I have a SRX240 that seems to be hitting a bandwidth limit at 100Mbps.  I have 4 SFP Mini-PIM modules installed in addition to the 16 on-board 10/100/1000 ports.  The SRX currently uplinks to our border router via fiber off Mini-PIM 3 (ge-3/0/0).  There are 2 links to a downstream switch that feeds our residence halls (ge-0/0/0 and ge-0/0/1).  These 2 links have MSTP configured, and I weight the links to send traffic through a NetEqualizer with the other link as a backup in case the NE fails.  All of this works fine.

What's weird is that the link maxes out at 100Mbps every night during heavy usage.  We can see on our newtork monitoring tools the bandwidth graph flatlines at 100Mbps (actually around 93Mbps... overhead and all...).  My first thought was that the NetEqualizer was misbehaving -- so I took it out of the chain.  Same exact issue, so it's not the NetEqualizer.

I have verified, double, triple, and quadruple checked all the links involved.  Both ends of every link report they're running at 1000Mbps, full-duplex.  I have no QoS or policers configured on the SRX or on the upstream or downstream devices.

I'll attach a sample of the bandwidth monitoring graph, you can see that the speed never goes over 100M and it flatlines under heavy use periods.  I'll also attach my complete SRX config (minus security policies, since they're of no imortance here), and the output of "show interface <xxx> extensive" for the 2 physical ge interfaces and the vlan interface that routes the reshall networks.  I've watched them closely and the input/output stats never go over 100M.

Has anyone seen this before?  Anyone have a clue what might be causing this really odd behavior?

Thanks!

Attachment(s)

srx240-interfaces.txt   18 KB 1 version

srx240-config.txt   19 KB 1 version

These mini-PIMs support 1000BaseSX and report 1000Mb in the show commands, but the actual bandwidth is limited to 100Mbps.

http://www.juniper.net/us/en/local/pdf/datasheets/1000315-en.pdf

Well, that certainly explains it.  I'm glad I've spent hours racking my brain trying to figure this out.

I gotta say though.....  are you effin kidding me?!?!   Why in the world would a 1G SFP module be limited to 100M ?  Thanks a lot, Juniper.   *sigh*   Would you happen to know if there are any plans to release a 1G Mini-PIM that can, oh, actually support 1G?  I didn't buy these 4 modules to do 100M, I bought them to do 1G.   Looks like I'll be replacing that SRX240 with something that can actually do what we need it to do.

Thanks for the quick reply, Doug.

The old SRX-MP-1SFP had a 100Mbps backplane.  There's a new mPIM SRX-MP-1SFP-GE that has a 1Gbps backplane.

http://kb.juniper.net/InfoCenter/index?page=content&id=KB19913

It was just released and is supported as of 10.4.  See the release notes for detail.

http://www.juniper.net/techpubs/en_US/junos10.4/information-products/topic-collections/release-notes/10.4/topic-51161.html#jd0e9025

Contact your account team regarding this and I am sure you guys can work something out.

Thanks, I did some digging around and discovered that the mention in the 10.4 release notes is for a new version of the module.

Old version:  SRX-MP-1SFP

New version:  SRX-MP-1SFP-GE

I'm still a bit beside myself that the original version of this module was limited to 100M.  It wasn't disclosed in the data sheets for the SRX240, nor did our sales team warn us of any limitation on those modules.

This KB summarizes the changes:  http://kb.juniper.net/InfoCenter/index?page=content&id=KB19913

I'll be bringing this up with our sales team.  I feel they should replace the original modules with the new versions, since that's what we thought we were buying in the first place.

Actually the datasheet does mention that actual bandwidth is limited to 100Mbps.

http://www.juniper.net/us/en/local/pdf/datasheets/1000315-en.pdf

It is stated on bottom of page 2. Keep in mind that the mPIM slot was meant to be used with WAN type ports. This is certainly the case on SRX210 and SSG20 platform which shares these mPIMs. The purpose of the 1-SFP mPIM was to be able to connect to providers which provide fiber connections. It was not meant to be able to handle full 1Gbps traffic (keep in mind that SRX210 and 240 are rated for 750M and 1.5G bps for the entire platform). Nevertheless the newer mPIM that the KB article describes will be capable of 1Gbps at least on 240. But if 4 true Gigabit links are required, perhaps SRX240 is not the right platform as the SRX650 can certainly handle more than 4 Gbps of traffic.

-Richard

Yes, the datasheet for the mPIM itself mentions the 100M limit, however, I never saw that.  Juniper spreads information across a bunch of different datasheets and apparently that one got by me.  However, I shouldn't have to go digging that deep every time I want to buy something to make sure I'm not being mislead.

We based our ordering off the SRX240 data sheet, http://www.juniper.net/us/en/local/pdf/datasheets/1000281-en.pdf

Also, on the PIM compatibility matrix for the SRX, http://www.juniper.net/us/en/local/pdf/datasheets/1000291-en.pdf

Both of those sheets list the SFP module, and neither of them mention a 100M throughput limit. Also, the compatibility matrix on page 3 lists a maximum of 20 10/100/1000 ports for the SRX240 (with no mention of "oh, by the way, 4 of those 10/100/1000 ports aren't really 10/100/1000 capable").  When we ordered this configuration our sales team said nothing to us regarding a throughput limit on those modules.  We were under the assumption, as I'm sure many others were as well, that a SFP module that lists 10/100/1000 would actually perform as such.  We have a lot of network equipment from a lot of vendors, and I've never seen something like this.  Even our old, crusty, Netscreen 500 didn't do that.  Granted, it was limited to 700M of firewall throughput, but at least I could get 500M through it across two fiber GBIC interfaces.  

The module supports 1G signaling, so both ends of the link show 1G.  I don't think it was an unreasonable assumption that a current-generation product which advertises SFP modules and GigE SX and LX optics would actually operate at a 1G throughput, rather than hiding a major limitation under the rocks.  

I suppose the argument can be made that I should have read the datasheet for the module, but I'm going to counter that and say that such important information shouldn't be buried away in one place, hidden quietly at the bottom of a datasheet that didn't seem critical to read.  We read the SRX datasheet and the module compatibility matrix datasheet, based on that it was pretty clear what we needed to order.

I think it was kind of a dirty trick on Juniper's part to bury that information.  I can guarantee you we're not the only ones who have been or will be burned by this.

The SRX240 is a perfectly adequate platform, I don't need an SRX650 for this task.  I don't need 4G of throughput.  I understand where limits are.  What I do need, however, is the ability to connect over GigE fiber to my routers and not have a 100M choke point on those connections. Moving 200-300M across all 4 fiber links simultaneously is within the 1.5G throughput for the SRX240.

We've run into this as well and had to replace a bunch of these modules. I understand why such modules existed from the old SSG product line. But the way it was hidden in a little line of text in the module datasheet, seems like a deliberate attempt to hide the fact that no fast SFP interfaces were available yet. Talk to your sales rep to get the modules replaced, the least they can do is make a little extra effort 🙂

The old modules also do not support jumbo frames, which was a bit of an issue. But they do support an MTU of around 1600, which was enough for our needs.

As we have received the new modules, first thing we did is hook them up to a traffic generator to make sure there were no more hidden limits. The good news is that they can handle 1gig troughput, but beware that thats 1gig total for ingress and egress combined. I don't think thats really an issue as the srx240 is only rated at around 750mbit throughput with normal traffic.