SRX Services Gateway
Reply
Distinguished Expert
keithr
Posts: 979
Registered: ‎09-10-2009
0

Beware! Junos 10.4R2.7 technical issue and somewhat long rant combined in this post! :)

This is a combination Public Service Announcement (PSA) and a cry for help.

 

I think I've just about reached my wits' end with Juniper and the SRX products.  Juniper, are you reading this?

 

Following hot on the heels of the "1GB mPIM card that's actually a 100M mPIM card" debacle for our SRX 240s, we learn that Juniper has released a new version of the 1GE mPIM card for SRX 2xx platforms. This one is actually <*gasp*> a 1GE card.

 

http://kb.juniper.net/InfoCenter/index?page=content&id=KB19913

 

We went back and forth a couple times with our sales team about the original card.  We felt we were sold something under false pretenses.  Check the datasheet, check the PIM Compatibility Matrix... the matrix says the SRX240 has a maximum of 20 10/100/1000 ports.   That's 16 fixed plus 4 on mPIM cards. The SRX-MP-1SFP PIM was the only available card for the SRX240, and it accepted 1GE optics (SX or LX).  Nowhere is it mentioned that this card could only actually pass 100M of traffic through it, even though it linked up at 1G ethernet signaling.  Talk about misleading.  Yes, there is a blurb at the bottom of the datasheet for the particular mPIM that says "100/1000 Mbps link speed supported; actual bandwidth limited to 100 Mbps."  But, please, raise your hand if you bought these modules and actually saw that note and knew about the limitation.  We bought the modules that were available, the 1GE optics for them, and they linked up at 1GE.  I don't think I was out-of-line to expect that they would actually be 1GE interfaces.

 

The aforementioned KB article makes it pretty clear that the SRX-MP-1SFP-GE card is what the SRX-MP-1SFP card was *supposed* to have been in the first place.  "The SRX-MP-1SFP-GE is a newer model meant to replace the original SRX-MP-1SFP mPIMs."  Seems clear to me that Juniper realized they goofed pretty badly on this one.

 

Juniper didn't want to replace the modules for us.  Shame on you, Juniper. The best they said they would do is "deeply discount" them to our VAR.  Our VAR told us that and I said that I refused to pay twice for something.  We were sold 1G modules.  We didn't get them.  Juniper didn't see the failure of this transaction.

 

I find it hard to believe that somebody at Juniper didn't have the power of the pen, or the desire to make a large, long-time customer happy and just write off what boils down to about $100 each (estimating what these cards actually cost Juniper) worth of cards in the name of making things right with a customer.

 

Where am I going with this?  Our VAR shipped us one at his expense, because he agreed with us that it was completely unfair of Juniper to expect us to pay even a little more money to them for these pieces. Thank you VAR, -10 points for Juniper as a company, again.  (I find it even more unsavory that Juniper let the burden of this fall on their VAR, when the VAR wasn't responsible for the misleading transaction in the first place.)

 

Support for the SRX-MP-1SFP-GE mPIM requires Junos 10.4.  We were currently running 10.2R3.10 on the SRX240 that I was going to put the new card into to test. I loaded 10.4R2.7 and reset the SRX240 overnight.  The next day, people started complaining that they were dropping offline and couldn't get back on.  After some digging, I found that the problem was that nobody was getting DHCP addresses from our server. DHCP relay had stopped working.  I did some digging, tried to do some debugging, but thanks to the bug noted in KB20228, I couldn't run proper flow traceoptions because packet filters don't work on 10.4R2. At this point I just had to say "Really, Juniper? REALLY?  COME ON!!!!!"

 

The best I could find was that under 10.4R2, with a fairly simple DHCP relay setup:

 

 

forwarding-options {
    helpers {
        bootp {
            relay-agent-option;
            description "Main DHCP Servers";
            server 1.1.1.1;
            server 1.1.1.2;
            interface {
                vlan.100;
                vlan.600;
            }
        }
    }
}

 

.... clients in both VLANs 100 and 600 would send the DHCPDISCOVER or DHCPREQUEST messages (to renew a lease), the server would get the packets (packet capture at the DHCP server), so relay was working from the SRX to the server, but when the server replied, the SRX then dropped the reply packets.  Note that no configuration changes (security policies, etc.) were made, the only difference was the version 10.4 rather than 10.2.

 

I rolled the system back to 10.2R3.10, and DHCP began working again.  For confirmation, I rolled to to 10.4 again to test, and DHCP stopped. Back to 10.2R3, DHCP works.

 

This is not a "bash on Juniper" post.  This is a "will *somebody* at Juniper please wake up and realize that you cannot keep putting out half-baked products (hardware and software) and expect to keep a happy customer base!?" post.

 

No vendor is perfect. Products have bugs, things go wrong.  It just seems that in the past few years, Juniper, and especially the SRX products, have had a very HIGH volume of bugs and problems, and they continue to this day. When things go wrong Juniper, how you make things right with the customer is how you keep them a customer.  Juniper is falling flat on their faces on this front, too.

 

-kr


---
If this solves your problem, please mark this post as "Accepted Solution."
Kudos are always appreciated.
Super Contributor
billp
Posts: 126
Registered: ‎05-01-2008
0

Re: Beware! Junos 10.4R2.7 technical issue and somewhat long rant combined in this post! :)

Your post said:

This is a "will *somebody* at Juniper please wake up and realize that you cannot keep putting out half-baked products (hardware and software) and expect to keep a happy customer base!?" post.

 

While I can't release a full, detailed response, I respect your position and wanted to make sure you got a reply. I've been involved in a lot of quality discussions around Juniper's security software in the past few weeks; there are programs that started a while back that are just starting to be felt (with the software release cycle, it takes a while for effects to filter out to released code), and new programs are spinning up now to drive even further improvements. Junos started its life as a service-provider-oriented OS - rock solid code every time - and we're working to get back to a 'rock-solid' reputation. Your message was heard, and is being taken very seriously.

 

Thank you for caring enough to be involved in this forum, and please accept my apologies for the frustration that you had to endure. I know that a vague answer and apology is not much, but I do hope it helps at least a bit.

 

-Bill

Trusted Contributor
mawr
Posts: 236
Registered: ‎06-11-2010
0

Re: Beware! Junos 10.4R2.7 technical issue and somewhat long rant combined in this post! :)

Half-baked software is something I trust will be improved over time but I'm a little more concerned about the half-baked hardware comment.  Should we know something?

 

Thanks,

 

mawr

Qin
Contributor
Qin
Posts: 12
Registered: ‎01-12-2011
0

Re: Beware! Junos 10.4R2.7 technical issue and somewhat long rant combined in this post! :)

Good thing that I just read this.   I was going to upgrade to 10.4R2.7 today, and currently running 10.4R1 and seems ok, with some issues with VPN dropping.

Visitor
pradc
Posts: 2
Registered: ‎09-16-2010
0

Re: Beware! Junos 10.4R2.7 technical issue and somewhat long rant combined in this post! :)

 

Just happened to see this.

Sorry and apologies for the problem you guys are facing

we have taken seriously the problems you faced  on the cards and we are working internally to improve in handling it in future.

About the  DHCP-relay issue. We have fixed all these DHCP relay issues in coming 10.4R3 release  along with many Enhancements. pls wait till 10.4R3 is out and let me know your opinion The issue faced by you are known issues and  found internally  by testing teams and we were waiting for the release cycle to get this committed. Since we are getting 10.4R3 in the near future, this seems to be best vehicle. Please wait till then.

 

 

 

Contributor
Kurlon
Posts: 28
Registered: ‎02-19-2010
0

Re: Beware! Junos 10.4R2.7 technical issue and somewhat long rant combined in this post! :)

How is the buffering on the earlier mPIM?  I'm about to swing some hefty traffic flows over to a pair of 240s sporting these modules and would hope that they handle bursty traffic or sustained flows at and beyond 100mbs gracefully?

Super Contributor
motd
Posts: 221
Registered: ‎12-16-2008
0

Re: Beware! Junos 10.4R2.7 technical issue and somewhat long rant combined in this post! :)

The 10.4R2 release was quite a mess. Honestly, one of the worst SRX releases I've seen in quite a while. I definitely wouldn't recommend using it (although I'm forced to - for the new GE mPIMs).

Its as if they simply published some code just to make the deadline, without even testing anything. I know all the stories about internal improvements etc, but this time they all failed. Time for a new release (R or S) so that we can all forget about R2 :smileyhappy:

Trusted Expert
Automate
Posts: 784
Registered: ‎11-01-2007
0

Re: Beware! Junos 10.4R2.7 technical issue and somewhat long rant combined in this post! :)

@keithr

 

I wasn't involved with the conversations that occurred between your reseller, Juniper, yourself, etc. But the outcome you described certainly seems counter to our ideals for customer service and is certainly not the outcome I personally would endorse.

 

I have escalated this issue internally and will report back on this thread for any actions (some of your asks are already addressed).

 

As for 10.4 quality (@keithr/motd)...having stuck my neck out somewhat on prior posts about the progress you should have seen in 10.4 I'm distressed to hear this.  We have objective measures (incoming field-found defects, etc) which are very positive for this release, but quality is in perception and if your particular objectives are foiled by a bug, obviously it won't look good no matter how "high" the objective quality is.

 

Other recent posts are fairly positive on 10.4 (I didn't write them - honest! :smileyhappy: ) so a lot will depend on features in play.

 

So stay tuned to this thread for more on this and related issues.

 

-Keith

 

Distinguished Expert
keithr
Posts: 979
Registered: ‎09-10-2009
0

Re: Beware! Junos 10.4R2.7 technical issue and somewhat long rant combined in this post! :)

 


mawr wrote:

Half-baked software is something I trust will be improved over time but I'm a little more concerned about the half-baked hardware comment.  Should we know something?


 

Mawr,

 

The first thing that comes to mind, naturally, is the mPIM card that my post mentioned.  Seems like they rushed a card to market without fully implenting what the card needed to do.  I don't want to get into a big rehash of what I already posted, but it took 1G optics, was listed in the documentation as a 10/100/1000 interface, but had major limitations such as only being able to support 100M of throughput on a 1G interface, and other limitations as mentioned in the KB (no jumbo frames, couldn't be used as a cluster reth interface, etc.)  The problems were obivously not based on a limitation of the chassis, since the new card corrects all these problems.  1G (or near...) throughput, jumbo frames, cluster support, etc.  It connects to the same backplane connector, etc., so it's just that Juniper put the original card out with all of those limitations that just didn't need to be there.

 

I've also had problems with ISG2000 management modules failing, ASIC cards needing to be replaced, IDP security blades failing, etc.

-kr


---
If this solves your problem, please mark this post as "Accepted Solution."
Kudos are always appreciated.
Distinguished Expert
keithr
Posts: 979
Registered: ‎09-10-2009
0

Re: Beware! Junos 10.4R2.7 technical issue and somewhat long rant combined in this post! :)

If these issues were found in testing, why did the software get released with known problems like that?

 

As motd mentioned, we hear lots of things about a commitment and rededication to quality and rock-solid code, and 10.4 was supposed to be the showcase of these new policies for quality control.

 

Did the release notes for 10.R2 mention "DHCP Relay doesn't work in this release", or any of the other bugs or problems that were found in testing?  If those things were mentioned in the release notes, then I suppose it was my fault for missing them.

 

A commitment to quality and rededication to rock-solid code means that a company shouldn't be rushing a software release out the door with known major deficiencies or bugs.  If 10.4R3 is coming soon, and these problems were found and will be correctected in 10.4R3, why was R2 even released if it had such problems?  I, for one, would rather have seen a solid, reliable release of software come out a month or two later than releasing something into the wild and then letting the customers feel the pain of problems that were known to be there.

-kr


---
If this solves your problem, please mark this post as "Accepted Solution."
Kudos are always appreciated.
Copyright© 1999-2013 Juniper Networks, Inc. All rights reserved.