SRX Services Gateway
Reply
Contributor
kbrookov
Posts: 60
Registered: ‎09-03-2009
0

SRX-650 crashed to high CPU

Hard reboot got it back online, we were seeing this message via console:

 

"kern.maxfiles limit exceeded by uid 2001, please see tuning(7)."

 

We do have a case open with Juniper, just curious if anyone else has ran into this?  We think NSM is the source of the problem, but aren't sure.

 

Thanks,

Distinguished Expert
Raheel
Posts: 414
Registered: ‎06-18-2008
0

Re: SRX-650 crashed to high CPU

which image are you using? are you using FWauth anywhere?

 

thanks

raheel 

Follow me on Twitter @anwar_raheel

--
If this post was helpful, please mark this post as an "Accepted Solution".
Kudos are always appreciated!
Contributor
kbrookov
Posts: 60
Registered: ‎09-03-2009
0

Re: SRX-650 crashed to high CPU

10.0R2.10

 

We are not using FWauth.

 

Thanks,

Contributor
amjain
Posts: 26
Registered: ‎03-29-2010
0

Re: SRX-650 crashed to high CPU

Your box seems to have hit the maximum files open limit. What is the customer case id for this problem?

 

Contributor
kbrookov
Posts: 60
Registered: ‎09-03-2009
0

Re: SRX-650 crashed to high CPU

2010-0414-0848

Contributor
abhilash.rajappan
Posts: 31
Registered: ‎10-14-2010
0

Re: SRX-650 crashed to high CPU

Hi,

 

Could you please share how this issue was solved by JTAC. We faced the same issue yesterday on our srx650 cluster.

 

Thanks,

Abhilash

Distinguished Expert
rkim
Posts: 755
Registered: ‎11-06-2007
0

Re: SRX-650 crashed to high CPU

We have seen instances where NSM was causing high CPU and maxproc issues. There are fixes in latest NSM 2010.3 version. I would recommend upgrading NSM to latest version and schema. Be sure to also run at the very least the latest maintenance release for your JUNOS version (i.e. for 10.0 it would be 10.0R4). Current JTAC recommended JUNOS release for SRX is 10.2R3.

 

-Richard

Contributor
abhilash.rajappan
Posts: 31
Registered: ‎10-14-2010
0

Re: SRX-650 crashed to high CPU

Thanks Richard. This issue looks to be the one mentioned in http://kb.juniper.net/KB15068

 

The KB however doesn't point to the NSM version upgrade as solution.

 

Regards,

Abhilash

Contributor
kbrookov
Posts: 60
Registered: ‎09-03-2009
0

Re: SRX-650 crashed to high CPU

There were a lot of issues with this case :smileyvery-happy

 

The main one:

 

"1.  Changed the config of the VPNs to dynamic hostname to an email address format.  The
VPN config was originally created by an NSM template, and it created the dynamic
hostname with a format of hostname.global.  The code had problems when trying to resolve
DNS name with this format.  Changing the hostname to email format is a workaround for
this issue.
2.  Downgraded JUNOS code to 10.0R3.1.

The plan moving forward is to go through the same steps on the production site this
weekend.

10.1R3 is going to be out soon, and the when NSM schema update supports this, then an
upgrade to that version of code will be planned.

Putting case into monitoring for now.  Also got the okay to bring this to a P2, since we
seem to have a workable solution on the DR site."

 

We are presently setting on [10.1R3.7], and it's been pretty stable.

 

NSM version:

Version: 2010.2
Build: LGB13z1n63

 

Our SE is telling us to hold off on 2010.3.

 

I would definately recomend going to a .3 release of SRX code.  Juniper code (netscreen or SRX) seems to stabilize by the R3 release. 

Contributor
abhilash.rajappan
Posts: 31
Registered: ‎10-14-2010
0

Re: SRX-650 crashed to high CPU

Thanks a lot for this detailed reply. Its hard to conclude now the NSM caused this or a bug on the 10.0R3.10. If we create a case, JTAC would directly recommend to upgrade to the recommended version 10.2R3.10.

 

We are planning to upgarde our box to 10.2 in the coming weekend and will remove out the NSM.

 

Regards,

Abhilash 

Copyright© 1999-2013 Juniper Networks, Inc. All rights reserved.