SRX Services Gateway
Highlighted
SRX Services Gateway

CPU spike on SRX650, causing TCP sessions to pile up and drop.

‎02-17-2015 10:24 AM

Hi folks,

 

I have a very annoying problem here where randomly, the CPU/SPU on our SRX650 cluster (active node only) will spike up to 99% and TCP sessions will pile up and get dropped.  The session spike does appear tp happen a minute or so before the CPU spike  The issue will magically correct itself one to two minutes later and traffic will begin flowing normally again.  There is not a corresponding spike in bandwidth, either inbound or outbound.  I have been over the systems with support backwards and forwards and nothing has come up.  The only thing I can find that is close is this PR, but that was apparently resolved in 12.1X44-D25 and we are running 12.1X44-D40.2.  Our firewalls have a simple setup with three zones - trust, untrust, and dmz, with traffic hitting our load balancers in the DMZ and servers in the trust zone occasionally talking to servers on the internet via NAT on the SRX cluster.  Does anyone have any suggestions on what to look for as a cause of the TCP session spike?

1 REPLY 1
Highlighted
SRX Services Gateway

Re: CPU spike on SRX650, causing TCP sessions to pile up and drop.

‎02-18-2015 07:41 PM

I think you can first start looking at the traffic on the device when the issue occurrs.

Monitor input bytes/sec, PPS on all interfaces. You can also add SNMP monitoring for total number of sessions.

 

If you have total sessions increasing during the issue, You can use session analyzer to understand who is creating these sessons. (http://forums.juniper.net/t5/SRX-Services-Gateway/SRX-Session-Analyzer/td-p/113798)

 

If you are seeing increase in traffic, but no increase in sesions, then you can do a packet capture on respective interface and understand what traffic is causing this issue.

 

And you can reach out to JTAC if you need assistance doing this Smiley Very Happy

regards,
Avd
JNCIE-SEC #320

Please Mark My Solution Accepted if you think it helped!
Feedback