SRX

HI there,

 

I have the process flowd_octeon_hm that use lots CPU. My SSH is quite slow when I commit or excute a command. The J-WEB also.

 

 

last pid: 45306;  load averages:  3.04,  3.22,  3.24                                                                                                                         up 8+04:21:54  13:19:11
87 processes:  4 running, 82 sleeping, 1 zombie
CPU states: 76.7% user,  0.0% nice,  0.9% system,  0.0% interrupt, 22.4% idle
Mem: 226M Active, 116M Inact, 588M Wired, 39M Cache, 112M Buf, 1500K Free
Swap:
1049 root        6   8    0   457M 41788K nanslp 0 607.5H 274.37% flowd_octeon_hm

 

I'm using 10.0R4.7. Does anyone have this issue?

 

Hi,

 

some month ago, I asked JTAC about this and got the following answers:

 

 

Problem Description: As I can understand you have recognized high cpu utilization after 
viewing the command:
>show system process extensive
The output displayed the load averages as 3.03 and the flowd_octeon_hm having a CPU 
percentage of 281.10%.

Solution:
I had called you but unfortunately was not able to reach you.
The output displayed is not a cause of worry.As:
1)The load averages is related to the cpu cycle that is running and 3.03 is normal with 
respect to the same.
2)In SRX devices, the process are handled by various threads as 
flowd_octeon,utmd,chassisid.
The flowd_octeon thread usually runs at high percentage in order to make sure that all 
the packets that are received are processed at a faster rate.
Hence the device is working normally and the CPU is not running at high capacity.

In order to check the cpu utilization you can run the following command in operational 
mode:
>show chassis routing-engine
From the outut of the command you would be able to view the CPU utilization,if the idle 
time for CPU utilization is more, then there is nothing to worry.
Please let me know if i was able to answer to your queries.
At any point you need assistance please let me know.

 

 

HI,

 

As far as i remember, i didn't have that process before. Is there any chance that you could stop it?

I think it may have more to do with the amount of free memory, can you reboot and see how much is free after a reboot and if it speeds up? I would also upgrade to (at least) 10.2 R2, it has been a lot more stable for me.

Hi,

 

---

@PowerRanger wrote:

As far as i remember, i didn't have that process before. Is there any chance that you could stop it?

---

 

The flowd is your stateful firewall process, without it you will have a pure router.

 

But you may disable the security module bycommiting the following:

 

set security forwarding-options family mpls mode packet-based

 

Naa, you don't want that.

 

 

hf,

 

Klauzi

 

Hi,

 

Reboot master, when connected to the backup there is still flowd_octeon_hm

 

Reboot backup, when connected to the master there is still flowd_octeon_hm

 

Entrada: I do not have "mode packet-based" on the command

 

If it's in idle, why do we have +70% of CPU ??

And i do not use MPLS

Flowd is responsible for packet handling on SRX. You cannot disable it as this will kill all traffic forwarding through the SRX.

 

What you are seeing is misleading. The 'show system processes' command will only show uKern portion of flowd and not real-time domain which is the portion that is responsible for actual traffic handling. System always reports high due to the nature of the CPU used. The SRX240 uses a multicore processor (run shell command 'top -H' to see breakout of the individual cores) that runs more or less in a continuous loop without interrupts. This has the affect of flowd seemingly showing high always, but in fact does not mean that system is busy. This was done to maximize performance for transit traffic through the SRX.

 

A more accurate way to see actual CPU usage is below.

 

For RE (control plane) CPU usage:

  show chassis routing-engine

 

For PFE (data plane) CPU usage:

  show chassis forwarding

  show security monitoring fpc 0

 

 

Hope this helps.

-Richard

Try using top -H. This shows the processes running on each individual thread.

Hi , 

 

I have a problem with two SRX 650 in HA cluster.Node 0(primary) have constantly high CPU utilization 

 

PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU COMMAND
1233 root 15 76 0 979M 52140K select 0 234.9H 1020.85% flowd_octeon_hm
983 root 1 139 0 13476K 6040K RUN 0 891:11 71.39% eventd
1237 root 1 79 0 13772K 7268K select 0 134:57 6.74% rtlogd

 

Do you have idea what is the problem? What is eventd doing ?

 

Regards,

 

Srdjan

Hi Srdjan

 

Your eventd is consuming lot of cpu.

Have you configured syslog in event mode?

You could try moving it to stream mode.

 

Regards,

Raveen

Hi Raveen,

 

Thanks for answer.

 

I do not have configured any mode in security log hierarchy,it is empty , but i think that event mod is default behavior for SRX 650 platform.

 

I cannot see in any log messages that a problem exist.

 

Regards,

 

Srdjan

What is the Junos version that you are using?

Have you configured any sampling or traceoptions?

If so, can you disable them and check if it helps!

 

Regards,

Raveen

Hi,

 

JUNOS Software Release [11.4R3.7]
JUNOS AIS Script Suite [3.2R1.2]

 

This is configuration

srdjan@BOP_SRX650-DCFW# show system syslog
archive size 100k files 3;
user * {
any emergency;
}
file messages {
any critical;
authorization info;
}
file FLOW {
any any;
match RT_FLOW_SESSION;
}
file IDP {
any any;
match RT_IDP;
}
file SCREEN {
any any;
match RT_SCREEN;
}
file INT {
any any;
match LINK;
}
file cli {
authorization any;
interactive-commands any;
archive size 125k files 8 no-world-readable;
}
time-format year millisecond;

 

Regards,

 

Srdjan

Hi, working on a TV company in Brasilia, Brazil, where it is used the Juniper SRX240 with Junos. We are in the firmware update process, and we did in the first MASTER (node-0) to test its functionality and performance. So we turned off the SLAVE (node-1) to no conflicts occur. But we note that the processing equipment (CPU) increased considerably. Processes already disabling Kaspersky yet permece CPU consumption too high, impacting negatively on our internet access. We observed that the process "flowd_octeon_hm" is consuming a lot of processing appliance. I wonder how I can reduce this consumption? Do not think it's safe to update the two devices due to this problem. Have googled a lot and can not find a solution.
Thank in advance.

 

* Firmware Junos 11.4R9.4

What is the next process you see after flowd ?
Branch SRX has multi core/multi thread architecture, so it is normal to see 1000+% allocated to flowd (out of total 32 threads 31 are allocated to flowd or PFE)

We should look at the next highest process .

Can you send following logs:
Show chassis routing-engine
Show system process extensive
Show security monitoring performance spu

Dear,

 

I work in the same company as follows Alam data as requested

 

show chassis routing-engine
Routing Engine status:
Temperature 33 degrees C / 91 degrees F
CPU temperature 33 degrees C / 91 degrees F
Total memory 1024 MB Max 799 MB used ( 78 percent)
Control plane memory 560 MB Max 448 MB used ( 80 percent)
Data plane memory 464 MB Max 348 MB used ( 75 percent)
CPU utilization:
User 39 percent
Background 0 percent
Kernel 33 percent
Interrupt 1 percent
Idle 28 percent
Model RE-SRX240H
Serial ID AADX9774
Start time 2014-06-02 18:27:23 BRT
Uptime 16 hours, 55 minutes, 45 seconds
Last reboot reason 0x20:power-button soft power off
Load averages: 1 minute 5 minute 15 minute
1.67 1.70 1.55

show system process extensive
node0:
--------------------------------------------------------------------------
last pid: 40317; load averages: 2.45, 1.88, 1.62 up 0+16:56:39 11:23:33
142 processes: 20 running, 110 sleeping, 1 zombie, 11 waiting

Mem: 161M Active, 118M Inact, 580M Wired, 53M Cache, 112M Buf, 58M Free
Swap:

PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU COMMAND
1360 root 7 76 0 499M 49352K select 0 52.6H 286.43% flowd_octeon_hm
22 root 1 171 52 0K 16K RUN 0 763:25 13.53% idle: cpu0
1097 root 1 -4 0 12712K 4732K getblk 0 4:52 7.47% eventd
1415 root 1 80 0 10280K 3584K select 0 3:40 6.74% nstraced
40315 root 1 -8 0 2636K 992K biowr 0 0:00 6.01% gzip
40293 nobody 1 89 0 14940K 8168K select 0 0:01 3.71% php
1425 root 1 79 0 21756K 14508K select 0 2:43 2.98% snmpd
40313 root 1 77 0 3276K 1552K RUN 0 0:00 2.25% jps
1424 root 1 77 0 27792K 10972K select 0 5:20 1.46% mib2d
24 root 1 -20 -139 0K 16K RUN 0 7:09 0.00% swi7: clock
5 root 1 -16 0 0K 16K rtfifo 0 3:27 0.00% rtfifo_kern_recv
1366 root 1 76 0 19728K 11296K select 0 2:59 0.00% utmd
19 root 1 171 52 0K 16K RUN 3 2:55 0.00% idle: cpu3
20 root 1 171 52 0K 16K RUN 2 2:55 0.00% idle: cpu2
21 root 1 171 52 0K 16K RUN 1 2:55 0.00% idle: cpu1
1367 root 1 76 0 12300K 6336K select 0 2:44 0.00% license-check
23 root 1 -40 -159 0K 16K WAIT 0 2:22 0.00% swi2: net
1395 root 1 76 0 20248K 8932K select 0 1:26 0.00% l2ald
1361 root 1 76 0 15584K 3696K select 0 1:01 0.00% shm-rtsdbd
1400 root 1 76 0 12960K 3252K select 0 1:00 0.00% fud
1350 root 1 76 0 40100K 16808K select 0 0:55 0.00% chassisd
9 root 1 171 52 0K 16K pgzero 0 0:53 0.00% pagezero
1437 nobody 6 76 0 13464K 6892K ucondt 0 0:52 0.00% httpd
1365 root 1 76 0 13324K 6368K select 0 0:43 0.00% rtlogd
1423 root 2 76 0 35568K 9172K select 0 0:40 0.00% pfed
1351 root 1 76 0 14652K 8324K select 0 0:39 0.00% alarmd
1362 root 1 76 0 13536K 6912K select 0 0:36 0.00% jsrpd
1353 root 1 84 0 37424K 21544K select 0 0:32 0.00% mgd
49 root 1 -16 0 0K 16K psleep 0 0:28 0.00% vmkmemdaemon
1347 root 1 76 0 3288K 1376K select 0 0:28 0.00% bslockd
1413 root 1 76 0 13608K 6540K select 0 0:27 0.00% httpd-gk
56 root 1 -8 0 0K 16K mdwait 0 0:20 0.00% md0
1401 root 1 76 0 9376K 3644K select 0 0:20 0.00% ppmd

show security monitoring performance spu

show security monitoring performance spu
node0:
--------------------------------------------------------------------------
fpc 0 pic 0
Last 60 seconds:
0: 94 1: 88 2: 92 3: 88 4: 92 5: 89
6: 89 7: 93 8: 98 9: 91 10: 86 11: 91
12: 91 13: 86 14: 85 15: 87 16: 89 17: 88
18: 94 19: 91 20: 87 21: 94 22: 93 23: 92
24: 96 25: 93 26: 93 27: 91 28: 92 29: 91
30: 90 31: 92 32: 87 33: 90 34: 87 35: 95
36: 84 37: 93 38: 89 39: 90 40: 89 41: 89
42: 90 43: 86 44: 95 45: 96 46: 96 47: 92
48: 95 49: 91 50: 92 51: 91 52: 91 53: 90
54: 92 55: 99 56: 94 57: 75 58: 79 59: 93

Looking through the logs I see 2 issues.
1 - control plane highe cpu (not very high)
1097 root 1 -4 0 12712K 4732K getblk 0 4:52 7.47% eventd
1415 root 1 80 0 10280K 3584K select 0 3:40 6.74% nstraced
40315 root 1 -8 0 2636K 992K biowr 0 0:00 6.01% gzip

This indicates that you have some traceoptions enabled and too much of logging.

2 - dataplane high CPU
I think for this you should raise case with JTAC.
There are multiple reasons for it. It is also possible that the traffic on the device is reaching its limit.