SRX

Hello! I have SRX 650 with minimal level of transit traffic:

myst@VL_SRX650_1> show security flow statistics 
    Current sessions: 3643
    Packets forwarded: 10019423129
    Packets dropped: 37583878
    Fragment packets: 757395

 with single physical interface:

myst@VL_SRX650_1> show interfaces ge-0/0/3 
Physical interface: ge-0/0/3, Enabled, Physical link is Up
  Interface index: 137, SNMP ifIndex: 511
  Link-level type: Ethernet, MTU: 1518, Link-mode: Full-duplex, Speed: 1000mbps,
  BPDU Error: None, MAC-REWRITE Error: None, Loopback: Disabled,
  Source filtering: Disabled, Flow control: Enabled, Auto-negotiation: Enabled,
  Remote fault: Online
  Device flags   : Present Running
  Interface flags: SNMP-Traps Internal: 0x0
  CoS queues     : 8 supported, 8 maximum usable queues
  Current address: 40:b4:f0:80:5f:03, Hardware address: 40:b4:f0:80:5f:03
  Last flapped   : 2013-05-13 21:58:17 VLAST (15w2d 18:21 ago)
  Input rate     : 1311872 bps (315 pps)
  Output rate    : 1292656 bps (342 pps)
  Active alarms  : None
  Active defects : None
  Interface transmit statistics: Disabled

With no IDP/IDP etc 

But i get extremely high CPU utilisation with this config:

SRX650_1> show chassis routing-engine no-forwarding 
Routing Engine status:
    Temperature                 35 degrees C / 95 degrees F
    CPU temperature             35 degrees C / 95 degrees F
    Total memory              2048 MB Max  1761 MB used ( 86 percent)
      Control plane memory    1104 MB Max   938 MB used ( 85 percent)
      Data plane memory        944 MB Max   831 MB used ( 88 percent)
    CPU utilization:
      User                      46 percent
      Background                 0 percent
      Kernel                    54 percent
      Interrupt                  0 percent
      Idle                       0 percent
    Model                          RE-SRXSME-SRE6
    Serial ID                      AAFC6067
    Start time                     2013-05-13 21:55:18 VLAST
    Uptime                         107 days, 18 hours, 18 minutes, 30 seconds
    Last reboot reason             0x1:power cycle/failure 
    Load averages:                 1 minute   5 minute  15 minute
                                       1.57       1.73       1.19

SRX650_1> show system processes extensive no-forwarding 
last pid: 43178;  load averages:  1.47,  1.69,  1.19  up 107+18:19:20    16:14:10
136 processes: 28 running, 95 sleeping, 2 zombie, 11 waiting

Mem: 216M Active, 459M Inact, 1132M Wired, 123M Cache, 112M Buf, 42M Free
Swap:


  PID USERNAME  THR PRI NICE   SIZE    RES STATE  C   TIME   WCPU COMMAND
 1170 root       15  76    0   979M 49068K select 0    ??? 1027.59% flowd_octeon_hm
43023 root        1 139    0 36068K  4080K RUN    0   5:46 61.62% mgd
 1141 root        1  82    0   418M   392M RUN    0  31.4H 12.01% rpd
   22 root        1 171   52     0K    16K RUN    0 2026.4  0.00% idle: cpu0
 1144 root        1  76    0 39440K 28256K select 0  60.4H  0.00% kmd
 1162 root        1  76    0 18312K 10996K select 0  36.4H  0.00% snmpd
 1161 root        1  76    0 17796K  8364K select 0  31.2H  0.00% mib2d
 1104 root        1  76    0 35808K 15460K select 0  28.7H  0.00% chassisd
 1112 root        1  76    0  9028K  3436K select 0  17.7H  0.00% ppmd
   24 root        1 -20 -139     0K    16K RUN    0 813:44  0.00% swi7: clock
   23 root        1 -40 -159     0K    16K WAIT   0 473:27  0.00% swi2: net
    9 root        1 171   52     0K    16K RUN    0 348:21  0.00% pagezero
 1127 root        1  76    0 10568K  3948K select 0 346:42  0.00% license-check
    5 root        1 -84    0     0K    16K rtfifo 0 263:14  0.00% rtfifo_kern_recv
 1142 root        1  76    0 17564K  6992K select 0 163:11  0.00% l2ald
 1154 root        1   4    0  8380K  3632K kqread 0 128:46  0.00% mcsnoopd
 1160 root        2  76    0 33992K  7436K select 0  86:33  0.00% pfed
 1151 root        1  76    0 21960K 13340K select 0  76:52  0.00% utmd
 1105 root        1  76    0 10808K  3400K select 0  71:22  0.00% alarmd
 1175 root        1  76    0     0K    16K select 0  69:29  0.00% peer proxy
  893 root        1  76    0  8244K  2968K select 0  67:48  0.00% eventd
   47 root        1 -16    0     0K    16K psleep 0  64:22  0.00% vmkmemdaemon
 1101 root        1  76    0  3252K  1360K select 0  56:22  0.00% bslockd
 1125 root        1  76    0 11824K  4700K select 0  45:11  0.00% rtlogd
   26 root        1 -16    0     0K    16K -      0  30:19  0.00% yarrow
 1153 root        3  76    0 13588K  4408K select 0  21:29  0.00% wland
 1158 root        1  76    0  7620K  2960K select 0  19:58  0.00% irsd
27203 root        1  76    0 11632K  4896K select 0  19:53  0.00% rmopd
    2 root        1  -8    0     0K    16K -      0  19:12  0.00% g_event
    4 root        1  -8    0     0K    16K -      0  17:28  0.00% g_down
 1156 root        1  76    0 16272K  8916K select 0  16:22  0.00% nsd
 1149 root        1  76    0 11612K  4596K select 0  15:12  0.00% jsrpd
    3 root        1  -8    0     0K    16K -      0  15:11  0.00% g_up
   39 root        1  20    0     0K    16K syncer 0  14:19  0.00% syncer
 1150 root        1  76    0 12740K  4512K select 0  13:37  0.00% pkid
   41 root        1  20    0     0K    16K vnlrum 0  13:35  0.00% vnlru_mem
 1163 root        1  76    0 31996K  7692K select 0  11:39  0.00% dcd
 1140 root        1  76    0  3112K  2744K select 0  11:14  0.00% ntpd
 1164 root        1  76    0 28796K  5800K select 0  10:27  0.00% smid
 1100 root        1  76    0  2228K   868K select 0   9:50  0.00% watchdog
    6 root        1   8    0     0K    16K -      0   9:22  0.00% kqueue taskq
 1124 root        1  76    0 54792K  6284K select 0   7:28  0.00% idpd
 1116 root        1  76    0 11344K  4524K select 0   6:01  0.00% bfdd
   11 root        1 171   52     0K    16K RUN    b   4:40  0.00% idle: cpu11
   12 root        1 171   52     0K    16K RUN    a   4:40  0.00% idle: cpu10
   14 root        1 171   52     0K    16K RUN    8   4:37  0.00% idle: cpu8
 1143 root        1  76    0 19500K  6868K select 0   4:37  0.00% cosd
   15 root        1 171   52     0K    16K RUN    7   4:29  0.00% idle: cpu7
   13 root        1 171   52     0K    16K RUN    9   4:28  0.00% idle: cpu9
 1106 root        1  76    0 11396K  3420K select 0   4:22  0.00% craftd
   18 root        1 171   52     0K    16K RUN    4   3:44  0.00% idle: cpu4
   17 root        1 171   52     0K    16K RUN    5   3:43  0.00% idle: cpu5
   16 root        1 171   52     0K    16K RUN    6   3:41  0.00% idle: cpu6
27206 root        1  76    0 30360K  5996K select 0   3:07  0.00% smihelperd

 What is wrong and what can i do?

TNX!

+ Which Junos Version r u using ?

+ I see mgd and rpd process consuming CPU cycles , could you please check :

- Are there too many management connection ( Like ssh , telnet , WebGUI) opened to the box ?

- Are you running Dynamic routing protocol and which has huge database or is not very stable ?

--- JUNOS 11.2R4.3 built 2011-11-24 08:11:51 UTC

1)Now i have one telnet and one SecureWEB conncection

2) Yes, I have 2 bgp FV + 40 ospf proccess in 40 VRF. But in my netvork I have some same SRX with similar configuration/transit traffic etc and it works fine.

I.E:

KRAS_SRX650_2> show chassis routing-engine no-forwarding 
Routing Engine status:
    Temperature                 36 degrees C / 96 degrees F
    CPU temperature             36 degrees C / 96 degrees F
    Total memory              2048 MB Max  1700 MB used ( 83 percent)
      Control plane memory    1104 MB Max   872 MB used ( 79 percent)
      Data plane memory        944 MB Max   821 MB used ( 87 percent)
    CPU utilization:
      User                       9 percent
      Background                 0 percent
      Kernel                     6 percent
      Interrupt                  0 percent
      Idle                      85 percent
    Model                          RE-SRXSME-SRE6
    Serial ID                      AAFC6052
    Start time                     2013-08-26 10:00:34 KRAST
    Uptime                         3 days, 5 hours, 20 minutes, 4 seconds
    Last reboot reason             0x1:power cycle/failure 
    Load averages:                 1 minute   5 minute  15 minute
                                       0.42       0.21       0.18

KRAS_SRX650_2> show system processes extensive no-forwarding 
last pid: 20424;  load averages:  0.41,  0.23,  0.19  up 3+05:21:15    15:21:20
137 processes: 26 running, 100 sleeping, 11 waiting

Mem: 395M Active, 226M Inact, 1117M Wired, 145M Cache, 112M Buf, 89M Free
Swap:


  PID USERNAME  THR PRI NICE   SIZE    RES STATE  C   TIME   WCPU COMMAND
 1229 root       15  76    0   979M 53692K select 0 858.3H 1020.21% flowd_octeon_hm
   22 root        1 171   52     0K    16K RUN    0  60.9H 78.17% idle: cpu0
 1199 root        1  76    0 21072K  9856K select 0 116:44  0.00% kmd
 1195 root        1   4    0   411M   384M kqread 0  86:16  0.00% rpd
 1162 root        1  76    0 35872K 15592K select 0  51:49  0.00% chassisd
 1219 root        1  76    0 18260K 11016K select 0  39:04  0.00% snmpd
 1170 root        1  76    0  9020K  3360K select 0  32:09  0.00% ppmd
 1218 root        1  76    0 17560K  8072K select 0  31:19  0.00% mib2d
   24 root        1 -20 -139     0K    16K RUN    0  24:22  0.00% swi7: clock
   23 root        1 -40 -159     0K    16K WAIT   0  13:43  0.00% swi2: net
    9 root        1 171   52     0K    16K pgzero 0  11:50  0.00% pagezero
 1186 root        1  76    0 10568K  3876K select 0  10:41  0.00% license-check
    5 root        1 -84    0     0K    16K rtfifo 0   7:53  0.00% rtfifo_kern_recv
 1196 root        1  76    0 17544K  6984K select 0   4:56  0.00% l2ald
 1210 root        1   4    0  8380K  3592K kqread 0   4:28  0.00% mcsnoopd
   19 root        1 171   52     0K    16K RUN    3   2:57  0.00% idle: cpu3
   20 root        1 171   52     0K    16K RUN    2   2:56  0.00% idle: cpu2
 1217 root        2  76    0 33992K  7500K select 0   2:36  0.00% pfed
 1207 root        1  76    0 21960K 13516K select 0   2:19  0.00% utmd
   14 root        1 171   52     0K    16K RUN    8   2:15  0.00% idle: cpu8
   12 root        1 171   52     0K    16K RUN    a   2:15  0.00% idle: cpu10
   11 root        1 171   52     0K    16K RUN    b   2:15  0.00% idle: cpu11
   13 root        1 171   52     0K    16K RUN    9   2:15  0.00% idle: cpu9
   15 root        1 171   52     0K    16K RUN    7   2:15  0.00% idle: cpu7
   18 root        1 171   52     0K    16K RUN    4   2:14  0.00% idle: cpu4
   17 root        1 171   52     0K    16K RUN    5   2:14  0.00% idle: cpu5
   16 root        1 171   52     0K    16K RUN    6   2:13  0.00% idle: cpu6
 1163 root        1  76    0 10804K  3380K select 0   2:13  0.00% alarmd
   21 root        1 171   52     0K    16K RUN    1   2:10  0.00% idle: cpu1
 1238 root        1  76    0     0K    16K select 0   2:06  0.00% peer proxy
   47 root        1 -16    0     0K    16K psleep 0   1:55  0.00% vmkmemdaemon
  952 root        1  76    0  8176K  2840K select 0   1:50  0.00% eventd
 1159 root        1  76    0  3252K  1364K select 0   1:41  0.00% bslockd
 1184 root        1  76    0 11824K  4688K select 0   1:21  0.00% rtlogd
   26 root        1 -16    0     0K    16K -      0   0:55  0.00% yarrow
    6 root        1   8    0     0K    16K -      0   0:41  0.00% kqueue taskq
 1209 root        3  76    0 13588K  4620K select 0   0:39  0.00% wland
 1215 root        1  76    0  7620K  2956K select 0   0:36  0.00% irsd
    2 root        1  -8    0     0K    16K -      0   0:36  0.00% g_event
 1212 root        1  76    0 16264K  9088K select 0   0:32  0.00% nsd
    4 root        1  -8    0     0K    16K -      0   0:32  0.00% g_down
    3 root        1  -8    0     0K    16K -      0   0:29  0.00% g_up
 1204 root        1  76    0 11604K  4648K select 0   0:26  0.00% jsrpd
   39 root        1  20    0     0K    16K syncer 0   0:25  0.00% syncer
 1205 root        1  76    0 12740K  4472K select 0   0:24  0.00% pkid
   41 root        1  20    0     0K    16K vnlrum 0   0:23  0.00% vnlru_mem
 1220 root        1  76    0 31928K  7652K select 0   0:21  0.00% dcd
 1194 root        1  76    0  3112K  2076K select 0   0:20  0.00% ntpd
 1214 root        1  76    0 28792K  5904K select 0   0:19  0.00% smid
   53 root        1  -8    0     0K    16K mdwait 0   0:19  0.00% md0
 1158 root        1  76    0  2228K   868K select 0   0:18  0.00% watchdog
 1182 root        1  76    0 54788K  7260K select 0   0:14  0.00% idpd
 1197 root        1  76    0 11568K  4648K select 0   0:11  0.00% rmopd
 1174 root        1  76    0 11344K  4460K select 0   0:11  0.00% bfdd
 1164 root        1  76    0 11392K  3396K select 0   0:10  0.00% craftd
 1198 root        1  76    0 19332K  6624K select 0   0:08  0.00% cosd
 1206 root        1  76    0 30360K  5880K select 0   0:07  0.00% smihelperd

Hi,

You can try to restard MGD

check this KB

http://kb.juniper.net/InfoCenter/index?page=content&id=KB25382

if it is not working u may to restart via shell

check this link

https://kb.juniper.net/InfoCenter/index?page=content&id=KB11188&cat=EX_SERIES&actp=LIST

then login to shell using root user

then double check the process id

ps -ax | grep mgd
 500  ??  I      0:15.82 /usr/sbin/mgd -N

then kill the process

kill -9 (pid)

kill -9 500

Regards,

Mohamed Elhariry

JNCIE-M/T # 1059, CCNP & CCIP

----------------------------------------------------------------------------------------------------------------------------------------

If this post was helpful, please mark this post as an "Accepted Solution". Kudos are always appreciated!

For this much load the CPU is too high.

Since you are running a very old OD hence I would first recommen you to upgarde itto 11.4R8.

For now , could you please try restarting mgd process as I believe it could have got stuck due to some defect.