M7i high memory utilization

[ricky72]

Hi
My customer is sometimes experiencing slowness in internet access.
Internet bgp peering is provided using M7i, the only think I see is high memory utilization (see attached info).

Any help is appreciated.

# run show chassis routing-engine       
Routing Engine status:
    Temperature                 29 degrees C / 84 degrees F
    CPU temperature             33 degrees C / 91 degrees F
    DRAM                       768 MB
    Memory utilization          97 percent
    CPU utilization:
      User                      32 percent
      Background                 8 percent
      Kernel                    50 percent
      Interrupt                 11 percent
      Idle                       0 percent
    Model                          RE-5.0
    Serial ID                      9009043529
    Start time                     2010-11-12 15:16:56 CET
    Uptime                         565 days, 1 hour, 41 minutes, 29 seconds
    Last reboot reason             Router rebooted after a normal shutdown.
    Load averages:                 1 minute   5 minute  15 minute
                                       2.47       1.98       1.79

Tks
Ric

[tomi]

Hello.

Could you paste show chassis cfeb output.

[ricky72]

> show chassis cfeb   
CFEB status:
  State                                 Online   
  Intake temperature                 23 degrees C / 73 degrees F
  Exhaust temperature                28 degrees C / 82 degrees F
  CPU utilization                    15 percent
  Interrupt utilization               8 percent
  Heap utilization                   61 percent
  Buffer utilization                 28 percent
  Total CPU DRAM                    128 MB
  Internet Processor II                 Version 2, Foundry IBM, Part number 164
  Start time:                           2010-11-12 15:21:18 CET
  Uptime:                               566 days, 16 hours, 54 minutes, 45 seconds

[Saverio]

Hi Ricky.

It is quite clear what is happening; you are running into a severe dfcd memory leak:

run show system processes extensive    
..
  PID USERNAME  THR PRI NICE   SIZE    RES STATE    TIME   WCPU COMMAND
 1299 root        1 132    0   392M   367M RUN    326.9H 53.42% rpd
...
 1274 root        1  96    0   818M  1736K select  37:39  0.00% dfcd

It is clearly a leak (RES is very low, yet the SIZE is huge - this means most of the allocated memory is being swapped to disk, and never touched).

Most probably it is PR562242, but your friendly JTAC engineers will be able to confirm it for sure:

https://prsearch.juniper.net/InfoCenter/index?page=prcontent&id=PR562242

It may be possible to recover by just restarting the process (unless you are in the middle of some critical intercept session), but I would do this only with jtac support.

Let me know how it goes.

Saverio

[Saverio]

---

ricky72 wrote:

My customer is sometimes experiencing slowness in internet access.

---

Just another point: your M7i is really running out of memory due to the leak I mentioned, but I do not think this is the only cause of your customer's 'slow internet'  - there may some other issue besides this.

I would suggest to fix the memory issue first, then double-check if there is anything else going on, forwarding-wise.

Saverio