Switching

Hello list,

From time to time, some random port on my VC stops working. The log shows a lot of lines like this:

Jun 17 08:27:24  sw-loc01 /kernel: kmem type session using 57932K, exceeding limit 40960K
Jun 17 08:27:27  sw-loc01 /kernel: rt_pfe_veto: Memory over consumed. Op 8, rtsm_id 47, msg type 10
Jun 17 08:27:32  sw-loc01 /kernel: rt_pfe_veto: Memory over consumed. Op 1, rtsm_id 47, msg type 2
Jun 17 08:27:38  sw-loc01 /kernel: rt_pfe_veto: Memory over consumed. Op 8, rtsm_id 47, msg type 10
Jun 17 08:27:42  sw-loc01 /kernel: rt_pfe_veto: Memory over consumed. Op 1, rtsm_id 47, msg type 2
Jun 17 08:27:47  sw-loc01 /kernel: rt_pfe_veto: Memory over consumed. Op 1, rtsm_id 47, msg type 2
Jun 17 08:27:47  sw-loc01 mib2d[1204]: SNMP_TRAP_LINK_DOWN: ifIndex 677, ifAdminStatus up(1), ifOperStatus down(2), ifName ge-2/0/26
Jun 17 08:27:52  sw-loc01 /kernel: rt_pfe_veto: Memory over consumed. Op 1, rtsm_id 47, msg type 2
Jun 17 08:27:52  sw-loc01 cosd[1210]: COSD_GENCFG_WRITE_FAILED: GENCFG write failed (op, minor_type) = (delete, policy inline) for tbl 2 if 301 ge-2/0/28 Reason: No buffer space available
Jun 17 08:27:57  sw-loc01 /kernel: rt_pfe_veto: Memory over consumed. Op 1, rtsm_id 47, msg type 2
Jun 17 08:28:02  sw-loc01 /kernel: rt_pfe_veto: Memory over consumed. Op 1, rtsm_id 47, msg type 2
Jun 17 08:28:22  sw-loc01 last message repeated 4 times
Jun 17 08:28:24  sw-loc01 /kernel: kmem type session using 57933K, exceeding limit 40960K
Jun 17 08:28:28  sw-loc01 /kernel: rt_pfe_veto: Memory over consumed. Op 1, rtsm_id 47, msg type 2
Jun 17 08:28:32  sw-loc01 /kernel: rt_pfe_veto: Memory over consumed. Op 1, rtsm_id 47, msg type 2
Jun 17 08:29:22  sw-loc01 last message repeated 10 times
Jun 17 08:29:24  sw-loc01 /kernel: kmem type session using 57934K, exceeding limit 40960K
Jun 17 08:29:27  sw-loc01 /kernel: rt_pfe_veto: Memory over consumed. Op 1, rtsm_id 47, msg type 2
Jun 17 08:29:32  sw-loc01 /kernel: rt_pfe_veto: Memory over consumed. Op 1, rtsm_id 47, msg type 2
Jun 17 08:30:07  sw-loc01 last message repeated 7 times
Jun 17 08:30:07  sw-loc01 /kernel: rt_pfe_veto: Possible slowest client is sfid. States processed - 3549580. States to be processed - 10
Jun 17 08:30:07  sw-loc01 /kernel: rt_pfe_veto: Possible second slowest client is pfem. States processed - 3549580. States to be processed - 10

After a reboot everything is working fine, until the next session of memory exhaustion.

The VC has 4 members and the config is the most basic possible.

Any hints for a junOS noob ?

Would configuring remote logging save me some memory ?

Thanks,

Marius

Forgot to add, when this happens I also get an "administrative interface down" error and the only way to access the switches is over serial link.

Hello,

Are You using Your EX kit within supported route/MAC address/ARP entries limits?

Or do You exceed the number of routes/MAC addresses/ARP entries?

FYI, all EX supported limits are publshed here http://www.juniper.net/us/en/products-services/switching/ex-series/ 

For instance, EX4200 supports 32K MAC addresses, 16K IPv4 unicast routes and 16K ARP entries.

---

@mariusster1 wrote:

Forgot to add, when this happens I also get an "administrative interface down" error and the only way to access the switches is over serial link.

 

---

Do You have "mac-move" limiting with action "shutdown" configured?

http://www.juniper.net/techpubs/en_US/junos9.4/topics/concept/port-security-mac-limiting-and-mac-move-limiting.html#jN1004B

HTH

Thanks
Alex
 

I'm well below those limits, ounr network has around 200 computers.

And secondly, the ports have absolutely no restriction.

Looks like a memory leak.  What version of code are you running?

All four switches are running JUNOS 12.3R1.7

Bug, PR 864551.  Fixed in 12.3R2 and later.  12.3R3 released on Wednesday, I suggest going to that version.

-Chad

Is this one reason to stick to JTAC-recommended JUNOS versions?