Ethernet Switching
Highlighted
Ethernet Switching

fxpc

[ Edited ]
‎05-30-2019 06:55 PM

Hi All,

Some times FXPC on ex4600 is getting very high level like (80% or 90%).  How to investigate which thing is causing this. And what is the solution? Any one experience this. And also there is a interrupt 2%. Can I ask what is this and how to troubleshooting? 

 

>show system processes extensive | except 0.00
last pid: 86889; load averages: 0.45, 0.50, 0.48 up 217+00:57:54 11:04:23
146 processes: 4 running, 124 sleeping, 18 waiting

Mem: 473M Active, 175M Inact, 484M Wired, 729M Cache, 69M Buf, 25M Free
Swap: 614M Total, 614M Free


PID USERNAME THR PRI NICE SIZE RES STATE TIME WCPU COMMAND
10 root 1 171 52 0K 12K RUN 2980.0 73.54% idle
1708 root 2 44 -52 1190M 444M select 1134.5      17.68% fxpc <--------
1891 root 1 96 0 67304K 23772K select 779.2H      1.03% jdhcpd
13 root 1 -40 -159 0K 12K WAIT 71.7H                       0.83% swi2: netisr 0
1916 root 1 96 0 33576K 22028K select 621:43         0.15% l2ald
1699 root 1 96 0 52136K 18508K select 38.6H           0.10% chassisd

 

>show chassis routing-engine
Routing Engine status:
Slot 0:
       Current state                                                                                  Master
       Temperature                                                                40 degrees C / 104 degrees F
       CPU temperature                                                       40 degrees C / 104 degrees F
         DRAM 1953 MB
         Memory utilization                                       52 percent
        CPU utilization:
                    User                                                         13 percent
                    Background                                         0 percent
                    Kernel                                                  12 percent
                    Interrupt                                              2 percent  <------------------
                   Idle                                                      74 percent

Thx

Ar

4 REPLIES 4
Ethernet Switching

Re: fxpc

‎05-30-2019 11:27 PM

Hi Arix,

 

Is the fxpc stuck high or fluctuates? Are there any cores in /var/tmp?  Check if there was a significant config change or network change that marked this high CPU and troubelshoot around reverting that.  Following commands are a first step:
show system core-dumps -------------->look for any fxpc cores
show log messages | last 100
show chassis fpc -------------------->look for utilization

show chassis fpc detail
request pfe execute command "show syslog messages" target fpc0

 

Please check for any unusual spikes in the interface utilization and from above outputs look for log messages related to topology/next-hop changes etc.  Sometimes loops causing unwanted traffic forwarding back and forth across the switch can create this issue.  Also look for any other repetitive error log messages, they could also give a clue.

 

If the above doesn't help narrow it down or if you see cores, you might need some advanced help and I'd suggest you to contact TAC with this info:

 

request pfe execute command "show pfe manager statistics" target fpc0 | no-more 
request pfe execute command "set dc bc \"show c cpu0\"" target fpc0 | no-more
request pfe execute command "show threads" target fpc0 | no-more 
start shell
gcore -c /var/tmp/fxpc.live.core <pid> ------------> If there are no cores already

 

Hope this helps.

 

Regards,
-r.

--------------------------------------------------

If this solves your problem, please mark this post as "Accepted Solution."
Kudos are always appreciated Smiley Happy.

Ethernet Switching

Re: fxpc

‎07-17-2019 07:34 AM

Hi Arix,

 

Does the fxpc process stays stuck in the same percentage or does it fluctuate?

 

Can you also share "show configuration | display set | match ddos"?

 

Thanks,
Gautham

Ethernet Switching

Re: fxpc

‎08-01-2019 06:38 PM

Hi Arix,

 

-Try doing monitor interface traffic  , check for any interface with high volume of traffic

-Try doing monitor traffic interface irb size 1500 no-resolve , and check for any unsual traffic hitting the routing engine.

-show ethernet-switching mac-learning-log , show ethernet-switching mac-move-buffer , for any mac move address.

 

Please Mark My Solution Accepted if it Helped, Kudos are Appreciated too!!!

Thank you, 
Franky

Ethernet Switching

Re: fxpc

‎08-02-2019 08:15 AM

Hi Arix,

 

Since others members are already handling FXPC related query. Let me handle following query

 

<Arix>And also there is a interrupt 2%. Can I ask what is this and how to troubleshooting?

<Vishal>Interrupt in Junos is same as interrupt in a linux system. 2% interrupt is not something I would worry about.

If you still want to troubleshoot, you can try to remove USB stick if that's plugged into the system, changing console cable might also help.

You can check if any interrupt queue is getting too many of interrupts from any device in following output:

start shell
% vmstat -i
interrupt        total    rate
irq4: uart0     801     0
irq14: ata0     4183901 19
<snipped>

Some juniper device won't give the irq to device association as show in above output. In that case, you can check following output and determine what are the devices attached to particular irq which is getting flooeded and you can dtermine if something can be done about that device:

show system boot-messages | match "irq"

Kudos are appreicated if it answered part of your query!!


Thanks
Vishal