SRX

Apr 21 07:24:58  FW1jfw002 chassisd[1258]: CHASSISD_SNMP_TRAP6: SNMP trap generated: Power Supply OK (jnxContentsContainerIndex 2, jnxContentsL1Index 4, jnxContentsL2Index 0, jnxContentsL3Index 0, jnxContentsDescr node1 PEM 1, jnxOperatingState/Temp 2)
Apr 21 07:25:38  FW1jfw002 alarmd[1259]: Alarm set: ??? color=RED, class=CHASSIS, reason=PEM 1 Output Failure
Apr 21 07:25:38  FW1jfw002 craftd[1260]:  Major alarm set, PEM 1 Output Failure
Apr 21 07:25:38  FW1jfw002 craftd[1260]: forwarding display request to chassisd: type = 4, subtype = 43
Apr 21 07:25:38  FW1jfw002 chassisd[1258]: CHASSISD_IPC_UNEXPECTED_RECV: Received unexpected message from craftd: type = 4, subtype = 43
Apr 21 07:25:43  FW1jfw002 alarmd[1259]: Alarm cleared: ??? color=RED, class=CHASSIS, reason=PEM 1 Output Failure
Apr 21 07:25:43  FW1jfw002 craftd[1260]: Major alarm cleared, PEM 1 Output Failure
Apr 21 07:25:43  FW1jfw002 craftd[1260]: forwarding display request to chassisd: type = 4, subtype = 44
Apr 21 07:25:43  FW1jfw002 chassisd[1258]: CHASSISD_IPC_UNEXPECTED_RECV: Received unexpected message from craftd: type = 4, subtype = 44
Apr 21 07:26:13  FW1jfw002 alarmd[1259]: Alarm set: ??? color=RED, class=CHASSIS, reason=PEM 1 Output Failure
Apr 21 07:26:13  FW1jfw002 craftd[1260]:  Major alarm set, PEM 1 Output Failure
Apr 21 07:26:13  FW1jfw002 craftd[1260]: forwarding display request to chassisd: type = 4, subtype = 43
Apr 21 07:26:13  FW1jfw002 chassisd[1258]: CHASSISD_IPC_UNEXPECTED_RECV: Received unexpected message from craftd: type = 4, subtype = 43
Apr 21 07:26:13  FW1jfw002 chassisd[1258]: CHASSISD_SNMP_TRAP6: SNMP trap generated: Power Supply failed (jnxContentsContainerIndex 2, jnxContentsL1Index 4, jnxContentsL2Index 0, jnxContentsL3Index 0, jnxContentsDescr node1 PEM 1, jnxOperatingState/Temp 6)
Apr 21 07:26:23  FW1jfw002 alarmd[1259]: Alarm cleared: ??? color=RED, class=CHASSIS, reason=PEM 1 Output Failure
Apr 21 07:26:23  FW1jfw002 craftd[1260]: Major alarm cleared, PEM 1 Output Failure
Apr 21 07:26:23  FW1jfw002 craftd[1260]: forwarding display request to chassisd: type = 4, subtype = 44
Apr 21 07:26:23  FW1jfw002 chassisd[1258]: CHASSISD_IPC_UNEXPECTED_RECV: Received unexpected message from craftd: type = 4, subtype = 44
Apr 21 07:26:28  FW1jfw002 chassisd[1258]: CHASSISD_SNMP_TRAP6: SNMP trap generated: Power Supply OK (jnxContentsContainerIndex 2, jnxContentsL1Index 4, jnxContentsL2Index 0, jnxContentsL3Index 0, jnxContentsDescr node1 PEM 1, jnxOperatingState/Temp 2)
Apr 21 07:26:33  FW1jfw002 alarmd[1259]: Alarm set: ??? color=RED, class=CHASSIS, reason=PEM 1 Output Failure
Apr 21 07:26:33  FW1jfw002 craftd[1260]:  Major alarm set, PEM 1 Output Failure
Apr 21 07:26:33  FW1jfw002 craftd[1260]: forwarding display request to chassisd: type = 4, subtype = 43
Apr 21 07:26:33  FW1jfw002 chassisd[1258]: CHASSISD_IPC_UNEXPECTED_RECV: Received unexpected message from craftd: type = 4, subtype = 43
Apr 21 07:26:38  FW1jfw002 alarmd[1259]: Alarm cleared: ??? color=RED, class=CHASSIS, reason=PEM 1 Output Failure
Apr 21 07:26:38  FW1jfw002 craftd[1260]: Major alarm cleared, PEM 1 Output Failure
Apr 21 07:26:38  FW1jfw002 craftd[1260]: forwarding display request to chassisd: type = 4, subtype = 44
Apr 21 07:26:38  FW1jfw002 chassisd[1258]: CHASSISD_IPC_UNEXPECTED_RECV: Received unexpected message from craftd: type = 4, subtype = 44

 

This can be a few things, Power failure towards the inlet of the PEM. But also someone switching the powerbutton on the device, or an indication that the pem is about to die

I RMA this power supply ,but still has this error log

does it mean the new power supply is bad?

 

there is no one switching power button at this moment.this log last all day

 

 

 

inlet of the PEM? is this equal to power supply?

 

 

I don't think the power supply is faulty. When did you swap the PEM with this one ?

 

With inlet I mean the ac / dc power going into the PEM from the powergrid.

 

 

1 week ago,I had juniper swapped PEM with this one(since the old one also had this log)

Apr 21 01:29:40 CHASSISD_IPC_UNEXPECTED_RECV: Received unexpected message from craftd: type = 4, subtype = 44
 
Apr 21 01:29:50 CHASSISD_SNMP_TRAP6: SNMP trap generated: Power Supply OK (jnxContentsContainerIndex 2, jnxContentsL1Index 4, jnxContentsL2Index 0, jnxContentsL3Index 0, jnxContentsDescr node1 PEM 1, jnxOperatingState/Temp 2)
Apr 21 01:29:55 LCC: send: red alarm set, device Power Supply 1 1, reason PEM 1 Output Failure
Apr 21 01:29:55 POE (pem_poe_srxsme_remove): Power Supply 1 removed or faulty....decreasing POEpower limit
 
Apr 21 01:29:55 CHASSISD_IPC_UNEXPECTED_RECV: Received unexpected message from craftd: type = 4, subtype = 43
 
Apr 21 01:30:00 LCC: send: red alarm clear, device Power Supply 1 1, reason PEM 1 Output Failure
Apr 21 01:30:00 POE (pem_poe_srxsme_add): Added Power supply 1...increasing POE power limit
 
Apr 21 01:30:00 CHASSISD_IPC_UNEXPECTED_RECV: Received unexpected message from craftd: type = 4, subtype = 44
 
Apr 21 01:30:05 LCC: send: red alarm set, device Power Supply 1 1, reason PEM 1 Output Failure
Apr 21 01:30:05 POE (pem_poe_srxsme_remove): Power Supply 1 removed or faulty....decreasing POEpower limit
 
Apr 21 01:30:05 CHASSISD_IPC_UNEXPECTED_RECV: Received unexpected message from craftd: type = 4, subtype = 43
 
Apr 21 01:30:05 CHASSISD_SNMP_TRAP6: SNMP trap generated: Power Supply failed (jnxContentsContainerIndex 2, jnxContentsL1Index 4, jnxContentsL2Index 0, jnxContentsL3Index 0, jnxContentsDescr node1 PEM 1, jnxOperatingState/Temp 6)
---(more 29%)---[abort]
                                        

 

In the last log you pasted, I see different error messages then in the first one.

according to the last pasted log:

\Apr 21 01:29:55 POE (pem_poe_srxsme_remove): Power Supply 1 removed or faulty....decreasing POEpower limit
 
Apr 21 01:29:55 CHASSISD_IPC_UNEXPECTED_RECV: Received unexpected message from craftd: type = 4, subtype = 43
 
Apr 21 01:30:00 LCC: send: red alarm clear, device Power Supply 1 1, reason PEM 1 Output Failure
Apr 21 01:30:00 POE (pem_poe_srxsme_add): Added Power supply 1...increasing POE power limit

 

I think you power supply or the "backplane ports"it connects to are broken.

 

According to the first logs it looks like the power from the grid went away

Hi,Marc

first log is message,second log is chassid

 

so I guess it is backplane ports have issue

 

what is backplane ports

do u have a pic to show this?

 

is this one?

PEM<-----backplane ports<----power supply?

 

is backplane ports included in PEM module?

 

Juniper RMA PEM one week ago...

besides this,I also see Power Supply 1                switched between  Present   and OK..

Do u mean even with out power source no cord,if people insert ,then pull psu out can generate logs like the second one :power supply added power supply removed or failed?

Hi,

 

can you share the below  outputs :

 

>show chassis power

>show chassis hardware

>show chassis alarms

 

 

it is 650,no show chassic power command

 

1111@FW1jfw002> show chassis hardware
node0:
--------------------------------------------------------------------------
Hardware inventory:
Item             Version  Part number  Serial number     Description
Chassis                                11111111111      SRX650
Midplane         REV 11   710-023875   111111        
System IO        REV 10   710-023209   111111          SRXSME System IO
Routing Engine   REV 26   750-023223   11111          RE-SRXSME-SRE6
FPC 0                                                    FPC
  PIC 0                                                  4x GE Base PIC
FPC 2            REV 10   750-030454   11111111          FPC
  PIC 0                                                  2x 10G xPIM
    Xcvr 0       REV 01   740-021308   11111111          SFP+-10G-SR
    Xcvr 1       REV 01   740-021308   11111111          SFP+-10G-SR
FPC 6            REV 09   750-026182   11111111          FPC
  PIC 0                                                  16x GE gPIM
Power Supply 0   Rev 04   740-024283   1111111           PS 645W AC
Power Supply 1   Rev 04   740-024283   1111111           PS 645W AC

node1:
--------------------------------------------------------------------------
Hardware inventory:
Item             Version  Part number  Serial number     Description
Chassis                                111111111111      SRX650
Midplane         REV 11   710-023875   11111111         
System IO        REV 10   710-023209   11111111          SRXSME System IO
Routing Engine   REV 26   750-023223   11111111          RE-SRXSME-SRE6
FPC 0                                                    FPC
  PIC 0                                                  4x GE Base PIC
FPC 2            REV 10   750-030454   1111113          FPC
  PIC 0                                                  2x 10G xPIM
    Xcvr 0       REV 01   740-021308   11111111          SFP+-10G-SR
    Xcvr 1       REV 01   740-021308   11111111          SFP+-10G-SR
FPC 6            REV 08   750-026182   11111111          FPC
  PIC 0                                                  16x GE gPIM
Power Supply 0   Rev 04   740-024283   1111111           PS 645W AC
Power Supply 1   Rev 04   740-024283   1111111           PS 645W AC

{secondary:node1}
11111@FW1jfw002> show chassis alarms
node0:
--------------------------------------------------------------------------
No alarms currently active

node1:
--------------------------------------------------------------------------
No alarms currently active

With the connection to the backplane i mean the connectors on the chassis where the PEM is connected when you insert it.

Have you checked if their is no power failure om the grid ? Otherwise I suggest opening a TAC case.

Hi 

 

Please find my lab testing:

 

PEM0 is connected to chassis  with  no power cord was connected to it.

Chassid logs when No Power Cord on PEM0 module: SNMP Alert is generated every one hour :
++++++++++++++++++++++++++++++++++++++++++++++++

root@SRX-650-3# run show log chassisd | last 300
Apr 21 18:23:26 CHASSISD_SNMP_TRAP6: SNMP trap generated: Power Supply failed (jnxContentsContainerIndex 2, jnxContentsL1Index 1, jnxContentsL2Index 0, jnxContentsL3Index 0, jnxContentsDescr node0 PEM 0, jnxOperatingState/Temp 6)
Apr 21 19:23:27 CHASSISD_SNMP_TRAP6: SNMP trap generated: Power Supply failed (jnxContentsContainerIndex 2, jnxContentsL1Index 1, jnxContentsL2Index 0, jnxContentsL3Index 0, jnxContentsDescr node0 PEM 0, jnxOperatingState/Temp 6)
Apr 21 20:23:29 CHASSISD_SNMP_TRAP6: SNMP trap generated: Power Supply failed (jnxContentsContainerIndex 2, jnxContentsL1Index 1, jnxContentsL2Index 0, jnxContentsL3Index 0, jnxContentsDescr node0 PEM 0, jnxOperatingState/Temp 6)
Apr 21 21:23:30 CHASSISD_SNMP_TRAP6: SNMP trap generated: Power Supply failed (jnxContentsContainerIndex 2, jnxContentsL1Index 1, jnxContentsL2Index 0, jnxContentsL3Index 0, jnxContentsDescr node0 PEM 0, jnxOperatingState/Temp 6)
Apr 21 22:23:31 CHASSISD_SNMP_TRAP6: SNMP trap generated: Power Supply failed (jnxContentsContainerIndex 2, jnxContentsL1Index 1, jnxContentsL2Index 0, jnxContentsL3Index 0, jnxContentsDescr node0 PEM 0, jnxOperatingState/Temp 6)

Connected Power Cord with Power Source to PEM0 and Removed it :

Apr 22 21:09:56 SRX-650-3 alarmd[1319]: Alarm cleared: ??? color=RED, class=CHASSIS, reason=PEM 0 Output Failure
Apr 22 21:09:56 SRX-650-3 craftd[1320]: Major alarm cleared, PEM 0 Output Failure
Apr 22 21:09:56 SRX-650-3 craftd[1320]: forwarding display request to chassisd: type = 4, subtype = 44
Apr 22 21:09:57 SRX-650-3 chassisd[2132]: CHASSISD_SNMP_TRAP6: SNMP trap generated: Power Supply OK (jnxContentsContainerIndex 2, jnxContentsL1Index 1, jnxContentsL2Index 0, jnxContentsL3Index 0, jnxContentsDescr node0 PEM 0, jnxOperatingState/Temp 2)
Apr 22 21:09:57 SRX-650-3 chassisd[2132]: CHASSISD_IPC_UNEXPECTED_RECV: Received unexpected message from craftd: type = 4, subtype = 44
Apr 22 21:10:00 SRX-650-3 cron[20347]: (root) CMD ( /usr/libexec/atrun)
Apr 22 21:10:11 SRX-650-3 craftd[1320]: Major alarm set, PEM 0 Output Failure
Apr 22 21:10:11 SRX-650-3 craftd[1320]: forwarding display request to chassisd: type = 4, subtype = 43
Apr 22 21:10:11 SRX-650-3 alarmd[1319]: Alarm set: ??? color=RED, class=CHASSIS, reason=PEM 0 Output Failure
Apr 22 21:10:11 SRX-650-3 chassisd[2132]: CHASSISD_IPC_UNEXPECTED_RECV: Received unexpected message from craftd: type = 4, subtype = 43
Apr 22 21:10:12 SRX-650-3 chassisd[2132]: CHASSISD_SNMP_TRAP6: SNMP trap generated: Power Supply failed (jnxContentsContainerIndex 2, jnxContentsL1Index 1, jnxContentsL2Index 0, jnxContentsL3Index 0, jnxContentsDescr node0 PEM 0, jnxOperatingState/Temp 6)

Chassisd Logs after Power cord and Power Source are connected to PEM0:

Apr 22 21:09:56 POE (pem_poe_srxsme_add): Added Power supply 0...increasing POE power limit

Apr 22 21:09:57 CHASSISD_SNMP_TRAP6: SNMP trap generated: Power Supply OK (jnxContentsContainerIndex 2, jnxContentsL1Index 1, jnxContentsL2Index 0, jnxContentsL3Index 0, jnxContentsDescr node0 PEM 0, jnxOperatingState/Temp 2)
Apr 22 21:09:57 CHASSISD_IPC_UNEXPECTED_RECV: Received unexpected message from craftd: type = 4, subtype = 44

Apr 22 21:10:11 LCC: send: red alarm set, device Power Supply 0 0, reason PEM 0 Output Failure
Apr 22 21:10:11 POE (pem_poe_srxsme_remove): Power Supply 0 removed or faulty....decreasing POEpower limit

Apr 22 21:10:11 CHASSISD_IPC_UNEXPECTED_RECV: Received unexpected message from craftd: type = 4, subtype = 43

Apr 22 21:10:12 CHASSISD_SNMP_TRAP6: SNMP trap generated: Power Supply failed (jnxContentsContainerIndex 2, jnxContentsL1Index 1, jnxContentsL2Index 0, jnxContentsL3Index 0, jnxContentsDescr node0 PEM 0, jnxOperatingState/Temp 6)
Apr 22 21:14:37 SCC: ipc pipe 0xd385c0 created

 

Chassis Alarm:

{primary:node0}[edit]
root@SRX-650-3# run show chassis alarms
node0:
--------------------------------------------------------------------------
1 alarms currently active
Alarm time Class Description
2015-04-22 21:10:11 IST Major PEM 0 Output Failure

{primary:node0}[edit]
root@SRX-650-3# run show chassis environment
node0:
--------------------------------------------------------------------------
Class Item Status Measurement
Temp Routing Engine OK 48 degrees C / 118 degrees F
Routing Engine CPU OK 48 degrees C / 118 degrees F
Routing Engine Absent
Routing Engine CPU Absent
Fans SRXSME Chassis Fan 0 OK Spinning at normal speed
SRXSME Chassis Fan 1 OK Spinning at normal speed
SRXSME Chassis Fan 2 OK Spinning at normal speed
SRXSME Chassis Fan 3 OK Spinning at normal speed
Power Power Supply 0 Present <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
Power Supply 1 OK

So these alerts looks like more of Powercord and Power Source issue. You will see these alerts when someone connects and disconnects the power cord or power source.

 

Try replacing the power cord and test it.

 

Regards,
rparthi

 

thanks, rparthi,you are the man

 

 

it is very helpful

 

I am curious if the psu(pem) itself has issue,which type of logs it should generate?or power led in front plane should show this?

I agree with @rparthi but as said before have you checked if  the powergrid had a "failure" at the moment the PEM lost its power.

 

 

no,we changed to another powergrid,still same issue

as @rparthi suggested ,I will have people try another power cable