Routing
Routing

how to check how long the power supply /pem is power on?

‎05-31-2019 05:24 AM

Hi all,

 

Is there any command that we can use to check how long the PEM is power on? We want to investigate whether have power interruption or not that make the chassis reboot.

 

Thanks and appreciate any feedback

 

16 REPLIES 16
Routing

Re: how to check how long the power supply /pem is power on?

‎05-31-2019 06:19 AM

Hi Kronicklez,

 

The available commands like "show chassis environment pem" and "show chassis power" do not show the uptime of PEM. To investigate any issue with PEM you can use the below steps and commands :

1. Check if the PEM is online, AC input, DC output is fine  using "show chassis environment pem" and "show chassis power" 

2. To check if there was some fault with the PEM, use the below commands. When the PEM goes faulty, chassisd sets an alarm and SNMP trap. When the PEM issue is cleared or it comes online, the alarm is cleared. From the timestamp in the log messages using the below commands you can check events and conclude the PEM uptime : 

 

show log messages | match alarm 

show log messages | match CHASSISD_SNMP_TRAP 

show log messages |  match PEM 

 

 

Regards,

Rahul Gautam

Please mark my solution as accepted if it helped.

 

Routing

Re: how to check how long the power supply /pem is power on?

‎05-31-2019 06:39 AM

There're command to check how long the PEM has been powered on. But I am afraid that you won't get desired information from the output. The command tells you the cumulative duration that this particular PEM has been used. I think this counter won't reset after a reboot. Follow the instructions to check power AC/DC input. If external factors are fine, we could RMA the PEM 


Mengzhe Hu
JNCIE x 3 (SP DC ENT)
Routing

Re: how to check how long the power supply /pem is power on?

‎05-31-2019 06:55 AM

Hi Mhu,

 

Sometimes it difficult to fight with power electricity person if we dont have evidence said we see power distruption source in our router. So if there is command to check how long the PEM is stable then it easy for us.

 

Thanks and appreciate any feedback command

 

Routing

Re: how to check how long the power supply /pem is power on?

‎05-31-2019 08:20 AM

Hi Kronicklez,

 

In case of any power issue outside the PEM will log alarm for bad input in the logs. You can reference this to check the power feeds and input. Also when there is issue with PEM, the status of the PEM is not "online"  which can be checked with command "show chassis environment pem". It may be present/check. I think these output should be enough to initiate an investigation.  

 

 send: red alarm set, device PEM 1, reason PEM 1 Not OK

 CHASSISD_PEM_INPUT_BAD: status failure for power supply 1 (status bits: 0x4e); check circuit breaker

 

Regards,

Rahul Gautam

Please mark my solution as accepted if it helped.

Routing

Re: how to check how long the power supply /pem is power on?

‎05-31-2019 09:42 AM

Hi Rahul,

 

The issue now on both RE when i execute command "show chassis routing-engine" both RE show below output. But the power electricity people said the power is stable (DC power). For me it's very rare both RE reboot at same time without power distruption. I'm already open Case and for now waiting JTAC to analysis the /var/log/. Please take note the box is MX480.

 

Last reboot reason             0x1Smiley Tongueower cycle/failure

 

https://kb.juniper.net/InfoCenter/index?page=content&id=KB31157&cat=SRX_SERIES&actp=LIST

 

Routing

Re: how to check how long the power supply /pem is power on?

‎05-31-2019 10:07 AM

Hi all,

 

I'm cannot open this KB url. Can someone verify whether this KB is exists.

 

https://kb.juniper.net/InfoCenter/index?page=content&id=KB14846&act=login

 

user@mx480> start shell

%

%

% sysctl hw.re.reboot_reason

hw.re.reboot_reason: 1

 

Routing

Re: how to check how long the power supply /pem is power on?

‎05-31-2019 10:11 AM

Hi Kronicklez,

 

Thank you for sharing the details. It is good that you opened a JTAC case since both RE rebooted at the same time. The input feeds may be providing proper power, but what if the PEMs have issue. There could be issue with Control board which houses the  routing engine.

The last reboot reason of power cycle/failure in the output does not only mean that power modules have failed or have issues. There could be other reasons also. Please refer below:

 

Last reboot reason             0x1Smiley Tongueower cycle/failure

 

https://www.juniper.net/documentation/en_US/junos/topics/reference/command-summary/show-chassis-rout...

power cycle/failure—Halt of the Routing Engine using the halt command, powering down using the power button on the chassis or any other method (such as removal of the control board or Routing Engine), and then powering back the Routing Engine. A halt of the operating system also occurs if you enter the request system halt command. You can enter this command to halt the system operations on the chassis or specific Routing Engines. To restart the software, press any key on the keyboard.

 

 

Regards,

Rahul Gautam

Please mark my solution as accepted if it helped.

Routing

Re: how to check how long the power supply /pem is power on?

‎05-31-2019 10:17 AM

Hi Kronicklez,

 

KB14846  shows "Article is in review and not yet ready for viewing"

 

 

Regards,

Rahul Gautam

Please mark my solution as accepted if it helped.

Routing

Re: how to check how long the power supply /pem is power on?

‎05-31-2019 10:20 AM

Hi all -

 

It looks like I'm able to view the KB link now. Is that the case for everyone else?

 

 

Jeremiah
J-Net Community Manager
Routing

Re: how to check how long the power supply /pem is power on?

‎05-31-2019 10:27 AM

 Hi Jeremiah,

 

 

I'm still cannot view the that KB url.

 

Hi Rahul,

 

When i try to search the meaning of  below code then i see the old post that u said code "1" is related to watchdog issue. There is no core-dump detected on both RE

 

% sysctl hw.re.reboot_reason

hw.re.reboot_reason: 1

 

https://forums.juniper.net/t5/Junos/Unusual-Reboot-Reason-on-Juniper/td-p/309913

 

Thanks

 

Routing

Re: how to check how long the power supply /pem is power on?

‎05-31-2019 12:04 PM

Hi Kronicklez,

 

Yes,  Bit 1 means reboot was triggered by hardware watchdog. 


Watchdog daemon on the RE talks to the hardware watchdog timer. With the available data it needs to be checked why the watchdog was triggered.
The watchdog may reboot the system due to some unrecoverable software/hardware errors on the RE.
You can check for reboot in "show system uptime | match boot" and "show system boot-messages". I hope all the FRUs are online as of now.

 

Regards,
Rahul Gautam
Please mark my solution as accepted if it helped.

Routing

Re: how to check how long the power supply /pem is power on?

‎05-31-2019 12:44 PM


Hi kronicklez,

 

Here are the different bit meanings which are supported across RE-3.0, RE-4.0, M120, and MX-series Routing Engines:

Bit 0 is set when there is a reboot due to power failure or power cycle.  

Bit 1 is set when there is a reboot triggered by hardware watchdog. 

Bit 2 is set when a reboot is initiated by the reset b

 

You would get this information using the following commands

 

% sysctl hw.re.reboot_reason
%sysctl -a | grep boot
% sysctl -a | grep dual
% sysctl -a | grep bios
% sysctl -a

 

If this solves your problem, please mark this post as "Accepted Solution."
Kudos are always appreciated Smiley Happy

 

 

 

 

Routing

Re: how to check how long the power supply /pem is power on?

‎06-01-2019 01:27 AM

Hi Rahul,

 

Below is the log. I think not have any error/issue right based on the log below:

 

{master}
test@mx480> show system boot-messages
Copyright (c) 1992-2016 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD JNPR-10.3-20170601.351063_builder_stable_10 #0: Thu Jun 1 00:39:07 PDT 2017
builder@basith.juniper.net:/volume/build/junos/occam/freebsd/stable_10/20170601.builder.351063/obj/amd64/juniper/kernels/JNPR-AMD64-PRD/kernel amd64
gcc version 4.2.1 (for JUNOS)
VT(vga): resolution 640x480
sysctl: OID number(10) is already in use for 'option_Q_maxlen'
CPU: Intel(R) Xeon(R) CPU C5518 @ 1.73GHz (1729.02-MHz K8-class CPU)
Origin="GenuineIntel" Id=0x106e4 Family=0x6 Model=0x1e Stepping=4
Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>
Features2=0x9ce3bd<SSE3,DTES64,MON,DS_CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,DCA,SSE4.1,SSE4.2,POPCNT>
AMD Features=0x28100800<SYSCALL,NX,RDTSCP,LM>
AMD Features2=0x1<LAHF>
VT-x: PAT,HLT,MTF,PAUSE,EPT,VPID
TSC: P-state invariant, performance statistics
real memory = 34359738368 (32768 MB)
avail memory = 33274347520 (31732 MB)
platform_early_bootinit: M/T/EX Series Early Boot Initialization
Event timer "LAPIC" quality 400
ACPI APIC Table: <ALASKA A M I>
FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs
FreeBSD/SMP: 1 package(s) x 4 core(s)
cpu0 (BSP): APIC ID: 0
cpu1 (AP): APIC ID: 2
cpu2 (AP): APIC ID: 4
cpu3 (AP): APIC ID: 6
random: <Software, Yarrow> initialized
Security policy loaded: MAC/veriexec (mac_veriexec)
Security policy loaded: JUNOS MAC/privcheck (mac_privcheck)
Security policy loaded: Junos MAC/fips (mac_fips)
MAC/veriexec fingerprint module loaded: SHA1
MAC/veriexec fingerprint module loaded: SHA256
ACPI BIOS Warning (bug): 32/64X FACS address mismatch in FADT: 0xCF261F40/0x00000000CF261F80, using 64-bit address (20150515/tbfadt-309)
ioapic0 <Version 2.0> irqs 0-23 on motherboard
ioapic1 <Version 2.0> irqs 24-47 on motherboard
netisr_init: !debug_mpsafenet, forcing maxthreads from 4 to 1
Initializing M/T/EX platform properties ..
rts_sysconf_ifstate_init: registerd rtcb osd: slotid 2
ofwbus0: <Open Firmware Device Tree> on motherboard
fdtio0: <FDT I/O port space> on ofwbus0
fpga0: <RE system controller> port 0x191-0x198 on fdtio0
cryptosoft0: <software crypto> on motherboard
acpi0: <ALASKA A M I> on motherboard
acpi0: Power Button (fixed)
cpu0: <ACPI CPU> on acpi0
cpu1: <ACPI CPU> on acpi0
cpu2: <ACPI CPU> on acpi0
cpu3: <ACPI CPU> on acpi0
attimer0: <AT timer> port 0x40-0x43 irq 0 on acpi0
Timecounter "i8254" frequency 1193182 Hz quality 0
Event timer "i8254" frequency 1193182 Hz quality 100
atrtc0: <AT realtime clock> port 0x70-0x71 irq 8 on acpi0
Event timer "RTC" frequency 32768 Hz quality 0
Timecounter "ACPI-fast" frequency 3579545 Hz quality 900
acpi_timer0: <24-bit timer at 3.579545MHz> port 0x408-0x40b on acpi0
pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0
pci0: <ACPI PCI bus> on pcib0
pcib1: <ACPI PCI-PCI bridge> mem 0xfb900000-0xfb90ffff irq 24 at device 3.0 on pci0
pci3: <ACPI PCI bus> on pcib1
em0: <Intel(R) PRO/1000 Network Connection 7.6.1-k> port 0x4000-0x401f mem 0xfbd00000-0xfbd1ffff,0xfbd20000-0xfbd23fff irq 24 at device 0.0 on pci3
em0: Using an MSI interrupt
if_initname: em
pcib2: <ACPI PCI-PCI bridge> irq 32 at device 4.0 on pci0
pci4: <ACPI PCI bus> on pcib2
em1: <Intel(R) PRO/1000 Network Connection 7.6.1-k> port 0x3000-0x301f mem 0xfbc00000-0xfbc1ffff,0xfbc20000-0xfbc23fff irq 32 at device 0.0 on pci4
em1: Using an MSI interrupt
if_initname: em
pcib3: <ACPI PCI-PCI bridge> irq 35 at device 5.0 on pci0
pci5: <ACPI PCI bus> on pcib3
em2: <Intel(R) PRO/1000 Network Connection 7.6.1-k> port 0x2000-0x201f mem 0xfbb00000-0xfbb1ffff,0xfbb20000-0xfbb23fff irq 35 at device 0.0 on pci5
em2: Using an MSI interrupt
if_initname: em
pcib4: <ACPI PCI-PCI bridge> irq 38 at device 6.0 on pci0
pci6: <ACPI PCI bus> on pcib4
pcib5: <PCI-PCI bridge> irq 38 at device 0.0 on pci6
pci7: <PCI bus> on pcib5
acb0: <Atlas Control Subsystem> mem 0xfba10000-0xfba103ff irq 39 at device 13.0 on pci7
acb0: acb: softc at 0xfffff8000a54e380, fpga regs at 0xfffff800fba10000
pci7: <network> at device 14.0 (no driver attached)
ehci0: <Intel PCH USB 2.0 controller USB-A> mem 0xfbe21000-0xfbe213ff irq 23 at device 29.0 on pci0
usbus0: EHCI version 1.0
usbus0 on ehci0
pcib6: <ACPI PCI-PCI bridge> at device 30.0 on pci0
pci25: <ACPI PCI bus> on pcib6
gpio0: <GPIO controller> at device 31.0 on pci0
gpio_lpc_dev_init, Found bios device: 0x3b148086
atapci0: <Intel 5 Series/3400 Series PCH SATA300 controller> port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0x5090-0x509f,0x5080-0x508f irq 19 at device 31.2 on pci0
ata0: <ATA channel> at channel 0 on atapci0
ata1: <ATA channel> at channel 1 on atapci0
ichsmb0: <Intel PCH SMBus controller> port 0x5000-0x501f mem 0xfbe20000-0xfbe200ff irq 18 at device 31.3 on pci0
atapci1: <Intel 5 Series/3400 Series PCH SATA300 controller> port 0x5070-0x5077,0x5060-0x5063,0x5050-0x5057,0x5040-0x5043,0x5030-0x503f,0x5020-0x502f irq 19 at device 31.5 on pci0
ata2: <ATA channel> at channel 0 on atapci1
ata3: <ATA channel> at channel 1 on atapci1
acpi_button0: <Power Button> on acpi0
uart0: <Non-standard ns8250 class UART with FIFOs> port 0x3f8-0x3ff irq 4 on acpi0
uart0: console (9600,n,8,1)
uart0: [GIANT-LOCKED]
uart1: <Non-standard ns8250 class UART with FIFOs> port 0x2e8-0x2ef irq 7 on acpi0
uart1: [GIANT-LOCKED]
uart2: <Non-standard ns8250 class UART with FIFOs> port 0x3e8-0x3ef irq 5 on acpi0
uart2: [GIANT-LOCKED]
uart3: <Non-standard ns8250 class UART with FIFOs> port 0x2f8-0x2ff irq 3 on acpi0
uart3: [GIANT-LOCKED]
est0: <Enhanced SpeedStep Frequency Control> on cpu0
p4tcc0: <CPU Frequency Thermal Control> on cpu0
est1: <Enhanced SpeedStep Frequency Control> on cpu1
p4tcc1: <CPU Frequency Thermal Control> on cpu1
est2: <Enhanced SpeedStep Frequency Control> on cpu2
p4tcc2: <CPU Frequency Thermal Control> on cpu2
est3: <Enhanced SpeedStep Frequency Control> on cpu3
p4tcc3: <CPU Frequency Thermal Control> on cpu3
virtual chassis member-id = 255
virtual chassis mode = 0
virtual chassis fpc-slots = 255
virtual chassis effective fpc-slots = 12
virtual chassis ifstate revert = 1
Initializing product: 33 ..
Setting up M/T interface operations and attributes
md0: Preloaded image </packages/sets/active/boot/os-kernel/contents.izo> 7983616 bytes at 0xffffffff81be7e30
random: unblocking device.
usbus0: 480Mbps High Speed USB v2.0
Timecounters tick every 1.000 msec
Loading Redundant LT driver
Registered AMT tunnel Encap with UDP Tunnel!
rts_init: registered inrtsock osd: slotid 3
###PCB Group initialized for udppcbgroup
###PCB Group initialized for tcppcbgroup
ugen0.1: <Intel> at usbus0
uhub0: <Intel EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus0
uhub0: 1 port with 1 removable, self powered
ugen0.2: <vendor 0x8087> at usbus0
uhub1: <vendor 0x8087 product 0x0020, class 9/0, rev 2.00/0.00, addr 2> on usbus0
ada0 at ata0 bus 0 scbus0 target 0 lun 0
ada0: <StorFly - VSFA18PI032G-1A0 0605-1A0> ACS-2 ATA SATA 2.x device
ada0: Serial Number P1T05004099611010222
ada0: 300.000MB/s transfers (SATA 2.x, UDMA5, PIO 512bytes)
ada0: 28496MB (58359808 512 byte sectors)
uhub1: 6 ports with 6 removable, self powered
ada0: Previously was known as ad0
ada1 at ata0 bus 0 scbus0 target 1 lun 0
ada1: <VTDCFAPI004G-KC2 130827> ATA8-ACS device
ada1: Serial Number 41359-0618
ada1: 300.000MB/s transfers (SATA 2.x, UDMA5, PIO 512bytes)
ada1: 3998MB (8187984 512 byte sectors)
ada1: Previously was known as ad1
Kernel thread "wkupdaemon" (pid 21) exited prematurely.
SMP: AP CPU #1 Launched!
SMP: AP CPU #2 Launched!
SMP: AP CPU #3 Launched!
Timecounter "TSC" frequency 1729021720 Hz quality 1000
hwpmc: SOFT/16/64/0x67<INT,USR,SYS,REA,WRI> TSC/1/64/0x20<REA> IAP/4/48/0x3ff<INT,USR,SYS,EDG,THR,REA,WRI,INV,QUA,PRC> IAF/3/48/0x67<INT,USR,SYS,REA,WRI> UCP/8/48/0x3f8<EDG,THR,REA,WRI,INV,QUA,PRC> UCF/1/48/0x60<REA,WRI>
Trying to mount root from cd9660:/dev/md0.uzip []...

 

Routing

Re: how to check how long the power supply /pem is power on?

‎06-01-2019 05:31 AM

Hi Kronicklez,

Yes, there are no errors in the above logs. 

 

Regards,
Rahul Gautam
Please mark my solution as accepted if it helped.

Routing

Re: how to check how long the power supply /pem is power on?

‎06-12-2019 03:35 AM

On MX 2010 and MX 2020 we can use the :  show chassis environment pem command to check the no of hours Power Supply or PSM is used.  Attached : sample output from MX 2010 device. 
Accept my solution if this helps .

 

Regards,

Sharat

JNCIS-ENT, JNCIS-SP

Attachments

Routing

Re: how to check how long the power supply /pem is power on?

‎06-12-2019 06:05 AM

Hi ,

Chassisd logs will give you PEM uptime . 

 

labroot@jtac-mx104-r2020-re0> show log chassisd | match PEM

Jun 12 10:58:25 Successfully created hwdb handle 0x214e090 for FRU PEM
Jun 12 10:58:42 PEM 0 added
Jun 12 10:58:42 pem altius add slot 0, fru 0x2235000
Jun 12 10:58:42 hwdb: entry for pem 0 at slot 0 inserted
Jun 12 10:58:42 PEM 1 added
Jun 12 10:58:42 pem altius add slot 1, fru 0x2237000
Jun 12 10:58:42 hwdb: entry for pem 0 at slot 1 inserted
Jun 12 12:59:53 pem show voltage : enter
Jun 12 12:59:54 pem show voltage : Exit
Jun 12 13:00:02 pem show voltage : enter
Jun 12 13:00:02 pem show voltage : Exit

 

Thx

Anish k t