Switching

Looking for an automated solution to perform a proper shutdown on juniper equipment if the UPS kicks in.

Anything out there that can leverage APC Powerchute?

Any advise?

MichAda,

The command on Junos is "request system halt". This can be automated a few different ways. I generated a rsa key, so I didn't have to send the password from my ssh client then you can pass the commands directly. See below:

ssh -i id-rsa -l juniper 172.22.157.105 request system halt
warning: This command will halt all the members.
If planning to halt only one member use the member option
Shutdown NOW!
[pid 3287]
message sent

 The "id-rsa" is the private key file which i generated via the ssh-keygen on the switch. Here is the process I used: 

1. On the EX switch go to the shell:

{master:0}
juniper@access2-2> start shell 
% su
Password:
root@access2-2:RE:0% ssh-keygen
Generating public/private rsa key pair.
Enter file in which to save the key (/root/.ssh/id_rsa): 
Created directory '/root/.ssh'.
Enter passphrase (empty for no passphrase): 
Enter same passphrase again: 
Your identification has been saved in /root/.ssh/id_rsa.
Your public key has been saved in /root/.ssh/id_rsa.pub.
The key fingerprint is:
84:2c:8d:53:19:7e:3d:cb:c0:5c:58:ee:11:cf:1d:8d root@access2-2
root@access2-2:RE:0% cd /root/.ssh/
root@access2-2:RE:0% ls
id_rsa		id_rsa.pub
root@access2-2:RE:0% cat id_rsa.pub 
ssh-rsa AAAAB3NzaC1yc2EAAAABIw....

Once the private and public key chains are generated you will enter the pub (id_rsa.pub) keychain on the server side (switch):

root@access2-2:RE:0% cli
{master:0}
juniper@access2-2> edit 
Entering configuration mode

{master:0}[edit]
juniper@access2-2# set system login user juniper authentication ssh-rsa    

juniper@access2-2# ...5utE3rWf0AC8TdpM"  

on the ssh client side you create a file with the private key:

> vi id-rsa

-----BEGIN RSA PRIVATE KEY-----
MIIEoQIBAAKCAQEA0LAFaHuuD9zUUmb9sawrEQKbnxS8OxKEPjd99/bMB71yDgKg
xwGAVdxEmxCEX4tMrRV4Kg8zsmVdw8b+ZFVKUCo4PZ7eBJOp2XaA19ZbY4wTXcAQ
SqHb2U5p==....
-----END RSA PRIVATE KEY-----
id-rsa: new file: 27 lines, 1675 characters.

Then use chmod 600 to create the right permissions for the private keychain file. 

once all this is done now you can pass commands from the client side via ssh commands: 

> ssh -i id-rsa -l juniper 172.22.157.105 request system halt
warning: This command will halt all the members.
If planning to halt only one member use the member option
Shutdown NOW!
[pid 3287]
message sent
> 

Thanks for the response...lots of good info in there.

We've had a few power outages that lasted long enough for the UPS batteries power to be exhausted and the switches go down hard (7 switches in a Virtual Chassis).  We have had 5 of these outages and after 4 of the 5 outages, junos failed to load on one of the physical switches (a different physical switch every time) and we had to re-image to get the switch back into the VC.

That is why I'm interested in putting something in place to detect that the system in running on UPS power and then perform a proper shutdown on the switches prior to them having a hard crash.

So if I had the client side set up with ssh that would give me the ability to pass the command, but how would I get the client to send the shutdown command  after 'X' minutes of being on UPS power?

The easiest and most basic way of detecting a power loss would be through snmp trap configuration. Every time a Power Supply fails or comes online an SNMP trap will be logged. Once you configure snmp and trap groups along with a trap receiver you can then monitor the time between power off and on. 

(KB on Health monitor of CPU, but has config options for trap-groups)

http://kb.juniper.net/InfoCenter/index?page=content&id=KB16450

The more complicated way would be to have a system polling the device via ssh and grep certain key words to verify its status: 

For the 4200 you would have to use:

When the power fails you will see "failed" for the status. So when you use the ssh login mentioned before poll this command and grep for "failed":

juniper@access2-2# run show chassis environment |match power    
Power FPC 0 Power Supply 0           Failed    
      FPC 0 Power Supply 1           OK       

If you gather this along with the system date then you can count down the time you have available for the UPS batteries:

juniper@access2-2# run show system uptime 
fpc0:
--------------------------------------------------------------------------
Current time: 2010-12-06 00:26:14 EST
System booted: 2010-11-24 01:36:29 EST (1w4d 22:49 ago)
Protocols started: 2010-11-24 01:37:52 EST (1w4d 22:48 ago)
Last configured: 2010-12-05 21:30:42 EST (02:55:32 ago) by juniper
12:26AM  up 11 days, 22:50, 1 user, load averages: 0.08, 0.05, 0.01

One recommendation I would make, if you are not already running it would be to upgrade to the 10.4R5 release, which has had a "resilient file system" feature, for cases much like you are seeing with "corrupted junos software" from a improper shutdown.

You can see the details of how to upgrade and the feature details in the 10.4R5 release notes:

Select "Junos OS Release Notes for EX Switches" ---> The first section is "New Features" which talks about the "resilient dual-root  Partitions":

http://www.juniper.net/techpubs/en_US/junos10.4/information-products/topic-collections/release-notes/10.4/index.html 

I hope this helps, 

Wade

==========================================================

If the information provided helps please check it as an "accepted solution" 

I believe I'll take your suggestion to upgrade to the dual partitioning and see how that works.

Again, thanks for the response.  Unfortunately our 7 unit VC was "value engineered" to only include a single power supply in lue of the more expensive dual power supply option 🙂 so the health monitor will not be of much help.

I do appreciate the helpful post.

MichAda, have you solved your problem with shutting down router with single power supply?