Ethernet Switching
Highlighted
Ethernet Switching

EX4200, PoE not working on all ports

‎05-01-2019 05:54 AM

We have a stack of four EX4200-48PX switches, running Junos 15.1R7.9. One of these switches has 31 Ruckus R710 access points patched in, the remaining 17 ports have nothing patched in. Yesterday we had a power outage and the entire stack went down after the UPS ran out of juice. Two hours later the power came back on, stack started up and now I can only get 24 ports on this switch to deliver PoE. The PoE firmware is upgraded to the latest. Below is the output of "show poe controller", the switch in question is number 1 which is only drawing 134.20 watts.

 

Controller index Maximum power Power consumption Guard band Management Status Lldp Priority
0 740.00W 69.00W 0W Class AT_MODE Disable
1 740.00W 134.20W 0W Class AT_MODE Disable
2 740.00W 41.20W 0W Class AT_MODE Disable
3 740.00W 126.80W 0W Class AT_MODE Disable

 

Below is the output of "show poe interface". You will notice that ports 0 through 24 are putting out PoE power and ports 25 to 31 are not. Any ideas?

 

Interface Admin status Oper status Max power Priority Power consumption Class
ge-1/0/0 Enabled ON 25.0W(L) Low 5.2W 4
ge-1/0/1 Enabled ON 25.0W(L) Low 5.2W 4
ge-1/0/2 Enabled ON 25.0W(L) Low 5.0W 4
ge-1/0/3 Enabled ON 25.0W(L) Low 5.6W 4
ge-1/0/4 Enabled ON 25.0W(L) Low 5.0W 4
ge-1/0/5 Enabled ON 25.0W(L) Low 4.8W 4
ge-1/0/6 Enabled ON 25.0W(L) Low 5.0W 4
ge-1/0/7 Enabled ON 25.0W(L) Low 4.8W 4
ge-1/0/8 Enabled ON 25.0W(L) Low 4.8W 4
ge-1/0/9 Enabled ON 25.0W(L) Low 4.8W 4
ge-1/0/10 Enabled ON 25.0W(L) Low 4.6W 4
ge-1/0/11 Enabled ON 25.0W(L) Low 4.6W 4
ge-1/0/12 Enabled ON 25.0W(L) Low 4.2W 4
ge-1/0/13 Enabled ON 25.0W(L) Low 4.2W 4
ge-1/0/14 Enabled ON 25.0W(L) Low 5.4W 4
ge-1/0/15 Enabled ON 25.0W(L) Low 4.8W 4
ge-1/0/16 Enabled ON 25.0W(L) Low 5.0W 4
ge-1/0/17 Enabled ON 25.0W(L) Low 5.2W 4
ge-1/0/18 Enabled ON 25.0W(L) Low 4.6W 4
ge-1/0/19 Enabled ON 25.0W(L) Low 4.8W 4
ge-1/0/20 Enabled ON 25.0W(L) Low 5.8W 4
ge-1/0/21 Enabled ON 25.0W(L) Low 4.8W 4
ge-1/0/22 Enabled ON 25.0W(L) Low 9.6W 4
ge-1/0/23 Enabled ON 25.0W(L) Low 4.2W 4
ge-1/0/24 Enabled ON 25.0W(L) Low 5.4W 4
ge-1/0/25 Enabled OFF 30.0W Low 0.0W not-applicable
ge-1/0/26 Enabled OFF 30.0W Low 0.0W not-applicable
ge-1/0/27 Enabled OFF 30.0W Low 0.0W not-applicable
ge-1/0/28 Enabled OFF 30.0W Low 0.0W not-applicable
ge-1/0/29 Enabled OFF 30.0W Low 0.0W not-applicable
ge-1/0/30 Enabled OFF 30.0W Low 0.0W not-applicable
ge-1/0/31 Enabled OFF 30.0W Low 0.0W not-applicable
ge-1/0/32 Enabled OFF 15.4W Low 0.0W not-applicable
ge-1/0/33 Enabled OFF 15.4W Low 0.0W not-applicable
ge-1/0/34 Enabled OFF 15.4W Low 0.0W not-applicable
ge-1/0/35 Enabled OFF 15.4W Low 0.0W not-applicable
ge-1/0/36 Enabled OFF 15.4W Low 0.0W not-applicable
ge-1/0/37 Enabled OFF 15.4W Low 0.0W not-applicable
ge-1/0/38 Enabled OFF 15.4W Low 0.0W not-applicable
ge-1/0/39 Enabled OFF 15.4W Low 0.0W not-applicable
ge-1/0/40 Enabled OFF 30.0W Low 0.0W not-applicable
ge-1/0/41 Enabled OFF 15.4W Low 0.0W not-applicable
ge-1/0/42 Enabled OFF 15.4W Low 0.0W not-applicable
ge-1/0/43 Enabled OFF 15.4W Low 0.0W not-applicable
ge-1/0/44 Enabled OFF 15.4W Low 0.0W not-applicable
ge-1/0/45 Enabled OFF 15.4W Low 0.0W not-applicable
ge-1/0/46 Enabled OFF 15.4W Low 0.0W not-applicable
ge-1/0/47 Enabled OFF 15.4W Low 0.0W not-applicable

 

 

44 REPLIES 44
Ethernet Switching

Re: EX4200, PoE not working on all ports

‎05-01-2019 06:29 AM

Hi HendersonD,

 

From this output, POE seems to be enabled on all the ports.  Are there any trunk ports on this switch? If yes, please turn off POE on the trunks (if any) and check.  May sometimes need to bounce the ports after the change.

 

Let me know if that worked.

 

Hope this helps.

 

Regards,
-r.

--------------------------------------------------

If this solves your problem, please mark this post as "Accepted Solution."
Kudos are always appreciated Smiley Happy.

Ethernet Switching

Re: EX4200, PoE not working on all ports

‎05-01-2019 07:49 AM

We have fiber connections to the top and bottom switch in this stack through the EX4200 uplink ports. PoE is not enabled on them. Every port on this switch that has an access point plugged in is setup as a trunk port since we need to carry several vlans to each AP. In other words, I cannot disable PoE on this ports since the AP needs PoE and multiple vlans

Ethernet Switching

Re: EX4200, PoE not working on all ports

‎05-01-2019 07:59 AM

I do not think your situation has anything to do with port configuration of Trunk or Access.  I did a lot of work with EX4200 and POE previously, BUT I only used 12.3 based code.

 

It looks to me like the switch/POE Controller "thinks" every port needs a reserve of 30W (class 4) despite LLDP-MED telling in 25W.  The reason I say this is 24 x 30 = 720, and POE Budget for EX4200-48PX is 740W max.  So if port 25 asks for 25 or 30W, no power would be applied.

 

Yes, this should NOT be happening -Smiley Sad

 

Now since this all worked once, potential re-boot(s) of whole VC might clear it, but I might suggest instead you try with 12.3R12-S[latest] and see if results change.  No need to worry about 12.3, as this code will be supported on legacy (non-ELS) EX switches for same amount of time as 15.1.

 

Good luck.

Ethernet Switching

Re: EX4200, PoE not working on all ports

‎05-01-2019 08:12 AM

We have a lot of stacks of EX4200 around campus and they are all running the same code at the moment. I am a bit reluctant to run a different version of code on this one stack. Is there anything else I can do to the switch to get PoE working properly? A little history might help here. Months ago we were running a previous version of 15.1 code. We upgraded to Junos 15.1R7.9 and never had a problem until this power outage. I did reboot the entire stack this morning and that did not bring back PoE on the ports in question. Why would PoE be working fine for months, we get a power outage that downs all of my switches, and when the come back up this one stack is having problems? I had a secretary call me a few minutes back and her IP phone and computer are not working. She is hooked to the same stack, different switch in the stack. Since her phone is not getting PoE I am thinking at least one other switch in this stack is having a similar issue

Ethernet Switching

Re: EX4200, PoE not working on all ports

‎05-01-2019 08:17 AM

One other thing that might figure into the problem. Our IP phone system we got up and running just a few weeks back. Our switch stacks have never been rebooted since the new phones went in. We do have this on every switch

poe {
interface all;
}

 

I cannot figure out any relationship between the IP phones going in, the switches being rebooted and losing PoE power on this one switch that only has wireless access point plugged into it, but I wanted to mention this

 

 

Ethernet Switching

Re: EX4200, PoE not working on all ports

‎05-01-2019 08:42 AM

Hi HendersonD,

 

The "set poe interface all" configuration should be it.  Re-iterating myself, what I meant is if you have any of the switch ports that are not actually connecting to APs/end-hosts that are POE devices, please disable POE on those ports.  Enabling POE on any other ports that simply connect to other switches for example, creates back pressure and POE issues for sure.

 

Hope this helps.

 

Regards,
-r.

--------------------------------------------------

If this solves your problem, please mark this post as "Accepted Solution."
Kudos are always appreciated Smiley Happy.

 

Ethernet Switching

Re: EX4200, PoE not working on all ports

‎05-01-2019 09:09 AM

Can't help you with the why, but I believe (at least internally) switch thinks 30W per port, which allows only 24 ports active.

 

As for why, you're going to have to work with TAC, but I still think trying 12.3 might get you a solution faster.

 

Up to you, . . .

Ethernet Switching

Re: EX4200, PoE not working on all ports

‎05-01-2019 09:17 AM

We do not have PoE enabled on any ports that connect to another switch. We have single mode fiber connections between switch stacks and our Juniper core in a hub and spoke configuration. There is one caveat to this, our Mitel phones (like all IP phones) have a two port switch on them. Ethernet patch cable from wall plate to phone. A second ethernet patch cable from phone to desktop computer or printer. So we are left with only a small number of ports that have just a printer or desktop (no phone daisy chained in between) connected that we could turn off PoE power.

 

Again, our phone system is brand new, I am guessing this one switch stack has 30-40 IP phones plugged in as well. None of these IP phones are plugged into the one switch that has all the access points plugged. This stack has been rebooted many times over the years as we upgrade Junos and never had a problem. I do not think it is a coincidence that the first time the stack is restarted after our IP phones go in and PoE is having a problem.

Ethernet Switching

Re: EX4200, PoE not working on all ports

‎05-01-2019 09:26 AM

I personally do not believe in coincedence when it comes to Networking.  Have not seen it in 40 years I've been involved!!

Ethernet Switching

Re: EX4200, PoE not working on all ports

‎05-01-2019 11:31 AM

Facts so far:

  1. This switch stack has been in place for 6 years with multiple Junos updates requiring restarts. PoE has never been a problem
  2. We have had for several PoE security cameras and PoE wireless access points plugged into these switches for years. Again, no problem even after a stack reboot
  3. We installed a new IP phone system. The phones need PoE and have a 2 port gig switch built into the back of them
  4. Many of these phones are daisy chained with printers or desktop computers. Patch cable from wall to phone, patch cable from phone to desktop or printer. Some phones have nothing daisy chained with them
  5. We have PoE enabled on all ports on this switch stack
  6. Juniper 4200-48PX should be able to provide plenty of PoE power to the items plugged in
  7. We are running Junos 15.1R7.9 which is a JTac recommended version
  8. The first time this switch stack is booted after the phone installation, we cannot get PoE on several ports that have been working fine for years

I have been doing this type of work for 20+ years myself and the phone/PoE connection is not a coincidence. Is there some other troubleshooting I can do or should I open a ticket through our local Mitel reseller/integrator?

Ethernet Switching

Re: EX4200, PoE not working on all ports

‎05-01-2019 07:19 PM

Hi HendersonD,

 

A couple of things to check:

 

a) Are any of these interfaces physically up?

show interfaces terse ge-1/0/25

show interfaces terse ge-1/0/26

show interfaces terse ge-1/0/27

etc.

If there are any phones (especially ones with a switch) already up with external power, please try disabling POE on them and bounce the rest of the interfaces or toggle POE on them.

 

b) Please check the status messages from logs (check older file "messages.x.gz" if this has rolled over)

show log messages | grep poe | no-more

 

Note that some of these might be expected during the interface state change at reboot time, but helpful to check if anything special for the "OFF" ports.

 

Hope this helps.

 

Regards,
-r.

--------------------------------------------------

If this solves your problem, please mark this post as "Accepted Solution."
Kudos are always appreciated Smiley Happy.

Ethernet Switching

Re: EX4200, PoE not working on all ports

‎05-02-2019 04:31 AM

Here is the output of a few different commands

root@HS102B> show interfaces terse ge-1/0/25
Interface        Admin    Link      Proto         Local       Remote
ge-1/0/25     up           down
ge-1/0/25.0  up           down   eth-switch

 

root@HS102B> show log messages | match ge-1/0/25
May 1 22:19:35 HS102B chassisd[1332]: PoE port name ge-1/0/25 status 60
May 1 22:42:55 HS102B chassisd[1332]: PoE port name ge-1/0/25 status 60

 

The command above is interesting. This is one of the ports that is not putting out PoE. Below is the same command from a port that is putting out PoE

root@HS102B> show log messages | match ge-1/0/24
May 1 22:19:35 HS102B chassisd[1332]: PoE port name ge-1/0/24 status 1
May 1 22:42:55 HS102B chassisd[1332]: PoE port name ge-1/0/24 status 1
May 2 07:29:50 HS102B chassisd[1332]: PoE port name ge-1/0/24 status 1
May 2 07:30:04 HS102B chassisd[1332]: PoE port name ge-1/0/24 status 1

 

Not sure what status 1 (this interface is putting out PoE) versus status 60 (this interface is not putting out PoE) means

 

Ethernet Switching

Re: EX4200, PoE not working on all ports

‎05-02-2019 05:32 AM

We are thinking that our PoE problem is a hardware failure. We are opening a case with Juniper

Ethernet Switching

Re: EX4200, PoE not working on all ports

‎05-02-2019 06:32 AM

Hi HendersonD,

 

Status 60 tells us that ‘Calculated power > power limit’.  Can you also check these two on the switch?

show log messages | grep poe_get_psu
show configuration poe | display set

 

Hope this helps.

 

Regards,
-r.

--------------------------------------------------

If this solves your problem, please mark this post as "Accepted Solution."
Kudos are always appreciated Smiley Happy.

 

 

Ethernet Switching

Re: EX4200, PoE not working on all ports

‎05-02-2019 06:50 AM

@HendersonD, I know you don't really want to listen, but I think I hit the hammer on the head when I said the issue was calculated power.  I very much doubt this is a hardware issue, but something screwed up in software.  Now could be HW issue with POE Controller, but looks much more like software to me.

 

TAC needs to work this, but once again (and I know it might be painful) I suggest you load this VC with 12.3R12-S[latest] and see what you get for results.  I would think log file messages for any/all interfaces not providing power show "status 60", yes?

 

The above may be something TAC recommends, at minimum for troubleshooting data point, to start with.

Ethernet Switching

Re: EX4200, PoE not working on all ports

‎05-02-2019 10:42 AM

When I run this command I get nothing

show log messages | grep poe_get_psu

 

root@HS102B> show configuration poe | display set
set poe interface all

Ethernet Switching

Re: EX4200, PoE not working on all ports

‎05-02-2019 10:43 AM

Tonight I will load the 12.3 code and report back in the morning

Ethernet Switching

Re: EX4200, PoE not working on all ports

‎05-02-2019 05:10 PM

I upgraded this switch stack to Junos 12.3R12-S12 which is the JTac recommended version. It did not make any difference, I still have the same ports without PoE power. I have another edge closet with only one EX4200-PX switch. This switch has nearly all of its ports requiring PoE between phones, security cameras, and wireless access points. It had two ports where PoE was working and now is not working. I also installed the Junos 12.3 code on this switch and it made no difference.

 

Several months back we had an EX4200 with PoE problems. We would boot the switch, all ports went active with blinking lights everywhere. We would watch as ports would start failing one after the other with lights going out.

Ethernet Switching

Re: EX4200, PoE not working on all ports

‎05-02-2019 05:25 PM

When I first started troubleshooting this problem, I went to the command line and issued a show poe controller command. This is the first time I had ever used this command since we have never had PoE problems before. The message I received was the PoE firmware needed upgrading. I was not even aware that upgrading PoE firmware was a separate process from upgrading Junos. We applied the upgrade to PoE to all of our switches. I do not know what PoE firmware version that was installed, but it is now version 4.1.6. Could the PoE firmware upgrade be the root cause of this issue?