Routing
Highlighted
Routing

pings failing and routing failing

‎11-14-2016 11:52 AM

Hello, we have a EX3200-24T switch with three VLAN management IP addresses.

It has suddenly started failing to respond to pings on one of it's VLAN management IPs (10.21.200.19). However the switch can ping itself internally to that address via the CLI so the interface was obviously up and responding.

Additionally, one attached client could not route any where on the network. Additionally, occassionally connection to one of the other VLAN management IP addresses fails.

Finally, yesterday, I attempted to SSH and GUI in to this switch on the management IP that was working (10.21.1.15) and suddenly all connected devices lost network connectivity. We had to shut the switch down and remove it from production.

Nothing has changed on this switch or it it's neighboring devices. It just suddenly seems to have gone faulty.

 

What steps should I take now to assess where the problem e.g. run some diagnostics tests somehow ?

Thank you for any advice.

 

Here is the config:

 

version 9.2R1.10;
system {
host-name ENG-Rack16;
time-zone Pacific/Auckland;
root-authentication {
encrypted-password "$1$IaYFi2Hv$kIKeMUn49YPJjqMv8xO6J."; ## SECRET-DATA
}
login {
message "***********************************************************\nUNAUTHORIZED ACCESS TO THIS DEVICE IS STRICTLY PROHIBITED.\nYou must have explicit permission to access or configure\nthis device. All activities performed on this device may\nbe logged, and violations of this policy may result in\ndisciplinary action and may be reported to law enforcement.\nThere is no right to privacy on this device.\nAttempted access and/or use of this system shall constitute\nconsent to monitoring.\nDISCONNECT IMMEDIATELY IF YOU DO NOT HAVE EXPLICIT\nPERMISSION TO ACCESS THIS DEVICE.\n***********************************************************\n";
user admin {
uid 2000;
class super-user;
authentication {
encrypted-password "$1$wP7COyMj$4SHR4EweMcnIH8riDRv.c."; ## SECRET-DATA
}
}
user myUserName {
uid 2001;
class super-user;
authentication {
encrypted-password "$1$pDukUzxF$NdSu1o9GPPiB/cTqLvVSx."; ## SECRET-DATA
}
}
user support {
uid 2002;
class read-only;
authentication {
encrypted-password "$1$tHFGaBnr$VbgRZvIKH/gegPuRI.GyA."; ## SECRET-DATA
}
}
}
services {
ssh {
protocol-version v2;
}
telnet;
netconf {
ssh;
}
web-management {
http;
}
}
syslog {
user * {
any emergency;
}
host 10.21.250.23 {
any error;
authorization info;
source-address 10.21.200.19;
}
}
ntp {
server 10.14.254.116;
server 10.14.254.117;
}
}
interfaces {
ge-0/0/0 {
unit 0 {
family ethernet-switching;
}
}
ge-0/0/1 {
unit 0 {
family ethernet-switching {
port-mode access;
vlan {
members IT_Subnet;
}
}
}
}
ge-0/0/2 {
unit 0 {
family ethernet-switching;
}
}
ge-0/0/3 {
unit 0 {
family ethernet-switching;
}
}
ge-0/0/4 {
unit 0 {
family ethernet-switching;
}
}
ge-0/0/5 {
unit 0 {
family ethernet-switching;
}
}
ge-0/0/6 {
unit 0 {
family ethernet-switching;
}
}
ge-0/0/7 {
unit 0 {
family ethernet-switching;
}
}
ge-0/0/8 {
unit 0 {
family ethernet-switching;
}
}
ge-0/0/9 {
unit 0 {
family ethernet-switching;
}
}
ge-0/0/10 {
unit 0 {
family ethernet-switching;
}
}
ge-0/0/11 {
unit 0 {
family ethernet-switching;
}
}
ge-0/0/12 {
unit 0 {
family ethernet-switching;
}
}
ge-0/0/13 {
unit 0 {
family ethernet-switching;
}
}
ge-0/0/14 {
unit 0 {
family ethernet-switching;
}
}
ge-0/0/15 {
unit 0 {
family ethernet-switching;
}
}
ge-0/0/16 {
unit 0 {
family ethernet-switching;
}
}
ge-0/0/17 {
unit 0 {
family ethernet-switching;
}
}
ge-0/0/18 {
unit 0 {
family ethernet-switching;
}
}
ge-0/0/19 {
unit 0 {
family ethernet-switching;
}
}
ge-0/0/20 {
unit 0 {
family ethernet-switching;
}
}
ge-0/0/21 {
unit 0 {
family ethernet-switching;
}
}
ge-0/0/22 {
unit 0 {
family ethernet-switching;
}
}
ge-0/0/23 {
unit 0 {
family ethernet-switching;
}
}
ge-0/1/0 {
unit 0 {
family ethernet-switching;
}
}
xe-0/1/0 {
unit 0 {
family ethernet-switching;
}
}
ge-0/1/1 {
unit 0 {
family ethernet-switching;
}
}
xe-0/1/1 {
unit 0 {
family ethernet-switching;
}
}
ge-0/1/2 {
unit 0 {
family ethernet-switching;
}
}
ge-0/1/3 {
unit 0 {
family ethernet-switching;
}
}
vlan {
unit 2 {
family inet {
address 10.21.1.15/24;
}
}
unit 4 {
family inet {
address 10.21.199.5/24;
}
}
unit 5 {
family inet {
address 10.21.200.19/24;
}
}
}
}
snmp {
community precision {
authorization read-only;
}
}
routing-options {
static {
route 0.0.0.0/0 next-hop 10.21.200.9;
route 10.21.199.0/24 next-hop 10.21.199.1;
}
}
protocols {
ospf {
disable;
area 0.0.0.0 {
interface all;
}
}
lldp {
interface all;
}
rstp;
}
ethernet-switching-options {
voip;
}
vlans {
IT_Subnet {
description "Auckland IT Subnet";
vlan-id 10;
interface {
ge-0/0/18.0;
ge-0/0/19.0;
ge-0/0/20.0;
ge-0/0/21.0;
ge-0/0/22.0;
ge-0/0/23.0;
ge-0/0/10.0;
ge-0/0/11.0;
}
l3-interface vlan.2;
}
MaiOffice {
vlan-id 30;
l3-interface vlan.4;
}
default {
description "Engineering Subnet";
vlan-id 31;
l3-interface vlan.5;
}
}
poe {
interface all;
}

 

 

5 REPLIES 5
Highlighted
Routing

Re: pings failing and routing failing

‎11-14-2016 01:28 PM

Hi,

 

Did you try to ping from the switch to other hosts.

Is the switch connected to other switched and running STP?

 

The evident suspect that first comes to mind is some layer2 storm happening on the network. You could check traffic levels on the ports at the time of incident if snmp stats data is available.

 

Cheers,

Ashvin

Highlighted
Routing

Re: pings failing and routing failing

‎11-15-2016 06:11 PM

Hello, yes the switch was running STP and connected to other STP switches.

It could not ping out to other devices from that particular VLAN interface but it could on other VLANs.

 

I noticed the logs had a lot of unusual entries as such:

Eng-Core chassisd[1069]: fpm_show_ex22x33_lcd_led_per_slot: port23 Duplex bit is set along with speed

 

Would these logs indicate anything specific ?

Highlighted
Routing

Re: pings failing and routing failing

‎01-12-2017 02:29 AM

Hi fran1942,

we're having a similar issue on a EX3300 48P with the same error message in the logs. Did you find out anything useful?

 

Thank you!

Highlighted
Routing

Re: pings failing and routing failing

‎01-12-2017 08:53 AM

Hi Folks,
Regarding your question, the facility "DAEMON-3" is of informational type and it has no impact on your EXBOX or network. The log " fpm_show_ex22x33_lcd_led_per_slot: port30 Duplex bit is set along with speed" appears when LCD information is extracted and it regards the bit set and speed on the physical ports.

 

If you want to exclude these notes from being written to the /var/log/messages file, you can do the following:
#set system syslog file messages match "!(fpm_show_ex22x33_lcd_led_per_slot)"

 

-Python JNCIE 3X [SP|DC|ENT] JNCIP-SEC JNCDS 3X [ WAN | DC|SEC] JNCIS-Cloud JNCIS-DevOps CCIP ITIL
#Please mark my solution as accepted if it helped, Kudos are appreciated as well.
Highlighted
Routing

Re: pings failing and routing failing

‎01-12-2017 08:56 AM

Thank you! I found out after hours of strunggling that it seems to appear only when the web GUI is open, so it has nothing to do with all the rest.

Back to the drawing board now 😞

Feedback