Screen OS

Hi All,

I have three Netscreen-25 firewalls on my LAN. Two are configured as a NSRP pair/cluster and the thrid is a standalone firewall. All firewalls were running ScreenOS 5.4.0r27.0 (Firewall+VPN) and they work/worked perfectly, though a few weeks ago I noticed that all the netscreen firewalls were logging critical errors:

One FW shows this - logged every 30 seconds

crit - arp req detected an IP conflict (IP 10.2.26.242, MAC 88f0310dba31) on interface ethernet1

Other FW shows this - logged every 30 seconds

arp req detected an IP conflict (IP 10.30.235.242, MAC 88f0310dba31) on interface ethernet2

Both show the same MAC.

Now I don't appear to have any problems with network serivces, but the these log entries are causing concern.

I have a 100% switched cisco network. I was able to track the MAC address down to a new Cisco C3650 48 port switch which i recently installed. As soon as I disconnect the switch, the critical alerts stop. As soon as I plug the C3650 switch back into the network the alerts start coming in. I have not configured this new C3650 in any special way, I have configured it in the same as all my other Cisco switches. If I plug a Cisco 3560, or 2960 (basically any other cisco switch i got) I do not get the alerts on the Netscreen FW's.

I have upgraded the software on my cisco switch to the latest version (IOS XE 03.03.04SE) and have upgraded one of my Netscreen firewalls to ScreenOS 5.4.0r28a.0 (Firewall+VPN) - the latest version. But still the critical "arp req detected an IP conflict" alerts are coming in every 30 seconds.

It's got to be something to do with the new Cisco 3650 - though I don't know what it could be. On the networking side of things everything seems to be working OK.

Please can anybody advise as what the problem might be?

Thanks in advance.

Sounds like the switch might be looping the ARP requests.

I would guess you have a layer 2 loop in the network causing the issue.  Do you have RSTP enabled on the switches and have you checked for loops ans spanning tree port status?

Hi All,

Firstly thanks for your replies. I have RSTP enabled on all my switches. These new Cisco C3650 series switches are connected to the exsiting switches (in a fibre ring) using a SFP modules /fibre patch leads.

In the current setup I cannot see how there could be a layer 2 loop because the 3650 is connected via a single physical link, whether that be using a SFP module/fibre patch lead or a single gigabit ethernet port directly connected using a cat5e patch lead into another gigabit ethernet port. So in both cases only 1 link/path exists.

On the netscreen-25 the critical error reports the MAC address of the connected/trunk link port on the Cisco 3650:

"arp req detected an IP conflict (IP 10.2.26.242, MAC 88f0310df431) on interface ethernet1"

On the cisco this is the:

xxxxxxx-hh1-cat15#sh interfaces gigabitEthernet 1/1/1
GigabitEthernet1/1/1 is up, line protocol is up (connected)
  Hardware is Gigabit Ethernet, address is 88f0.310d.f431 (bia 88f0.310d.f431)
  MTU 1500 bytes, BW 1000000 Kbit/sec, DLY 10 usec,
     reliability 255/255, txload 1/255, rxload 1/255
  Encapsulation ARPA, loopback not set
  Keepalive not set
  Full-duplex, 1000Mb/s, link type is auto, media type is 1000BaseSX SFP
  input flow-control is off, output flow-control is unsupported
  ARP type: ARPA, ARP Timeout 04:00:00
  Last input 00:00:00, output never, output hang never
  Last clearing of "show interface" counters never
  Input queue: 0/2000/0/0 (size/max/drops/flushes); Total output drops: 0
  Queueing strategy: fifo
  Output queue: 0/40 (size/max)
  5 minute input rate 16000 bits/sec, 25 packets/sec
  5 minute output rate 6000 bits/sec, 9 packets/sec
     350837 packets input, 29807313 bytes, 0 no buffer
     Received 234182 broadcasts (156724 multicasts)
     0 runts, 0 giants, 0 throttles
     0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored
     0 watchdog, 156724 multicast, 0 pause input
     0 input packets with dribble condition detected
     119555 packets output, 9923683 bytes, 0 underruns
     0 output errors, 0 collisions, 1 interface resets
     12154 unknown protocol drops
     0 babbles, 0 late collision, 0 deferred
     0 lost carrier, 0 no carrier, 0 pause output
     0 output buffer failures, 0 output buffers swapped out
xxxxxx-hh1-cat15#

And second Cisco 3650 also triggers a similar alert:

on the Netscreen-25

"arp req detected an IP conflict (IP 10.2.26.242, MAC 88f0310dba31) on interface ethernet1"

On the Cisco 3650:

xxxxx-hh1-cat14#sh interfaces gigabitEthernet 1/1/1
GigabitEthernet1/1/1 is up, line protocol is up (connected)
  Hardware is Gigabit Ethernet, address is 88f0.310d.ba31 (bia 88f0.310d.ba31)
  MTU 1500 bytes, BW 1000000 Kbit/sec, DLY 10 usec,
     reliability 255/255, txload 1/255, rxload 1/255
  Encapsulation ARPA, loopback not set
  Keepalive not set
  Full-duplex, 1000Mb/s, link type is auto, media type is 1000BaseSX SFP
  input flow-control is off, output flow-control is unsupported
  ARP type: ARPA, ARP Timeout 04:00:00
  Last input 00:00:00, output never, output hang never
  Last clearing of "show interface" counters never
  Input queue: 0/2000/0/0 (size/max/drops/flushes); Total output drops: 0
  Queueing strategy: fifo
  Output queue: 0/40 (size/max)
  5 minute input rate 1596000 bits/sec, 156 packets/sec
  5 minute output rate 83000 bits/sec, 77 packets/sec
     5236243 packets input, 4667733334 bytes, 0 no buffer
     Received 1400163 broadcasts (930724 multicasts)
     0 runts, 0 giants, 0 throttles
     0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored
     0 watchdog, 930724 multicast, 0 pause input
     0 input packets with dribble condition detected
     2353505 packets output, 204910425 bytes, 0 underruns
     0 output errors, 0 collisions, 1 interface resets
     75948 unknown protocol drops
     0 babbles, 0 late collision, 0 deferred
     0 lost carrier, 0 no carrier, 0 pause output
     0 output buffer failures, 0 output buffers swapped out
xxxxxx-hh1-cat14#

As per above if I change the uplink port on the Cisco 3650, all that happens is the MAC address reported on the Netscreen changes to show the MAC of the new physically connected port.

If I connect the switches redundantly, the STP recalculates and as expected some ports go into the BLK states. But in the end the Netscreen will still report the MAC addresses of the active/FWD'ing trunk link ports. As I have two Cisco 3650's I get alerts for two MAC addresses.

I must stress that if I replace any of the new Cisco 3650 with the older Cisco 3560, 3560v2, 2960 series switches (connected in exactly the same way) I do NOT get any alerts. I only get alerts when i plug in the Cisco C3650.

So something definitely to do with new switches, but I can't see what it can be?

If i can provide anymore info that you need please let me know..

Regards

Hello.

One question -- where is 10.2.26.242 configured?  Is it a vlan interface on the cisco switches?

Do you have both switches with the same vlanXXX interface ip addresses?

Regards,

Sam

Hi Sam,

The ip 10.2.26.242 is configured ethernet1 on one of my Netscreen-25 FW's - This FW is a standalone, not in a cluster.

I have another Netscreen-25 NSRP cluster with 2  Firewalls in it, this NSRP pair also logs the critical alerts:

There is no way the IP's are hardcoded anywhere else. As mentioned earlier, if shutdown/disconnect the Cisco 3650 the alerts stop.

The two FW sets above (1 standalone and 2 in a NSRP) have 1 ethernet port plugged into my switches.

If I swap the 3650, with a 3560, 3560v2 or 2960, I get no alerts.

All my switches have IP's from the same vlan, there is no conflict/duplication there either.

Regards

Was a root cause found for this?  We are seeing a similar issue with SSG5s and catalyst 4507 switches.  The juniper firewall is erring with dupIP, but the mac address reported is of the 4507's mgmt interface that is on a different VLAN.  Also, a data capture shows no conflict, nor are any error reported by the router. 

A reponse on the cisco forums lead me to the solution. The issue seems to be an erronous error caused by ARP probes generated by the switch. 

http://www.cisco.com/c/en/us/support/docs/ios-nx-os-software/8021x/116529-problemsolution-product-00.html

Thanks for your post - I'm glad i checked back as I thought no ones going to come back with any further replies.

Adjusting the value for probe delay from 10 - 120 didin't make any difference

ip device tracking probe delay 10

So in my case I had to adjust the value for the "probe interval"  from the default of 30 to 60

ip device tracking probe interval 60

And the alerts have now stopped,