SRX

Hi I am having a problem with the out-of-band management (fxp0)
It is connected to an access port on EX4300, from the EX the Management subnet is tagged onwards through the network(MPLS).
I see in the arp table on the SRX240:

00:10:db:ff:21:d0 172.24.0.1 172.24.0.1 fxp0.0 none
ec:3e:f7:11:9a:61 172.24.9.10 172.24.9.10 fxp0.0 none
54:1e:56:e5:56:40 172.24.9.113 172.24.9.113 fxp0.0 none
54:4b:8c:5f:98:37 172.24.9.164 172.24.9.164 fxp0.0 none

The default gw is 172.24.0.1/18. I am unable to ping the GW from Node0 which is the active SRX, From Node1 (standby), I can ping the GW. Also from the EX4300 I can ping the default GW 172.24.0.1
Also I am unable to access the SRX remotely through ssh. After I login to the EX4300 and do a couple of pings towards the SRX cluster, I can login again. The SRX cluster is managed by Junos Space. The strange thing is that the SRXs are shown online in Space always.

Any help would be greatly appreciated:

This is the arp table on the EX4300:
00:10:db:ff:21:d0 172.24.0.1 172.24.0.1 irb.906 [ae1.0] none
54:1e:56:e6:64:40 172.24.9.110 172.24.9.110 irb.906 [ae1.0] none
54:1e:56:e6:4a:c0 172.24.9.112 172.24.9.112 irb.906 [ge-2/0/0.0] none
54:1e:56:e5:56:40 172.24.9.113 172.24.9.113 irb.906 [ge-3/0/0.0] none
54:4b:8c:5f:98:37 172.24.9.164 172.24.9.164 irb.906 [ge-2/0/16.0] none
54:4b:8c:61:34:37 172.24.9.165 172.24.9.165 irb.906 [ge-3/0/16.0] none
00:10:db:ff:21:d0 172.24.15.254 172.24.15.254 irb.906 [ae1.0] none

172.24.9.112 is SRX240 node0
172.24.9.113 is SRX240 node1

The EX4300 is configured with an irb interface where its IP resides:

set interfaces irb unit 906 family inet address 172.24.9.10/18
set vlans MGT description Management
set vlans MGT vlan-id 906
set vlans MGT l3-interface irb.906

# Interface towards MX960 (MPLS node)

set interfaces xe-0/0/35 ether-options 802.3ad ae1
set interfaces xe-1/0/35 ether-options 802.3ad ae1
set interfaces ae1 description "TO-SWT-MX960-SNC-LAG-20G ae1"
set interfaces ae1 mtu 9192
set interfaces ae1 aggregated-ether-options lacp active
set interfaces ae1 unit 0 family ethernet-switching interface-mode trunk
set interfaces ae1 unit 0 family ethernet-switching vlan members MGT
set interfaces ae1 unit 0 family ethernet-switching vlan members EPC-DATAC

# Interface towards SRX240 cluster

set interfaces ge-2/0/0 description "OOB MNGT - to SRX240-SNC-CLST-NODE0 ge-0/0/0"
set interfaces ge-2/0/0 unit 0 family ethernet-switching interface-mode access
set interfaces ge-2/0/0 unit 0 family ethernet-switching vlan members MGT
set interfaces ge-3/0/0 description "OOB MNGT - to SRX240-SNC-CLST-NODE1 ge-5/0/0"
set interfaces ge-3/0/0 unit 0 family ethernet-switching interface-mode access
set interfaces ge-3/0/0 unit 0 family ethernet-switching vlan members MGT

### SRX240 cluster config:

set groups node0 system host-name SRX240-SNC-CLUSTER-NODE-0
set groups node0 system backup-router 172.24.0.1
set groups node0 system backup-router destination 0.0.0.0/0
set groups node0 system services
set groups node0 interfaces fxp0 unit 0 description "OOB Management"
set groups node0 interfaces fxp0 unit 0 family inet address 172.24.9.112/18
set groups node1 system host-name SRX240-SNC-CLUSTER-NODE-1
set groups node1 system backup-router 172.24.0.1
set groups node1 system backup-router destination 0.0.0.0/0
set groups node1 system services
set groups node1 interfaces fxp0 unit 0 description "OOB Management"
set groups node1 interfaces fxp0 unit 0 family inet address 172.24.9.113/18
set apply-groups "${node}"

set routing-options static route 0.0.0.0/0 next-hop 172.24.0.1

### MX960 config:

set interfaces xe-11/0/0 gigether-options 802.3ad ae1
set interfaces xe-11/1/0 gigether-options 802.3ad ae1
set interfaces ae1 description "TO-SWT-EX4300-SNC-LAG-20G ae1"
set interfaces ae1 vlan-tagging
set interfaces ae1 mtu 9192
set interfaces ae1 encapsulation flexible-ethernet-services
set interfaces ae1 aggregated-ether-options lacp active
set interfaces ae1 unit 906 description "CPE Management for SPACE"
set interfaces ae1 unit 906 encapsulation vlan-vpls
set interfaces ae1 unit 906 vlan-id 906
set interfaces ae1 unit 906 family vpls

can you paste the output of  "show chassis cluster status"

{primary:node0}
Test@SRX240-SNC-CLUSTER-NODE-0> show chassis cluster status
Monitor Failure codes:
CS Cold Sync monitoring FL Fabric Connection monitoring
GR GRES monitoring HW Hardware monitoring
IF Interface monitoring IP IP monitoring
LB Loopback monitoring MB Mbuf monitoring
NH Nexthop monitoring NP NPC monitoring
SP SPU monitoring SM Schedule monitoring

Cluster ID: 1
Node Priority Status Preempt Manual Monitor-failures

Redundancy group: 0 , Failover count: 1
node0 100 primary no no None
node1 1 secondary no no None

Redundancy group: 1 , Failover count: 7
node0 100 primary yes no None
node1 1 secondary yes no None

Hi,

Can you provide the routing and forwarding table inet.0 for the SRX:

show route table inet.0
show route forwarding-table family inet

Cheers,

Ashvin

FYI

inet.0: 3 destinations, 3 routes (3 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

0.0.0.0/0 *[Static/5] 6d 23:27:53
> to 172.24.0.1 via fxp0.0
172.24.0.0/18 *[Direct/0] 2w5d 02:38:33
> via fxp0.0
172.24.9.112/32 *[Local/0] 2w5d 02:38:33
Local via fxp0.0

Routing table: default.inet
Internet:
Destination Type RtRef Next hop Type Index NhRef Netif
default user 0 0:10:db:ff:21:d0 ucst 324 4 fxp0.0
default perm 0 rjct 36 1
0.0.0.0/32 perm 0 dscd 34 1
172.24.0.0/18 intf 0 rslv 323 1 fxp0.0
172.24.0.0/32 dest 0 172.24.0.0 recv 321 1 fxp0.0
172.24.0.1/32 dest 1 0:10:db:ff:21:d0 ucst 324 4 fxp0.0
172.24.9.10/32 dest 0 ec:3e:f7:11:9a:61 ucst 333 1 fxp0.0
172.24.9.112/32 intf 0 172.24.9.112 locl 322 2
172.24.9.112/32 dest 0 172.24.9.112 locl 322 2
172.24.9.113/32 dest 0 54:1e:56:e5:56:40 ucst 335 1 fxp0.0
172.24.9.164/32 dest 0 54:4b:8c:5f:98:37 ucst 334 1 fxp0.0
172.24.63.255/32 dest 0 172.24.63.255 bcst 320 1 fxp0.0
224.0.0.0/4 perm 0 mdsc 35 1
224.0.0.1/32 perm 0 224.0.0.1 mcst 31 1
255.255.255.255/32 perm 0 bcst 32 1

Routing table: __master.anon__.inet
Internet:
Destination Type RtRef Next hop Type Index NhRef Netif
default perm 0 rjct 1291 1
0.0.0.0/32 perm 0 dscd 1289 1
224.0.0.0/4 perm 0 mdsc 1290 1
224.0.0.1/32 perm 0 224.0.0.1 mcst 1286 1
255.255.255.255/32 perm 0 bcst 1287 1

Hi Ashvin,

do you have any idea? Or should I give up on OOB management?

Best Regards

Edson

Hi, 

The forwarding-table looks ok and arp for the gateway is present too.

Only difference between node 0 & node 1, is that the passive node does not have a routing/forwarding table and uses the backup-router as gateway.

Did you try monitoring traffic on the interfaces to confirm if the packets are seen in and out on either end.

Also, I assume the mac addresses are present on the EX4300 after arp.

Cheers,

Ashvin

Hi Ashvin,

Idid this and I get the following:

setnoc@SRX240-SNC-CLUSTER-NODE-0> monitor traffic interface fxp0 matching icmp
verbose output suppressed, use <detail> or <extensive> for full protocol decode
Address resolution is ON. Use <no-resolve> to avoid any reverse lookup delay.
Address resolution timeout is 4s.
Listening on fxp0, capture size 96 bytes

Reverse lookup for 172.24.0.1 failed (check DNS reachability).
Other reverse lookup failures will not be reported.
Use <no-resolve> to avoid reverse lookups on IP addresses.

07:39:12.887303 In IP truncated-ip - 68 bytes missing! 172.24.0.1 > 172.24.9.112: ICMP echo request, id 1024, seq 54500, length 108
07:39:12.887400 Out IP truncated-ip - 68 bytes missing! 172.24.9.112 > 172.24.0.1: ICMP echo reply, id 1024, seq 54500, length 108
07:39:13.777326 In IP truncated-ip - 68 bytes missing! 172.24.0.1 > 172.24.9.112: ICMP echo request, id 1024, seq 54600, length 108
07:39:13.777439 Out IP truncated-ip - 68 bytes missing! 172.24.9.112 > 172.24.0.1: ICMP echo reply, id 1024, seq 54600, length 108
07:39:14.777392 In IP truncated-ip - 68 bytes missing! 172.24.0.1 > 172.24.9.112: ICMP echo request, id 1024, seq 54700, length 108
07:39:14.777586 Out IP truncated-ip - 68 bytes missing! 172.24.9.112 > 172.24.0.1: ICMP echo reply, id 1024, seq 54700, length 108
07:39:15.777371 In IP truncated-ip - 68 bytes missing! 172.24.0.1 > 172.24.9.112: ICMP echo request, id 1024, seq 54800, length 108
07:39:15.777491 Out IP truncated-ip - 68 bytes missing! 172.24.9.112 > 172.24.0.1: ICMP echo reply, id 1024, seq 54800, length 108
07:39:16.777393 In IP truncated-ip - 68 bytes missing! 172.24.0.1 > 172.24.9.112: ICMP echo request, id 1024, seq 54900, length 108
07:39:16.777504 Out IP truncated-ip - 68 bytes missing! 172.24.9.112 > 172.24.0.1: ICMP echo reply, id 1024, seq 54900, length 108

So the icmp ariives and it responds. The arp is present on the EX4300.

So maybe the EX4300 IRB interface is blocking it somehow?

setnoc@SWT-SNC-EPC-4300> show arp
MAC Address Address Name Interface Flags
02:00:00:00:00:0a 128.0.0.5 128.0.0.5 bme0.0 permanent
02:00:00:00:00:0a 128.0.0.6 128.0.0.6 bme0.0 permanent
02:00:00:00:00:0a 128.0.0.16 128.0.0.16 bme0.0 permanent
02:00:00:00:00:0c 128.0.0.18 128.0.0.18 bme0.0 permanent
02:00:00:00:00:0d 128.0.0.19 128.0.0.19 bme0.0 permanent
00:10:db:ff:21:d0 172.24.0.1 172.24.0.1 irb.906 [ae1.0] none
54:1e:56:e6:64:40 172.24.9.110 172.24.9.110 irb.906 [ae1.0] none
54:1e:56:e6:4a:c0 172.24.9.112 172.24.9.112 irb.906 [ge-2/0/0.0] none
54:1e:56:e5:56:40 172.24.9.113 172.24.9.113 irb.906 [ge-3/0/0.0] none
54:4b:8c:5f:98:37 172.24.9.164 172.24.9.164 irb.906 [ge-2/0/16.0] none
54:4b:8c:61:34:37 172.24.9.165 172.24.9.165 irb.906 [ge-3/0/16.0] none
00:10:db:ff:21:d0 172.24.15.254 172.24.15.254 irb.906 [ae1.0] none

destination 0.0.0.0/0 on backup-router config is not supported. Can you remove 0/0 and add specific subnets and test the behavior.

Ref: https://kb.juniper.net/InfoCenter/index?page=content&id=KB15580&actp=search

Hi Rsuraj,

Thanx for your reply. I did what you suggested, but this but did not help:

set groups node0 system host-name SRX240-SNC-CLUSTER-NODE-0
set groups node0 system backup-router 172.24.0.1
set groups node0 system backup-router destination 172.24.0.0/18
set groups node0 system services ssh
set groups node0 system services netconf ssh
set groups node0 interfaces fxp0 unit 0 description "OOB Management"
set groups node0 interfaces fxp0 unit 0 family inet address 172.24.9.112/18
set groups node1 system host-name SRX240-SNC-CLUSTER-NODE-1
set groups node1 system backup-router 172.24.0.1
set groups node1 system backup-router destination 172.24.0.0/18
set groups node1 system services
set groups node1 system ntp
set groups node1 interfaces fxp0 unit 0 description "OOB Management"
set groups node1 interfaces fxp0 unit 0 family inet address 172.24.9.113/18

Guys,

After coincidentally needing to reboot the MX-960 MPLS node, the issue is fixed.

Now I can ping the SRX clusters connected on the EX4300 virtual chassis.

So probably some bug on the MX JUNOS 13.3R6.5 built 2015-03-26