SRX

Hi Guys,

I am trying to setup a pair of SRX 240 chassis cluster using LACP like the setup below (this diagram I borrowed from Juniper web site)

set chassis aggregated-devices ethernet device-count 2
set interfaces interface-range LAN1 member-range ge-0/0/6 to ge-0/0/11
set interfaces interface-range LAN1 unit 0 family ethernet-switching vlan members LAN1
...

set interfaces ge-0/0/24 ether-options 802.3ad ae1
set interfaces ge-0/0/24 unit 0 family ethernet-switching port-mode trunk
set interfaces ge-0/0/24 unit 0 family ethernet-switching vlan members 205-206
set interfaces ge-0/0/24 unit 0 family ethernet-switching vlan members 90
deactivate interfaces ge-0/0/24 unit 0  

set interfaces ge-0/0/26 ether-options 802.3ad ae1
set interfaces ge-0/0/26 unit 0 family ethernet-switching port-mode trunk
set interfaces ge-0/0/26 unit 0 family ethernet-switching vlan members 205-206
set interfaces ge-0/0/26 unit 0 family ethernet-switching vlan members 90
deactivate interfaces ge-0/0/26 unit 0

set interfaces ge-0/0/28 ether-options 802.3ad ae2
set interfaces ge-0/0/28 unit 0 family ethernet-switching port-mode trunk
set interfaces ge-0/0/28 unit 0 family ethernet-switching vlan members 205-206
set interfaces ge-0/0/28 unit 0 family ethernet-switching vlan members 90
deactivate interfaces ge-0/0/28 unit 0

set interfaces ge-0/0/30 ether-options 802.3ad ae2
set interfaces ge-0/0/30 unit 0 family ethernet-switching port-mode trunk
set interfaces ge-0/0/30 unit 0 family ethernet-switching vlan members 205-206
set interfaces ge-0/0/30 unit 0 family ethernet-switching vlan members 90
deactivate interfaces ge-0/0/30 unit 0

set interfaces ae1 aggregated-ether-options lacp active
set interfaces ae2 aggregated-ether-options lacp active

set interfaces vlan unit 90 family inet address 172.16.1.90/24
set routing-options static route 0.0.0.0/0 next-hop 172.16.1.1
set protocols igmp-snooping vlan all
set protocols rstp
set protocols lldp interface all
set protocols lldp-med interface all
set ethernet-switching-options storm-control interface all
set vlans LAN1 vlan-id 200
set vlans MGMT vlan-id 90
set vlans MGMT interface ge-0/0/47.0
set vlans default l3-interface vlan.0
set poe interface all

 Configuration on SRX240:

set groups node0 interfaces fxp0 unit 0 family inet address 192.168.100.1/24
set groups node1 system host-name SRX2
set groups node1 interfaces fxp0 unit 0 family inet address 192.168.100.2/24
set apply-groups "${node}"
set chassis cluster reth-count 1
set chassis cluster redundancy-group 0 node 0 priority 100
set chassis cluster redundancy-group 0 node 1 priority 1
set chassis cluster redundancy-group 1 node 0 priority 100
set chassis cluster redundancy-group 1 node 1 priority 1
set chassis cluster redundancy-group 1 interface-monitor ge-0/0/5 weight 255
set chassis cluster redundancy-group 1 interface-monitor ge-5/0/4 weight 255
set chassis cluster redundancy-group 1 interface-monitor ge-5/0/5 weight 255
set interfaces ge-0/0/4 gigether-options redundant-parent reth0
set interfaces ge-0/0/5 gigether-options redundant-parent reth0
set interfaces ge-5/0/4 gigether-options redundant-parent reth0
set interfaces ge-5/0/5 gigether-options redundant-parent reth0
set interfaces fab0 fabric-options member-interfaces ge-0/0/2
set interfaces fab1 fabric-options member-interfaces ge-5/0/2
set interfaces reth0 vlan-tagging
set interfaces reth0 redundant-ether-options redundancy-group 1
set interfaces reth0 redundant-ether-options lacp passive
set interfaces reth0 redundant-ether-options lacp periodic slow
set interfaces reth0 unit 90 vlan-id 90
set interfaces reth0 unit 90 family inet address 172.16.1.1/24
set security zones security-zone trust interfaces reth0.90

After the configuration, reth0 interface is not coming up.

root@SRX1> show interfaces terse 
Interface               Admin Link Proto    Local                 Remote
ge-0/0/0                up    up  
gr-0/0/0                up    up  
ip-0/0/0                up    up  
ge-0/0/1                up    up  
ge-0/0/2                up    up  
ge-0/0/2.0              up    up   aenet    --> fab0.0
ge-0/0/3                up    down
ge-0/0/4                up    up  
ge-0/0/4.90            up    up   aenet    --> reth0.90
ge-0/0/4.32767          up    up   aenet    --> reth0.32767
ge-0/0/5                up    up  
ge-0/0/5.90            up    up   aenet    --> reth0.90
ge-0/0/5.32767          up    up   aenet    --> reth0.32767
ge-0/0/6                up    down
ge-0/0/7                up    down
ge-0/0/8                up    down
ge-0/0/9                up    down
ge-0/0/10               up    down
ge-0/0/11               up    down
ge-0/0/12               up    down
ge-0/0/13               up    down
ge-0/0/14               up    down
ge-0/0/15               up    down      
ge-5/0/0                up    up  
ge-5/0/1                up    up  
ge-5/0/2                up    up  
ge-5/0/2.0              up    up   aenet    --> fab1.0
ge-5/0/3                up    down
ge-5/0/4                up    up  
ge-5/0/4.90            up    up   aenet    --> reth0.90
ge-5/0/4.32767          up    up   aenet    --> reth0.32767
ge-5/0/5                up    up  
ge-5/0/5.90            up    up   aenet    --> reth0.90
ge-5/0/5.32767          up    up   aenet    --> reth0.32767
ge-5/0/6                up    down
ge-5/0/7                up    down
ge-5/0/8                up    down
ge-5/0/9                up    down
ge-5/0/10               up    down
ge-5/0/11               up    down
ge-5/0/12               up    down
ge-5/0/13               up    down
ge-5/0/14               up    down
ge-5/0/15               up    down
fab0                    up    up  
fab0.0                  up    up   inet     30.17.0.200/24  
fab1                    up    up  
fab1.0                  up    up   inet     30.18.0.200/24  
fxp0                    up    up  
fxp0.0                  up    up   inet     192.168.100.1/24
fxp1                    up    up  
fxp1.0                  up    up   inet     129.16.0.1/2    
                                   tnp      0x1100001       
fxp2                    up    up  
fxp2.0                  up    up   tnp      0x1100001       
gre                     up    up  
ipip                    up    up  
irb                     up    up  
lo0                     up    up  
lo0.16384               up    up   inet     127.0.0.1           --> 0/0
lo0.16385               up    up   inet     10.0.0.1            --> 0/0
                                            10.0.0.16           --> 0/0
                                            128.0.0.1           --> 0/0
                                            128.0.0.4           --> 0/0
                                            128.0.1.16          --> 0/0
lo0.32768               up    up  
lsi                     up    up  
mtun                    up    up  
pimd                    up    up        
pime                    up    up  
pp0                     up    up  
ppd0                    up    up  
ppe0                    up    up  
reth0                   up    down
reth0.90               up    down inet     172.16.1.1/24  
reth0.32767             up    down
st0                     up    up  
swfab0                  up    down
swfab1                  up    down
tap                     up    up  
vlan                    up    down

Can anyone help please.

Thanks in advance.

Hi ,

I am not sure as what is the device that is connected to SRX.

if it is EX switch , then please follow the KB article which explains about support and not supported configuration.

http://kb.juniper.net/InfoCenter/index?page=content&id=KB22474

Also explain issue clearly as what is not coming up.

Regards

rparthi

Please Mark My Solution Accepted if it Helped, Kudos are Appreciated Too

Thanks - I thought I mentioned EX3300, looks like I haven't.

And yes, that's the exact article I have followed - as you can probably see in the configuration.

I would check on the LACP status first:  show lacp interfaces

I would think you should specify the LACP interval (fast or slow) on both ends -- I typically use fast since it should fail quicker (1 sec interval vs 30 sec interval).

Also, I think that if you only specify 2 aggregated devices, then you should be using ae0 and ae1, not ae1 and ae2.

Ron

Thanks ronf, I have only specified LACP interval slow on SRX side, this is as per that article shown above.

For ae interfaces - I don't have ae0 configured - I will try that on Monday and see how it goes.

Also,

Does anyone know what is this interface - reth0.32767? As I haven't configured it specifically.

Hi dlwrf,

• The link aggregation control PDUs run on the .32767 child logical interfaces for the VLAN-tagged aggregated Ethernet interface.
• The .32767 logical interface is created for the parent link and all child links.

http://www.juniper.net/documentation/en_US/junos12.1x46/topics/task/verification/interface-security-aggregated-ethernet-verifying.html

As per your EX configuration , physical interfaces are deactivated : why

deactivate interfaces ge-0/0/24 unit 0

deactivate interfaces ge-0/0/26 unit 0

deactivate interfaces ge-0/0/28 unit 0

deactivate interfaces ge-0/0/30 unit 0

Regards

rparthi

Please Mark My Solution Accepted if it Helped, Kudos are Appreciated Too

Hi rpathi,

Thank you for the explanation of .32767 part, it's very helpful.

Regarding the deactivation part, the physical interface is not deactivated, it's unit 0, my understanding is once interface is aggregated, you can't create logical unit on each individual physical interfaces.

You would just delete unit 0 on the underlying interface altogether, not deactivate it.

root@SRX1> show lacp interfaces 
Aggregated interface: reth0
LACP state: Role Exp Def Dist Col Syn Aggr Timeout Activity
ge-5/0/4 Actor No Yes No No No Yes Slow Passive
ge-5/0/4 Partner No Yes No No No Yes Fast Passive
ge-5/0/5 Actor No Yes No No No Yes Slow Passive
ge-5/0/5 Partner No Yes No No No Yes Fast Passive
ge-0/0/4 Actor No Yes No No No Yes Slow Passive
ge-0/0/4 Partner No Yes No No No Yes Fast Passive
ge-0/0/5 Actor No Yes No No No Yes Slow Passive
ge-0/0/5 Partner No Yes No No No Yes Fast Passive
LACP protocol: Receive State Transmit State Mux State 
ge-5/0/4 Defaulted No periodic Detached
ge-5/0/5 Defaulted No periodic Detached
ge-0/0/4 Defaulted No periodic Detached
ge-0/0/5 Defaulted No periodic Detached
{primary:node0}

Also this is the article I am trying to follow:

http://kb.juniper.net/InfoCenter/index?page=content&id=KB22474

Thanks guys.

Hi dlwfr,

For Testing Purpose ,

Disable LACP on the SRX and Switch and then verify if Reth interfaces are coming up.

if even after disabling LACP ,Reth is not up , then try commit full on the SRX configuration mode and verify it.

Regards,

rparthi

Please Mark My Solution Accepted if it Helped, Kudos are Appreciated Too

Thanks rparthi, I guess that means I will need to remove the aggregated interface from the EX side as well?

Hi Dlwfr,

LACP protocol has to be enabled on both Nodes.

If it is enabled on one device (SRX) and Not enabled on Switch then Reth interface will not come up.

so you need to disable LACP on both Devices SRX and Switch,

That does not mean you need to disable Aggregate Interfaces.

Aggregated interfaces on EX and Reth On SRX has to be enabled but with LACP protocol.

Regards

rparthi

Please Mark My Solution Accepted if it Helped, Kudos are Appreciated Too

Hi dlwfr,

Thanks for the update.

Now the reth interfaces are up when LACP is disabled on both nodes.

so it is LACP issue.

Check the following:

1. Physical connectivity between SRX to EX swtich interfaces connections.

ensure no cross links between them.

2. ensure you have the following LACP config;

set interfaces reth1 redundant-ether-options lacp passive
set interfaces reth1 redundant-ether-options lacp periodic slow

set interfaces ae1 aggregated-ether-options lacp active

3. if there is no cross link between the interfaces of both nodes , enable LACP and check again.

Regards

rparthi

Please Mark My Solution Accepted if it Helped, Kudos are Appreciated Too

Hi rparthi,

Thank you for your help so far, I really appreciate.

regarding the physical part, i.e. cabling etc, if we remove LACP, and reth is up, so physical should be OK?

I have manually set all the interfaces to 1g in speed, and still ae0 and ae1 on EX are showing physically down.

And I can confirm the following on in.

On SRX:

set interfaces reth1 redundant-ether-options lacp passive
set interfaces reth1 redundant-ether-options lacp periodic slow

On EX:

set interfaces ae1 aggregated-ether-options lacp active

Hi dlwfr,

Without LACP , Links will come up fine.

but with LACP , if there is cross link , then LACP BPDU's will not be processed by corresponding device so it will bring down the reth interface.

I would suggest you to check the Physical Connection First.

Regards

rparthi

Please Mark My Solution Accepted if it Helped, Kudos are Appreciated Too

Hi dlwfr,

U dont need to do LACP on RETH interface. By default RETH was act as active/passive.

Hi dlwfr,

Was there any cross connections between SRX and EX causing this LACP issue?  

Regards

rparthi

Please Mark My Solution Accepted if it Helped, Kudos are Appreciated Too

Hi rparthi,

Thank you very much for following up on this, I wish you had a very nice Christmas and New Year.

I had the problem solved, as Ronf has pointed out the ae interface should start from 0 to 1, not from 1 to 2, so after correcting this, the problem was resolved.

Thank you again.

I have removed LACP by running the following commands,

SRX Config: (Without LACP)

delete interfaces reth1 redundant-ether-options lacp

EX Config: (Without LACP)

delete interfaces ae1 aggregated-ether-options lacp
delete interfaces ae2 aggregated-ether-options lacp

And now the reth interface is coming up.

root@SRX1> show interfaces terse 
Interface               Admin Link Proto    Local                 Remote
ge-0/0/0                up    up  
gr-0/0/0                up    up  
ip-0/0/0                up    up  
ge-0/0/1                up    up  
ge-0/0/2                up    up  
ge-0/0/2.0              up    up   aenet    --> fab0.0
ge-0/0/3                up    down
ge-0/0/4                up    up  
ge-0/0/4.990            up    up   aenet    --> reth0.990
ge-0/0/4.32767          up    up   aenet    --> reth0.32767
ge-0/0/5                up    up  
ge-0/0/5.990            up    up   aenet    --> reth0.990
ge-0/0/5.32767          up    up   aenet    --> reth0.32767
ge-0/0/6                up    down
ge-0/0/7                up    down
ge-0/0/8                up    down
ge-0/0/9                up    down
ge-0/0/10               up    down
ge-0/0/11               up    down
ge-0/0/12               up    down
ge-0/0/13               up    down
ge-0/0/14               up    down
ge-0/0/15               up    down      
ge-5/0/0                up    up  
ge-5/0/1                up    up  
ge-5/0/2                up    up  
ge-5/0/2.0              up    up   aenet    --> fab1.0
ge-5/0/3                up    down
ge-5/0/4                up    up  
ge-5/0/4.990            up    up   aenet    --> reth0.990
ge-5/0/4.32767          up    up   aenet    --> reth0.32767
ge-5/0/5                up    up  
ge-5/0/5.990            up    up   aenet    --> reth0.990
ge-5/0/5.32767          up    up   aenet    --> reth0.32767
ge-5/0/6                up    down
ge-5/0/7                up    down
ge-5/0/8                up    down
ge-5/0/9                up    down
ge-5/0/10               up    down
ge-5/0/11               up    down
ge-5/0/12               up    down
ge-5/0/13               up    down
ge-5/0/14               up    down
ge-5/0/15               up    down
fab0                    up    up  
fab0.0                  up    up   inet     30.17.0.200/24  
fab1                    up    up  
fab1.0                  up    up   inet     30.18.0.200/24  
fxp0                    up    up  
fxp0.0                  up    up   inet     192.168.100.1/24
fxp1                    up    up  
fxp1.0                  up    up   inet     129.16.0.1/2    
                                   tnp      0x1100001       
fxp2                    up    up  
fxp2.0                  up    up   tnp      0x1100001       
gre                     up    up  
ipip                    up    up  
irb                     up    up  
lo0                     up    up  
lo0.16384               up    up   inet     127.0.0.1           --> 0/0
lo0.16385               up    up   inet     10.0.0.1            --> 0/0
                                            10.0.0.16           --> 0/0
                                            128.0.0.1           --> 0/0
                                            128.0.0.4           --> 0/0
                                            128.0.1.16          --> 0/0
lo0.32768               up    up  
lsi                     up    up  
mtun                    up    up  
pimd                    up    up        
pime                    up    up  
pp0                     up    up  
ppd0                    up    up  
ppe0                    up    up  
reth0                   up    up  
reth0.990               up    up   inet     172.17.10.1/24  
reth0.32767             up    up  
st0                     up    up  
swfab0                  up    down
swfab1                  up    down
tap                     up    up  
vlan                    up    down

{primary:node0}

Hi ronf,

I have re-configured it to use ae0 and ae1, it seems same thing for me.

But I noticed that on the EX side, the ae interfaces showing physical link is down:

root@EX3300> show interfaces ae0 
Physical interface: ae0, Enabled, Physical link is Down
  Interface index: 128, SNMP ifIndex: 600
  Link-level type: Ethernet, MTU: 1514, Speed: Unspecified, BPDU Error: None,
  MAC-REWRITE Error: None, Loopback: Disabled, Source filtering: Disabled,
  Flow control: Disabled, Minimum links needed: 1, Minimum bandwidth needed: 0
  Device flags   : Present Running
  Interface flags: Hardware-Down SNMP-Traps Internal: 0x4000
  Current address: 64:64:9b:f6:74:43, Hardware address: 64:64:9b:f6:74:43
  Last flapped   : 2014-11-23 00:48:11 GMT (01:32:02 ago)
  Input rate     : 0 bps (0 pps)
  Output rate    : 0 bps (0 pps)

{master:0}


root@EX3300> show interfaces ae1    
Physical interface: ae1, Enabled, Physical link is Down
  Interface index: 129, SNMP ifIndex: 601
  Link-level type: Ethernet, MTU: 1514, Speed: Unspecified, BPDU Error: None,
  MAC-REWRITE Error: None, Loopback: Disabled, Source filtering: Disabled,
  Flow control: Disabled, Minimum links needed: 1, Minimum bandwidth needed: 0
  Device flags   : Present Running
  Interface flags: Hardware-Down SNMP-Traps Internal: 0x4000
  Current address: 64:64:9b:f6:74:44, Hardware address: 64:64:9b:f6:74:44
  Last flapped   : 2014-11-23 00:48:11 GMT (01:32:08 ago)
  Input rate     : 0 bps (0 pps)
  Output rate    : 0 bps (0 pps)