Ethernet Switching
Highlighted
Ethernet Switching

Has anyone been able to successfully enable ICCP/MC-LAG on a pair of QFX5100 switches?

‎02-17-2014 06:23 AM

After spending seven hours on the phone with our SE and ATAC on Friday, I am beginning to wonder if this is even possible. The instructions on Juniper's site for MC-LAG simply do not work with the QFX5100 running JunosOS 13.2X51-D10.6 (the recommended version and the only version available).  We were able to get ICCP communicating using a very kludgy operation by which we configured the MAC address for each chassis to the IRB interface that peered over the ICCP link.  That got ICCP working, but the MC-LAG interfaces to the access layer switches still would not come up.  We attempted the same MAC configuration on these links with no success.  LACP showed things as working properly, but the MC-AE interface would not form correctly.  A sample from one of the switches:

 

interfaces {
    xe-0/0/4 {
        ether-options {
            802.3ad ae3;
        }
    }
    et-0/0/52 {
        ether-options {
            802.3ad ae0;
        }
    }
    et-0/0/53 {
        ether-options {
            802.3ad ae0;
        }
    }
    ae0 {
        aggregated-ether-options {
            lacp {
                active;
                periodic fast;
            }
        }
        unit 0 {
            family ethernet-switching {
                interface-mode trunk;
                vlan {
                    members all;
                }
            }
        }
    }
    ae3 {
        aggregated-ether-options {
            lacp {
                active;
                system-id 00:01:02:03:04:07;
                admin-key 5;
            }
            mc-ae {
                mc-ae-id 5;
                chassis-id 0;
                mode active-active;
                status-control active;
                init-delay-time 240;
            }
        }
        unit 0 {
            family ethernet-switching {
                interface-mode trunk;
                vlan {
                    members all;
                }
            }
        }
    }
    irb {
        unit 5 {
            family inet {
                address 172.28.30.14/30;
            }
        }
        unit 10 {
            family inet {
                address 10.64.0.1/16;
            }
        }
        unit 500 {
            family inet {
                address 3.3.3.1/24 {
                    arp 3.3.3.2 l2-interface ae0.0 mac 4c:96:14:f3:4f:60;
                }
            }
        }
        unit 1111 {
            family inet {
                address 172.31.253.3/24;
            }
        }
    }
    vlan {
        unit 1111 {
            family inet {
                address 172.31.253.1/24;
            }
        }
    }
}
multi-chassis {
    multi-chassis-protection 3.3.3.2 {
        interface ae0;
    }
vlans {
    core-vlan {
        vlan-id 10;
        l3-interface irb.10;
    }
    default {
        vlan-id 1;
    }
    management {
        vlan-id 1111;
        l3-interface irb.1111;
        mcae-mac-synchronize;
    }
    srx-vlan {
        vlan-id 5;
        l3-interface irb.5;
    }
    v500 {
        vlan-id 500;
        l3-interface irb.500;
    }
}

 I am not sure if this is due to an issue with the QFX5100s or with this release of Junos, but it simple is not working, and frankly these switches are feeling a bit half-baked.  Has anyone had any success with getting MC-LAG working?

11 REPLIES 11
Highlighted
Ethernet Switching

Re: Has anyone been able to successfully enable ICCP/MC-LAG on a pair of QFX5100 switches?

[ Edited ]
‎02-17-2014 01:28 PM

Have you ran this command yet?

set chassis aggregated-devices ethernet device-count 4

 

 Are you seeing LACP packetrs being sent and received?

[KUDOS PLEASE! If you think I earned it!
If this solution worked for you please flag my post as an "Accepted Solution" so others can benefit..]
Highlighted
Ethernet Switching

Re: Has anyone been able to successfully enable ICCP/MC-LAG on a pair of QFX5100 switches?

‎02-18-2014 07:25 AM

"Have you ran this command yet?

set chassis aggregated-devices ethernet device-count 4"

 


Yeah, this is what I have:

 

chassis {
    aggregated-devices {
        ethernet {
            device-count 6;
        }
    }
}

 "Are you seeing LACP packetrs being sent and received?"

 

When the MAC address is hard-coded and the ARP entry for the other host is statically assigned, then yes, ARP over the ICCP link does work as expected; however, that is not in the documentation for MC-LAG and it seems like a bad idea to hard-code that.  However, we do not see LACP establishing a connection to the access layer switches no matter what.

Highlighted
Ethernet Switching

Re: Has anyone been able to successfully enable ICCP/MC-LAG on a pair of QFX5100 switches?

‎02-18-2014 09:18 AM
"However, we do not see LACP establishing a connection to the access layer switches no matter what."

If the server does not support lacp and is not running lacp, then disable lacp on the switch for the lag.
[KUDOS PLEASE! If you think I earned it!
If this solution worked for you please flag my post as an "Accepted Solution" so others can benefit..]
Highlighted
Ethernet Switching

Re: Has anyone been able to successfully enable ICCP/MC-LAG on a pair of QFX5100 switches?

‎02-18-2014 10:44 AM

Sorry, I should have specified - this is connecting to two EX3300 switches configured as a virtual chassis.  LACP is enabled on both sides of the link between that VC and each QFX switch, but it is not working.

Highlighted
Ethernet Switching

Re: Has anyone been able to successfully enable ICCP/MC-LAG on a pair of QFX5100 switches?

‎02-18-2014 12:06 PM

That makes a difference. According to the Product Data sheet for the EX3300, it does not show support for MC-LAG, neither does the Software docs for the current version of software. So this may be unsupported unless Juniper does have a work around for it to work with EX3300

[KUDOS PLEASE! If you think I earned it!
If this solution worked for you please flag my post as an "Accepted Solution" so others can benefit..]
Highlighted
Ethernet Switching

Re: Has anyone been able to successfully enable ICCP/MC-LAG on a pair of QFX5100 switches?

‎02-18-2014 12:25 PM

The EX3300 shouldn't have to support MC-LAG, though -- it should see the connections as a single LAG.  From this guide:

 

http://www.juniper.net/techpubs/en_US/junos13.2/topics/example/multichassis-link-aggregation-qfx-ser...

 

"On one end of an MC-LAG is an MC-LAG client device, such as a server, that has one or more physical links in a link aggregation group (LAG). This client device does not need to detect the MC-LAG. On the other side of an MC-LAG are two MC-LAG switches. Each of the switches has one or more physical links connected to a single client device. The switches coordinate with each other to ensure that data traffic is forwarded properly."

 

In this instance, the EX3300 VC is the client device and doesn't need to detect the MC-LAG connection.  Correct?

Highlighted
Ethernet Switching

Re: Has anyone been able to successfully enable ICCP/MC-LAG on a pair of QFX5100 switches?

‎02-18-2014 12:36 PM
Hey, I am out of my breadth of knowledge so I will have to let the experts continue with you on this one. You want to start showing some troubleshooting capture information such as:
show interfaces mc-ae (on the swithces involved)
show lacp interfaces
show ethernet-switching table
show interfaces terse
show protocols (less ospf bgp etc)
show log messages | match ae

Just any other relevant information that we can look at.
[KUDOS PLEASE! If you think I earned it!
If this solution worked for you please flag my post as an "Accepted Solution" so others can benefit..]
Highlighted
Ethernet Switching

Re: Has anyone been able to successfully enable ICCP/MC-LAG on a pair of QFX5100 switches?

‎02-18-2014 12:46 PM
"In this instance, the EX3300 VC is the client device and doesn't need to detect the MC-LAG connection. Correct?"
Based on that information I have to say yes. I am not too sure if it makes a difference being that it is a Juniper switch. So I am sure you have done a lot, but provide the results of the following so if anything is missed, it can probably be detected. Check the spanning tree configuration is correct for this scenario.
Check the following:

Verify that there is no configuration mismatch.
Verify that all member ports are up.
Verify that the MC-LAG member is connected to the correct MC-LAG member at the other end.
As I have noticed in a few communication issues, can you schedule any maintenance for a reboot of all the devices? Short of a complete reboot, you could also schedule some minimal traffic interruption downtime, and run the "commit full" command. Set up some debugging (traceoptions so we can get a little bit more details)
[KUDOS PLEASE! If you think I earned it!
If this solution worked for you please flag my post as an "Accepted Solution" so others can benefit..]
Highlighted
Ethernet Switching

Re: Has anyone been able to successfully enable ICCP/MC-LAG on a pair of QFX5100 switches?

‎02-18-2014 12:50 PM

Check the following:

Verify that there is no configuration mismatch.
Verify that all member ports are up.
Verify that the MC-LAG member is connected to the correct MC-LAG member at the other end.
As I have noticed in a few communication issues, can you schedule any maintenance for a reboot of all the devices? Short of a complete reboot, you could also schedule some minimal traffic interruption downtime, and run the "commit full" command. Set up some debugging (traceoptions so we can get a little bit more details)

Configuration has been checked and double-checked, both by myself, our SE, and ATAC.  All port members have been confirmed as up and cabling is correct.  Fortunately these switches are not in production yet so I can do pretty much anything on them.  They are down right now but tomorrow I will bring them back up and post the results of ICCP, LACP, etc., commands.

Highlighted
Ethernet Switching

Re: Has anyone been able to successfully enable ICCP/MC-LAG on a pair of QFX5100 switches?

‎02-23-2014 09:15 PM

Were you able to get MLAG to work?

Highlighted
Ethernet Switching

Re: Has anyone been able to successfully enable ICCP/MC-LAG on a pair of QFX5100 switches?

‎12-12-2018 03:30 PM

we have 3300 connect to nexus running VPC, and we just configure a AE and it works fine....... in MCLAG is the juniper VPC version I would think this would work the same (the this is on a VC stack

 

description "xe-0/1/0, xe-1/1/0 bundel to CORE- VPC port eth1/36";
aggregated-ether-options {
minimum-links 1;
link-speed 10g;
lacp {
active;
periodic fast;
}
}
unit 0 {
family ethernet-switching {
port-mode trunk;
vlan {
members x,x,x,x,x;
}
native-vlan-id x;

 

show configuration interfaces xe-0/1/0
Dec 12 17:29:56
description LINK-TO-CORE-A-Eth1/36;
ether-options {
802.3ad ae1;

 

Feedback