Switching

Hello all,

 

I've recently built two VCs that I'm currently playing with to get the best performance and resiliency combination.

Each VC is comprised of a single EX4550 and four EX4200s. The VCs will be doing some Layer3 forwarding. The EX4550 is obviously configured as the RE master.

 

Bearing all that in mind, I have one major concern:

 

reloading/bouncing the EX4550 will result in a EX4200 becoming (and staying) the RE master in a preprovisioned VC. What affect does this have on routing performance?

 

In a non-preprovisioned setup, I made the EX4550 a higher priority RE than the backup, meaning that it would always become the master if it were to reboot. As a test, I split the stack by powering off all the members and then return the power in a horrible order  The result was that the backup RE did not assume the master role duing the split (which is good) and neither did the EX4550 - I had to re-activate it when it lost all its members. But when the VC was merged I saw this on the EX4550:

Before merge:

 

Virtual Chassis ID: 9ad5.d7d7.2c35
Virtual Chassis Mode: Mixed
                                           Mstr           Mixed Neighbor List
Member ID  Status   Serial No    Model     prio  Role      Mode ID  Interface
0 (FPC 0)  Prsnt    XSERIAL ex4550-32f 255  Master*      Y
1 (FPC 1)  NotPrsnt XSERIAL ex4200-48t

 

During merge:

 

Virtual Chassis ID: 9ad5.d7d7.2c35
Virtual Chassis Mode: Mixed
                                           Mstr           Mixed Neighbor List
Member ID  Status   Serial No    Model     prio  Role      Mode ID  Interface
0 (FPC 0)  Prsnt    XSERIAL ex4550-32f 255  Backup*      Y
1 (FPC 1)  NotPrsnt XSERIAL ex4200-48t
2 (FPC 2)  NotPrsnt XSERIAL ex4200-48t
3 (FPC 3)  NotPrsnt XSERIAL ex4200-48t
4 (FPC 4)  NotPrsnt XSERIAL ex4200-48t

 

After merge member 1 was left down intentionally):

 

Virtual Chassis ID: 9ad5.d7d7.2c35
Virtual Chassis Mode: Mixed
                                           Mstr           Mixed Neighbor List
Member ID  Status   Serial No    Model     prio  Role      Mode ID  Interface
0 (FPC 0)  Prsnt    XSERIAL ex4550-32f 255  Master*      Y  2  vcp-255/0/27
1 (FPC 1)  NotPrsnt XSERIAL ex4200-48t
2 (FPC 2)  Prsnt    XSERIAL ex4200-48t   0  Linecard     Y  4  vcp-1     
                                                                 0  vcp-255/1/2
3 (FPC 3)  Prsnt    XSERIAL ex4200-48t   0  Linecard     Y  4  vcp-0     
4 (FPC 4)  Prsnt    XSERIAL ex4200-48t 180  Backup       Y  3  vcp-0     
                                                                 2  vcp-1    

 

For those more observant of you, I've had to use VCP over 10ge, because my VC modules for the EX4550s have no yet arrived. I'm not sure if this has helped cause these results.

 

Another thing I did in the preprovisioned mode, was yank the power out of the EX4550 on one of the VCs. I had a laptop connected to each VC and was pinging between them on the same VLAN. I experienced a 30-40 second outtage! No that's not a typo - seconds! Admittedly, I did not have this set:

 

set virtual-chassis fast-failover xe

 I still need to test after configuring that.

 

The way I see it, these are my options:

 

• create one mammoth VC. I'm not experienced with VC enough to trust this. If it fails, it will fail big! 
• create two EX4200 VCs and one EX4550 to handle Layer2 and Layer3 respectively.
• create two EX4200 VCs and leave the EX4550s as standalone routing-switches. A more traditional design that I'm leaning towards.

Any thoughts or experience you wish to share on this would be greatly appreciated.

 

Thanks,

André

Hi André,

To answer your questions:

 

reloading/bouncing the EX4550 will result in a EX4200 becoming (and staying) the RE master in a preprovisioned VC. What affect does this have on routing performance?

 

None at all - routing is always handled by the node the packet ingresses in a VC and will continue to operate at line rate of the port - the Master and Backup roles are just for the routing-engine - eg: the copy of the routing table that is pushed down to the forwarding engines of all members.  Packets coming into a "Line Card" member do not traverse the Master member unless they are destined for a port on that switch.

 

Another thing I did in the preprovisioned mode, was yank the power out of the EX4550 on one of the VCs. I had a laptop connected to each VC and was pinging between them on the same VLAN. I experienced a 30-40 second outtage! 

 

It sounds like you need the following configured:

set chassis redundancy graceful-switchover
set routing-options nonstop-routing
set ethernet-switching-options nonstop-bridging

Add these to your config and run your test again - when you fail the chassis over, confirm that your spanning-tree topology has not re-converged and that all your routing-entries have ages higher than the time since you broke the VC.

 

I would recommend you stick with 2x VCs the 4550 as Master and the 2nd or 3rd 4200 as the backup.  I would also suggest you stick with pre-provisioned mode and do not use "no-splt-detection".  This may seem counter-intuitive only relying on two switches to be master instead of 5, and downing switches that get completely separated from the VC, but when it comes to re-merging, the process is a lot simpler and will not interrupt existing traffic.

 

Fast-failover is only used when you have a ring of VCe ports (VC over 10GE), though I've yet to see any real advantages in enabling it.

 

Hope this helps

 

Thank you very much Ben! That was very insightful.

 

I indeed have none of those options set.

 

I think the classic mistake I made here was RE vs. PFE. Part of my design was to terminate all Layer3 connections on the EX4550, which based on your explantation is the correct thing to do. That then leaves only the SVIs to do routing. Although again, that makes no difference based on your explanation.

 

Hopefully tomorrow I will get to play with this some more and will keep you updated.

 

Thanks again!

André

Here are the results of powering down the EX4550 in my one virtual chassis:

 

root@tweedle-dumb:~# fping -c 10000 -p 1 -d -s -q -t 1 -r 1 -b 1472 192.168.0.1 tweedle-dee.local : xmt/rcv/%loss = 10000/10000/0%, min/avg/max = 0.45/0.62/1.38

 

       1 targets

       1 alive

       0 unreachable

       0 unknown addresses

 

       0 timeouts (waiting for response)

   10000 ICMP Echos sent

   10000 ICMP Echo Replies received

       0 other ICMP received

 

 0.45 ms (min round trip time)

 0.62 ms (avg round trip time)

 1.38 ms (max round trip time)

      309.983 sec (elapsed real time)

 

 

In hind sight, I actually meant to do "ping -f", which I did do as I was leaving and also didn't drop a packet!

 

I'm impressed. Now all I want to do is rip out the processor card of a nexus 7k and see how it copes. 

It does seem a shame that, in a preprovisioned VC, you dont have a way of bumping up the mastership priority on a given unit. I have a similar issue with a VC comprising 2 x ex4500 + 2 x ex4200.  I would like to set one of the 4500 as the preferred master.

root@switch# set virtual-chassis member 0 mastership-priority ?  

Possible completions:

  <mastership-priority>  Member's mastership priority (0..255)

 

Hi smicker,

 

I think what jmcrady means is that you can't have both pre-provisioned, *and* a mastership priority configured at the same time.  See the commit error below:

 

[edit virtual-chassis]
  'member 0'
    Can't have mastership priority configuration with preprovisioned set
error: configuration check-out failed

 

---

@jmcgrady wrote:

It does seem a shame that, in a preprovisioned VC, you dont have a way of bumping up the mastership priority on a given unit. I have a similar issue with a VC comprising 2 x ex4500 + 2 x ex4200.  I would like to set one of the 4500 as the preferred master.

---

I assume you have the 4500s as routing-engines and the 4200s as line cards? Isn't that good enough?

 

I mistakingly thought that the 4550 was better than the 4200, but it just has more bandwidth really. (hope that's an accurate statement).

Got it. Yes that's true unfortunately.

http://www.juniper.net/techpubs/en_US/junos13.3/topics/task/configuration/virtual-chassis-ex4200-cli.html

Note: You cannot modify the mastership priority when you are using a preprovisioned configuration. The mastership priority values are generated automatically and controlled by the role that is assigned to the member switch in the configuration file. The two Routing Engines are assigned the same mastership priority value. However, the member that was powered on first has higher prioritization according to the master election algorithm. See Understanding How the Master in a Virtual Chassis Is Elected.