So what part of Layer 3 don’t they get?

[#1]

One of the keys in building a revolutionary new fabric architecture for the data center is not forcing unnecessary change on the rest of the data center.  Data centers want to evolve gracefully.  The goal is to unleash the promise of the modern data center without disrupting how the infrastructure connects to the fabric or how applications are implemented.

Thus the interfaces to servers and storage should track standards so that the existing servers and storage can connect while evolving to incorporate newer protocols such as Data Center Bridging (DCB), VEPA, and FCoE.  It should also be possible to implement existing applications on top of the fabric to take advantage of the lower latency and greater agility to improve the user experience, but without requiring any alterations to the applications themselves.

Here lies the challenge. Almost all data centers implement Layer 2 and Layer 3 protocols to manage traffic flows.  Layer 2 provides plug-and-play simplicity but struggles to provide effective inter-switch connectivity without inflicting loops and necessitating protocols such as Spanning Tree or TRILL. Layer 2 is also subject to flooding and ARP-table corruption issues.  Layer 3 does an excellent job at inter-switch connectivity and securing (by isolating) subnets but lacks plug-and-play capabilities.  Therefore, most data centers implement a combination of Layer 2 and Layer 3 and applications require traffic flows to constantly cross between Layer 2 and Layer 3.  For example, VLANs - a Layer 2 capability - are used to contain Layer 2 loop and flooding issues while providing traffic separation between different applications or tiers of the same application.  This traffic separation is essential to the security strategy within the data center.  However to enter into a VLAN or cross between VLANs, traffic must pass through Layer 3.

So what confuses me is why anyone would want to implement a data center “fabric” that only supports Layer 2 traffic.  This completely misses the point of a data center fabric.

I have spoken previously about the benefits of a flat, non-blocking, any-to-any fabric within the data center. That this is the ideal topology is not in doubt; all major switch vendors are now espousing their own fabric strategies. A number of them are proposing non-blocking networks built out of their existing switch products. The problem with this strategy is that it inherently exacerbates multi-pathing challenges, increasing the number of potential loops in the network.  To address this, vendors are proposing a set of proprietary or “pre-standard” multi-pathing protocols which create L2 tunnels and thus avoid loops.  This includes non-conforming TRILL-like and Shortest Path Bridging implementations.

Unfortunately these protocols only address Layer 2 traffic.  These tunnels must be terminated before traffic may cross between VLANs or exit the data center. This adds overhead and latency and adversely impacts every application.  In one case, the vendor actually requires separate hardware to handle Layer 2 vs. Layer 3 traffic. This creates a major capacity planning nightmare that did not exist before.  While they admit their solution is best suited for Layer 2-only environments (with the exception of some HPC environments), almost all current applications require Layer 2 AND Layer 3. And the applications are not about to change.

Finally, these approaches fail to address the most challenging issue facing the data center and its network: complexity.  In fact, they ADD complexity.  In the data center, where management and operating costs dominate the economics, why would the concept of introducing more complexity be considered acceptable?

With the Stratus Project, we looked at this problem and came to a very different conclusion.  If you properly engineer the fabric from the ground up, the fabric can deliver unprecedented scalability and resiliency while maintaining the operational simplicity of a single Layer 2/Layer 3 switch. No changes to the interfaces to the infrastructure.  No changes to the applications. Blazing performance, inordinate simplicity.  Stay tuned.

[#2]

You state, "Thus the interfaces to servers and storage should track standards...", but why stop at the interfaces to the servers and storage? The Stratus Project is going to rely on proprietary hardware to lay out the network fabric. I believe we should have standards throughout the entire network. I'm not saying the other vendors are doing a better job at this, FabricPath isn't standard-friendly either. At least with TRILL or SPB we have people working on defining a standard, a standard which can be implemented by any vendor in order to create an OPEN network fabric. A closed, locked-in solution is not a very good one in my opinion.

[#3]

Robert's question reflects his perception that the QFabric architecture is a network and, like any network, he strongly believes that it should be built out of standard interfaces.  We at Juniper completely agree—networks should be open.  However this is where the confusion about QFabric architecture most often arises - the natural assumption that the QFabric architecture is a network.  It is not.  The QFabric architecture is, in fact, a switch.

Robert, This is an excellent question, and speaks to the heart of the design of the QFabric architecture.  Now that we have gone public with the architecture, I can better respond.  Please see this blog posting which directly addresses your question : Taking the Network out of the Network