One of the keys in building a revolutionary new fabric architecture for the data center is not forcing unnecessary change on the rest of the data center. Data centers want to evolve gracefully. The goal is to unleash the promise of the modern data center without disrupting how the infrastructure connects to the fabric or how applications are implemented.
Thus the interfaces to servers and storage should track standards so that the existing servers and storage can connect while evolving to incorporate newer protocols such as Data Center Bridging (DCB), VEPA, and FCoE. It should also be possible to implement existing applications on top of the fabric to take advantage of the lower latency and greater agility to improve the user experience, but without requiring any alterations to the applications themselves.
Here lies the challenge. Almost all data centers implement Layer 2 and Layer 3 protocols to manage traffic flows. Layer 2 provides plug-and-play simplicity but struggles to provide effective inter-switch connectivity without inflicting loops and necessitating protocols such as Spanning Tree or TRILL. Layer 2 is also subject to flooding and ARP-table corruption issues. Layer 3 does an excellent job at inter-switch connectivity and securing (by isolating) subnets but lacks plug-and-play capabilities. Therefore, most data centers implement a combination of Layer 2 and Layer 3 and applications require traffic flows to constantly cross between Layer 2 and Layer 3. For example, VLANs - a Layer 2 capability - are used to contain Layer 2 loop and flooding issues while providing traffic separation between different applications or tiers of the same application. This traffic separation is essential to the security strategy within the data center. However to enter into a VLAN or cross between VLANs, traffic must pass through Layer 3.
So what confuses me is why anyone would want to implement a data center “fabric” that only supports Layer 2 traffic. This completely misses the point of a data center fabric.
I have spoken previously about the benefits of a flat, non-blocking, any-to-any fabric within the data center. That this is the ideal topology is not in doubt; all major switch vendors are now espousing their own fabric strategies. A number of them are proposing non-blocking networks built out of their existing switch products. The problem with this strategy is that it inherently exacerbates multi-pathing challenges, increasing the number of potential loops in the network. To address this, vendors are proposing a set of proprietary or “pre-standard” multi-pathing protocols which create L2 tunnels and thus avoid loops. This includes non-conforming TRILL-like and Shortest Path Bridging implementations.
Unfortunately these protocols only address Layer 2 traffic. These tunnels must be terminated before traffic may cross between VLANs or exit the data center. This adds overhead and latency and adversely impacts every application. In one case, the vendor actually requires separate hardware to handle Layer 2 vs. Layer 3 traffic. This creates a major capacity planning nightmare that did not exist before. While they admit their solution is best suited for Layer 2-only environments (with the exception of some HPC environments), almost all current applications require Layer 2 AND Layer 3. And the applications are not about to change.
Finally, these approaches fail to address the most challenging issue facing the data center and its network: complexity. In fact, they ADD complexity. In the data center, where management and operating costs dominate the economics, why would the concept of introducing more complexity be considered acceptable?
With the Stratus Project, we looked at this problem and came to a very different conclusion. If you properly engineer the fabric from the ground up, the fabric can deliver unprecedented scalability and resiliency while maintaining the operational simplicity of a single Layer 2/Layer 3 switch. No changes to the interfaces to the infrastructure. No changes to the applications. Blazing performance, inordinate simplicity. Stay tuned.
Exploring the vision for the networking industry and the issues shaping its future.