a row or level of a structure, typically one of a series of rows placed one above the other and successively receding or diminishing in size.
synonyms: row, line, layer, level, balcony
The terminologies of “single-tier” or “two-tiered” fabric have been used to describe network fabric or cluster-based products. I’m doubtful whether a two-tiered fabric can truly function as a single-tier network, but certainly not all two-tiered networks are made equal.
What do we mean by those terminologies?
The most prevalent use of the word tier in networking is its use when describing network topologies in campus and data center environment. Most people understand the phrases “two-tiered” and “three-tiered” refer to the physical topologies in the following figure.
The fundamental reason for constructing a two-tiered or three-tiered network is to achieve the required port fan-out with the desired over-subscription ratio of bandwidth. Increasing or decreasing a tier in the network topology is a big deal, because it directly affects the cost of the networks. If the technology allows unlimited fan-out on a simple box, a two-tiered topology would be a huge waste of ports, a three-tiered topology would “double” the waste.
What is a “tier” in network?
IMHO, the most essential meaning of “number of tiers” is the number of hops that most packets traverse the physical network. In a typical two-tiered network, it takes north-south traffic two hops and east-west traffic three hops; while in a typical three-tiered network, it takes north-south traffic three hops and east-west traffic five hops. In a two-tiered network, a network architect needs to design one layer of over-subscription (OS) ratio; while for three-tiered network, s/he has to design two OS ratios, one at the aggregation layer, and another at the core layer.
Tiers of a fabric
With the advent of multi-path technology, the inter-connectivity among network nodes has changed to exploit the benefits afforded by multi-pathing. The word “fabric” is often used to describe a cluster of nodes that are somewhat more “closely” interconnected.
In a QFabric, the QF-interconnects are chassis specifically designed for interconnecting QF nodes. Network ports are not allowed on Interconnects. In Virtual Chassis Fabric (VCF), network ports are allowed on every node. In Q-Fabric, QF-nodes must be connected with QF-Interconnects in a star-topology. In VCF, every node can be both an “interconnect” and an “edge” node, hence allowing “arbitrary” topologies. Such flexibility allows the customer to tailor their network topology design to meet their specific traffic needs. If we compare the QFabric (in star-topology) and a VCF (in a spine-leaf topology), we have the following table.
N-way (N is # of spine)
From the table above, it is clear that, from a physical topology perspective, both QFabric and VCF in Spine/Leaf topology resemble a two-tiered network. From a logical perspective, both QFabric and VCF appear as a single managed device, hence there may be a tendency to call both QFabric and VCF (in spine-leaf) a “single-tier” fabric. However, the “single-tier” terminology conflicts with its two-tiered nature in physical topology. Hence, the “single tier fabric” terminology may be confusing to some people.
Virtual Chassis Fabric Topologies and Tiers
As mentioned above, VCF can support “arbitrary” topologies. However, only a few typical physical topologies are used by network designers to achieve performance requirements within the cost limit. The figure below shows a few popular topologies.
Since VCF performs shortest path forwarding, for all of the above physical topologies, VCF will optimize for distance/latency. For a fully meshed VCF, unicast traffic will always travel the directly connected links. All traffic traverses the fabric in 2 hops. Hence it functions like a 1 ½ tier network. In a spine-leaf VCF, all shortest paths for east-west traffic consist of 3 hops, while all north-south paths consist of 2 hops. Hence a spine-leaf VCF functions like a two-tiered network, although a VCF will allow maximum bisectional traffic without blocking any links. A spine-leaf VCF will also allow the spine switches to perform multicast replication so that the receivers on the leaf switches will observe the same multicast latency. Due to its topology flexibility, a VCF can also be constructed as mesh-connected spine-leaf sub-topologies as shown above. Such a VCF will allow two small pods to be inter-connected and be managed as one device. It not only allows the servers and storages attached to the same pod enjoy the same low latency as a spine-leaf VCF, but also provide inter-pod resiliency. As a matter of fact, VCF technology will discover and exploit the best attributes of the underlying physical topology, and will strive to achieve optimal performance regardless whether it is a one-tiered or two-tiered network.
BTW, VCF is supported on QFX5100, QFX3500/3600, and EX4300. Customers who have previously purchased QFX3500/3600 will be able to use them as part of a VCF with a software upgrade.