Juniper Employee , Juniper Employee Juniper Employee
EVPN as a stepping stone for SDN
Sep 28, 2015

Meet Jim. Jim runs the data center network in a hospital. Actually, to be more accurate: Jim is the entire networking department – it’s just him, there is no one else.


Jim is a pretty straight-laced practical guy. He likes to go home at 6 pm and play with his kids. What he doesn’t like is being woken up at 3 am to solve network emergencies. He likes to use rock-solid, proven, carrier-grade networking technologies. He likes to automate whatever he can to save work and to avoid breaking the network with fat-fingered configuration changes. He does not drink any fancy Starbuck Frappuccinos and does not like any fancy SDN in his network. He likes his network simple, boring, and reliable - thank you very much, ma’am.


It will come as no surprise that Jim’s network is pretty simple and straight-forward: some access layer switches, some aggregation switches, some core switches, some edge routers, and some firewalls. He allocates one layer 2 VLAN for each department and he interconnects the VLANs at layer 3 at the aggregation layer through the firewalls for security.


The city has been growing and so has the hospital’s data center. Recently Jim’s network is not so boring as he likes it to be. There have been broadcast storms, there have been loop meltdowns, there have been spanning tree convergence problems.  The spreadsheet that keeps track of where each VLAN is, has become nearly impossible to keep up to date.  Jim is not a happy camper.


Also, the hospital has merged with another hospital, and now Jim needs to interconnect his data center with the data center of the other hospital. 


The final straw is that the hospital's CIO has been drinking a little bit too much McKinsey cool-aid and insists that it’s time to transform the data center into a private cloud and to move some applications into the public cloud. The CIO has kicked of the Next Generation Network Transformation Initiative (NGNTI) [1]. Sigh.


Jim, not one to avoid a challenge and being the practical guy that he is, has defined some selection criteria for the new network architecture including:

  • He wants the architecture to be open and multi-vendor interoperable. He doesn’t want to be locked into some vendor-proprietary technology. That, right there, eliminates Cisco APIC and ACI.
  • He wants the architecture to be simple, scalable, and reliable and based on carrier-grade proven technology.
  • He wants his equipment to be as simple and low-cost as possible. He doesn’t want to pay for any features he doesn’t need. That said, he does want his network to be future proof.
  • He wants to automate whatever he can to save operational cost and for increased reliability.

To cut a long story short: Jim has selected Ethernet Virtual Private Networks (EVPN) [2] as the corner stone for his next generation network for many reasons including the following ones:

  • EVPN is an open multi-vendor protocol.
  • EVPN-over-VXLAN in the data center brings the best of both worlds. The physical network can be an IP fabric that is rock-solid and can scale ad infinitum. The logical service provided to the tenants can be a traditional layer 2 service so that the applications don’t have to be re-written.
  • EVPN-over-VXLAN in the data center can connect seamlessly to EVPN-over-MPLS or L3VPN-over-MPLS in the WAN for Data Center Interconnect (DCI) and for hybrid cloud extension (connecting the data center to the public cloud).
  • Although EVPN itself is rather new, it is based on proven carrier-grade VPN technology. If it’s good enough for large service providers, Jim figures it’s good enough for him.

EVPN-VXLAN Network Design


Jim selected Juniper for the new network design: QFX5100-Series switches (with a growth path to QFX10000-Series if he needs it in the future for capacity), MX routers, and SRX firewalls.


After much consideration, Jim still doesn’t see any reason to put any fancy SDN in his network. He can do all the automation he wants using Python scripts that speak NETCONF to the switches, routers, firewalls, and BGP route reflector.


The new network was deployed and life was good. That said, like any transformation project, there was room for some improvement on the automation side.


I haven’t mentioned it before, but the data center is an eclectic mix of VMware virtualized servers for IT applications, OpenStack and KVM virtualized servers for the developers (another initiative of the CIO to save money), and bare-metal non-virtualized servers for those pesky X-ray applications that cannot run in a virtual machine.


Jim dealt with the virtual machines by writing a nifty set of Python scripts. He thought about using Ansible or Heat templates, but decided to keep it simple and used PyEZ [3] instead.  The script keeps track of which virtual machine is on which server. It keeps track of which virtual machine belongs to which tenant. It keeps track of which server (bare-metal or virtualized) is connected to which port on the top of rack switch. Etcetera. All this information is gathered using a combination of vSphere APIs, OpenStack APIs, LLDP MIBs, etc. The Python scripts use NETCONF to automatically put the right EVPN routing-instance on the right switch with the right VLANs on the right ports, the right route distinguishers, the right route targets etc. No need for a spreadsheet anymore.


Jim even automated the security policies with Python scripts. The policy script reads an input file that specifies which VLANs is allowed to talk to which other VLANs, and if so, whether it should go through the SRX firewall, and if so, what the security policy is. Jim is extremely proud of this script, and it took him several months to get it right. He had to do some extremely clever manipulation of the route targets and route distinguishers to force the inter-VLAN traffic through the firewall. To be completely honest – a lot of the ideas for this script were “borrowed” from an IETF Draft describing exactly how to do this [4].


One fine morning, Jim stumbled across the OpenContrail website [5]. As he’s reading about OpenContrail, a realization slowly dawns upon him. As he is writing all of his scripts he is in fact slowly introducing SDN into his network and he is slowly re-inventing Contrail. It turns out that OpenContrail is integrated with OpenStack and VMware to keep track of virtual machines. It uses BGP EVPN under the hood. It uses clever route target and route distinguisher manipulation to apply policies and implement service chains. It puts a nice framework and GUI on top of all of that. Aarghh! I guess Frappuccinos taste good after all.


As he is reading the OpenContrail documentation, he also realizes how much more room for improvement there is in his scripts. He realizes that he hasn’t even begun to scratch the surface. He doesn’t yet have any formal model or REST API for his abstractions (e.g. policy and service chaining). He certainly doesn’t have a nice GUI. He had not thought much about scaling out and high availability for the servers that run his scripts. He hadn’t thought about enforcing the security policy locally in the vSwitch in the hypervisor. He has not yet integrated rich analytics into his automation framework. Urrghh… much more work to do. Maybe Jim can do some more “borrowing” since OpenContrail is open source anyway.


EVPN-VXLAN Network with SDN Controller


The moral of the story is this. You might not be a fan of SDN. You may think that you just want a rock-solid, boring, and conservative network. You may think that you can just use Python scripts and NETCONF or Puppet to automate repetitive tasks. You might think that you don’t need any fancy magic SDN in your network. And you would be right. But when you look at the tasks that SDN will automate for you, and if you look at how SDN works under the hood you may very well realize that it is actually the exact same rock-solid BGP-based technology that you would deploy anyway. And you may very well realize that the “magic” part of SDN is –under the hood- actually the exact same process that you would implement if you wrote the Python scripts yourself. But if you choose to use OpenContrail you get all of that in a nicely packaged way, with a GUI, and with analytics, and with seamless integration with OpenStack and VMware, and many more nice things.


Have a look at Or alternatively, if you are a pure VMware shop, have a look at our integration with VMware NSX. Either way -- download it [6] and try it out. I bet that you will like it more than you think you will.

And remember: SDN doesn’t have to be an all-or-nothing deal. It’s perfectly fine to start with a traditional network design and add SDN to it later when the scale becomes large enough or when the automation becomes complex enough to justify it.


[1] Pet peeve: please don’t ever call anything Next Generation. When it’s done and implemented it won’t be Next Generation anymore; it will be Current Generation. And what will you call the thing after that? Next-Next Generation (NNG)?












Sep 28, 2015

Jim is the Samuel L. Jackson BMF type of guy. [7]


He wakes up one morning from being a boring, simple, playful father, 9 to 6 VLAN/STP nw admin to the person that runs a badass network involving knowledge of BGP, EVPN, VXLAN, MPLS, L2/L3 VPNs, underlay / overlay setups, Python scripting (because he knew enough Ansible and Heat to chose pyEZ instead) and some NETCONF.

As this is peanuts for a one man job, he takes the next step of building a set of server clusters based on redhat because, being a regular networking guy he knew perfectly well from heavy experience that canonical / mirantis / centos distros would not fit the CIO purpose and its easier for him to install, maintain, build and deploy a full blown openstack release along with opencontrail networking, and tie all things up in a sort of Frappuccino fashion.


[7] Because this is "To cut a long story short", he is now a grandfather of 4 and did not realize