A Blueprint for the New Network with Juniper Networks EX Series and vGW at the Core
byssl_boy10-26-201205:00 AM - edited 06-11-201307:10 AM
This is a guest blog post. Views expressed in this post are original thoughts posted by Glen Kemp, Solutions Consultant at SecureData Europe. These views are his own and in no way do they represent the views of the company he works for.
A customer recently asked me to provide a proposal for a consolidated data centre with some fairly fixed requirements. It had to fit in a fixed budget, a fixed amount of rack space and host a fixed number of servers.
It was the same week that Juniper Networks launched the EX4550 Ethernet Switch. This represented for me the last piece of a puzzle in terms of what data centre networks will increasingly look like. So when my customer asked me: “design me a data centre, I’ll need it operational within a month” what did I propose? Well, for a start a shopping list much shorter and a timescale much less pessimistic than you’d think.
Breaking it down
The modern data centre can be broken down into five basic components:
Compute Platform. Servers or anything with a general-purpose operating system such Windows, Linux, or Solaris which ultimately run any “line of business” application.
Interconnect. Anything which connects anything to anything else over any medium. In most cases this will be Ethernet but includes Fibre Channel and even legacy serial networks and Wi-Fi.
Security. This can either be embedded as a function of the operating system or discrete in the case of a network appliance.
Storage. At some point, data has to “rest” somewhere, in many cases it will be network attached (via Ethernet or Fibre Channel), but may well be directly attached or embedded to the compute platform.
Utility. This covers everything else such as HVAC, power and lighting, but also could be used to cover non-line services such as backup, network management overlay and resiliency services such as clustering.
Starting with a clean sheet of paper, how do we build a new data centre from these five basic components? If we select a single vendor and technology for each component we can standardise upon off-the shelf components. This reduces the quantity and classes of physical hardware needed; in turn this makes redundancy simpler and leaves less equipment idling. Here’s the design I put together:
At the heart there has to be a core fabric; in our case an Ethernet switch. Whilst Ethernet is an imperfect transmission medium, it’s fast, open and very easy to scale. The EX4550 is an ideal network core as it allows us to collapse the classic access, aggregation and core layers into a single pair of switches. A single rack unit can directly connect our compute platform to our storage at wire rate whilst handling full layer 3 routing. Each switch provides a frankly mind-boggling 480Gbps of bandwidth linked via a 256Gbps Virtual Chassis backplane. Whilst I can quote speeds and feeds till I’m blue in the face, what makes this truly impressive is the list price; $19,000. For a next generation switch with full Layer 3 routing features this provides ample capacity for our “east-west” server-to-server and server-to-storage traffic.
The Compute Platform
A quad-socket Intel Xeon E7 server can provide 40 cores of processing cores in 2u of rack space. This essentially provides a platform to host a very large number of virtualised servers in a hyper dense environment. The servers themselves would essentially be diskless; save an industrial flash drive to boot the hypervisor. To connect the platform to the fabric 10GbE direct attached copper (DAC) cables provide inexpensive, straightforward connectivity without having to mess around with expensive GBICs and delicate fibre. Four connections are all that’s required to link each server to the fabric; two dedicated for server traffic and two for storage I/O. Capacity planning is obviously critical, but the partnership of new processor and RAM technologies such as Intel Xeon E7 and LRDIMMS make this kind of server density not only feasible, but a practical solution. As few as four servers would provide 320 execution cores in a less than a quarter of a standard 42u rack. A slightly less avant-garde approach would be to use dual-socket, 1u servers with slightly fewer cores. This would provide a greater redundancy and lessen the impact of a single server failure, at the cost of increasing the number of physical connections.
Separating the storage from the compute platform is commonplace; however there are different ways of achieving this. The iSCSI standard is commonly deployed and supported by many vendors. It allows flexibility of deployment without proprietary lock-in of the storage adaptor, transit switch or disk array. By again utilising “of the shelf” technology, expensive cards for the servers and dedicated Fibre Channel switches can be avoided. Whilst Fibre Channel probably represents the highest performance solution for disk access, it doesn’t offer value for money or flexibility an iSCSI solution would. Given the need for a high-performance Ethernet switch for the server interconnect, utilising iSCSI to connect the compute platform to the storage represents a simple, elegant solution.
The Security Layer
By virtualising most, if not all, of the application servers into the compute platform, we create a new problem. Most virtualised environments place the servers on the same layer 2 subnet. Whilst this is “easy” it doesn’t provide a straightforward way to police server-to-server network traffic or provide in-line intrusion detection. Artificially breaking a virtualised network into subnets is painful; routing that traffic via a firewall (either physical or virtual) almost certainly creates a performance bottleneck and definitely increases the complexity. A more practical solution is to use a network security tool designed for the job at hand; a virtual security gateway integrated into the hypervisor such as Juniper’s vGW. The traditional approach of a “firewall in a virtual machine” is not a particularly efficient method of forwarding traffic; it is forcing a virtualised environment to behave like a physical one; a square peg in a round hole. By hooking into the VMware native APIs, vGW can directly intercept the network I/O of each virtualised machine as it heads down network stack, making the forwarding decision at the point of egress rather than a traditional “default gateway” approach. This allows much higher performance than forcing the traffic to an external firewall. vGW acts as a transparent layer 2 bridge between the guest and the virtual switch. This approach is also more cost effective than purchasing a dedicated firewall cluster capable of handing inter-server communication. Ultimately this means layer 3 routing decisions can be handled at layer 3 by a device designed to handle such high throughputs; the Juniper EX4550 switch.
One of the benefits of this methodology is that the data centre build out time is significantly reduced. Fewer physical devices mean less network cabling, less power and less cooling. The initial “stand up” on a new data centre deployment is perhaps the slowest and most painful part. Once the basic connectivity is addressed, the application, network and security teams can then start their individual tasks. If there is less to “stand up” the “time to live” can be radically reduced and project life cycles shortened.
The amazing thing about this design is that there is very little actual equipment to deploy. In fact, the total rack budget is tiny:
This design is obviously created with a fixed capacity in mind, but it can easily be scaled upwards to accommodate more physical and logical servers. Because the EX4550 is designed as a “top of rack” switch, up to 10 switches can be added in the same virtual chassis should you need to support dozens if not hundreds of physical hosts.
Ultimately, by consolidating the core of the infrastructure, you end up with a much simpler deployment. This means you save in real-terms on the fifth important part of the data centre; the bit which costs you money and never see any return on; the utility. Fewer devices mean less power draw, less rack space to rent, fewer network changes to make and fewer mechanical components to fail. The upshot is that the new network is physically and logically smaller than one, which it will replace. It will cost less to maintain and be faster to “stand up”. Anyone who has a pricing model based upon physical hosts deployed or number of rack units powered is going to very shortly run into a series of very big problems.
So, what aspects of have I missed out? Where is the Achilles heel in my plan? What other innovations are out there which I’ve not mentioned? I’d be pleased to hear about them in the comments section below.
Ben has been working with service providers around the world for the last 15 years developing business cases for a variety of product concepts and new ventures.
Ben holds an MBA from MIT and a BS & MS in Mechanical Engineering from Johns Hopkins University.
A Marketing and Business Development professional with 24 years extensive Sales/Business Development, Marketing and Technical experience in the Networking/Telecoms/Datacomms and Mobile market segments, focused on selling to Service Providers.
Fomerly VP Marketing at the Metro Ethernet Forum (MEF)
David Noguer Bau is the head of Telco Vertical Marketing at the SP Strategic Marketing team in Juniper Networks. He has extensive experience in Service Provider network evolution and regularly runs executive sessions with technical and marketing teams of important telecom operators to accelerate the adoption of virtualisation.
David is based in Barcelona and has over 15 years of experience in the telecommunications sector. Prior joining Juniper Networks, Mr. Noguer Bau spent seven years at Nortel where he was a Business Development Manager specializing in Carrier Ethernet and Broadband areas. Before Nortel he worked at Eicon-Dialogic as Technical Manager in Spain. David has been the Country Marketing Chair at Metro Ethernet Forum for Spain.
Mr. Noguer has wide experience speaking at international Conferences. He was graduated as Computer Engineer by Universitat Autonoma de Barcelona (UAB) and has an executive MBA from EADA Barcelona and executive education at the Thunderbird School of Global Management (Arizona) and the Henley Business School (UK).
The views expressed here are my personal opinions , have not been reviewed or authorized by Juniper Networks and do not necessarily represent the views of Juniper Networks.
I’ve been 29 years in the industry, first as a trainee IBM operator at Barclays Bank, later starting my own business which was ultimately acquired by French listed company EasyVista – [giving me great insight into working as part of an internationally focused company alongside organisations like Reuters, UBS Warburg, GlaxoSmithKline and London Electricity].
I am Sales & Marketing Director at Netutils – a specialist IT Networking and Security solutions provider. My passion continues to be making enterprise more efficient via the intelligent deployment of technology, with a view to delivering real value for my clients.
Donyel Jones-Williams is Senior Product Marketing Manager overseeing SDN and Core Service Provider Product line for Juniper Networks. In this role, he leads all of the internal and external marketing activities for T-Series, PTX, IP/MPLSView and NorthStar SDN Controller.
Prior to joining Juniper Networks in January 2014, Donyel was a Senior Product Line Manager for Cisco Systems with in the High End Optical Routing Group managing product lifecycle for multiple products lines helping telecom providers operate efficiently and effectively including; ONS 155xx Product Family, ONS 15216, ONS 15454 MSTP, Carrier Packet Transport Product Family, ME 2600x, & ASR 9000v. He also negotiated favorable agreements with 3rd-party vendors furnishing components and parts and conducted both outbound and inbound marketing (webinars, case study-development, developed and delivered both business & technical at Cisco Live 2005-2012).
Donyel graduated from California Polytechnic State University-San Luis Obispo with a Bachelor of Science in Computer Science. While attending Cal Poly SLO he was a collegiate student athlete playing football as a wide receiver and a key member of the National Society of Black Engineers. Donyel is now an active volunteer for V Foundation.
With 20+ years of global IT management experience, Gary Clark oversees all technology services to support 9,600 employees at Juniper Networks, a $4.5 billion networking innovator with operations spanning 123 offices in 47 countries. Prior to Juniper, Gary held senior IT management roles at BlackRock/Barclays Global Investors and Deutsche Post/DHL.
Senior Systems Engineer for NEC NZ. Focused on Juniper Networking equipment, SDN and NEC compute platforms. Busy studying for the JNCIP-SP and ENT.
Outside of work I enjoy the great outdoors: Mountaineering, Bouldering, Rock or Ice climbing, Tramping (hiking to non-Kiwis) and Snowboarding.
I have been in the networking industry for over 30 years: PBXs, SNA, Muxes, ATM, routers, switches, optical - I've seen it all. Eleven years in the US, over 20 in Europe, at companies like AT&T, IBM, Bay Networks, Nortel Networks and Dimension Data. Since 2007 my focus has been on services at Juniper: support services, professional services, service automation. Our market is characterized by amazing technological innovations, but technology is no use if you cannot get it to work and keep it working. That is why services are so exciting: this is where the technology moves out of the glossy brochures and into the real world!
Follow me on Twitter: @JoeAtJuniper
For more about me, go to my LinkedIn profile: http://fr.linkedin.com/pub/joe-robertson/0/4a/34a
Jon joined Fujitsu UK&I as Chief Technology Officer in January 2011 from the public sector, where he was Chief Information Officer, Transformation Director and SIRO at the Valuation Office Agency. Prior to this he was Her Majesty’s Revenue and Customs’ first Chief Technology Officer, leading the integration of the former Inland Revenue and Customs & Excise organizations.
His roles in both organizations drove out savings in excess of £600m, as well as bringing about significant technology transformation, building high performing teams in the process.
Jon was a founding and core member of the UK Government Chief Technology Officer Council and recruited and led a team creating Public Services Network, XBRL mandation and cross government channel strategy.
Jon’s client side board level experience is built on 11 years at Accenture, with clients including Barclaycard, Legal & General, BP, Castrol and BG Group.
Jon now leads the UK & Irelands 1,200 strong Architecture Community, driving standard solutions, reinforcing rigorous re-use and a collegiate collaborative community and culture, leading with courage and conviction.
Jon is a firm believer in the 4Ps – Pace, Passion, Pride and Professionalism. He is a Chartered Engineer, Fellow of the British Computer Society, founding Fujitsu Fellow and a member of the Advisory Board for AppDynamics.
I'm a Distinguished Systems Engineer at Juniper Networks. My main technical interests are routing protocols, MPLS, PCE/WAN Controllers, automation, and optical integration. Before joining Juniper Networks in 1999, I worked at BT for several years, at first in the Photonics Research Department and later in the data transport and routing area. I have a PhD in ultrahigh-speed optical transmission and processing and an MA in Physics, both from Cambridge University. I co-authored the book "MPLS-Enabled Applications: Emerging Developments and New Technologies", with Ina Minei. The book is now in its third edition.
Marcel Wiget is Consulting Engineer Specialist and member of the Advanced Technology team for EMEA. His career within Juniper started back in 2009 as a Senior Systems Engineer driving one of the first MX based Broadband Edge deployment to success. Prior to Juniper, Marcel held various positions in pre-sales, professional services and development at Chantry Networks, Spring Tide, Nortel Networks and Wellfleet.
I love the intracacy and intimacy of succesful communications. Why and how people engage with each other is fascinating. I am also consumed with the way IT changes behaviours, values and expectations in society.
I bring this sense of wonder to my role in EMEA Service Provider Marketing Programs at Juniper Networks.
Down time: My passions are music, reading, politics, Derby County and playing the guitar (and the harmonica).
You can follow me elsewhere:
my personal blog: http://neilpound.tumblr.com/
my LinkedIn account: Neil Pound
I am one of a small team of Network Engineers working for Lumison Ltd, a UK ISP/MSP based in Edinburgh, Scotland. I have been with the company for almost 6 years moving from frontline support to the Managed Services team dealing with customer network design and implementation before talking up the role of Network Engineer. As well as the JNCIE-ENT certification.
I am currently a Sr. Product Marketing Manager specializing in Juniper's Security Portfolio in the Service Provider industry. I am an experienced senior technical leader, technical marketing engineer, solutions architect, and product marketing manager with over 20 years of Internet and Enterprise industry experience developing solutions from scratch often in relation with business units and technology groups, my projects ranged from product, solution, and technology development to corporate technology strategies. I have strong analytical skills and I am able to crunch and articulate complex technology to a variety of audience knowledge levels. I possess a deep hands-on technology and business knowledge of Service Provider and Enterprise architectures with deployment hands-on skills. I also bring a unique perspective of open source philosophy, including but not limited to open innovation, software development methodologies, open source monetization and business models, and licensing and compliance in software integration. I am a strategic leader with proved ability to empower a team to improve their product, themselves, their team, and our company’s market position.
An inspirational marketing leader working across the entire marketing mix to transform brand into business value, activity into results and thought leadership into measurable pipeline. You can follow me on Twitter at @PaulGainham
I have been at Juniper Networks since 2004, focused on Corporate Communications (media relations, analyst relations, customer reference progam) for the Europe, Middle East & Africa region.
I have worked in the networking industry since 1988.
Raghu Subramanian is VP of Sales Engineering for Asia-Pacific at Juniper Networks. Prior to this, he has served Juniper as chief strategist for the security business, product evangelist to channel partners, and product manager for M-series routers.
In past lives, Raghu was a chip designer at Hewlett Packard, and an R&D manager at a start-up acquired by PMC-Sierra.
Raghu has an MBA from the MIT Sloan School of Management, Ph.D. in Computer Science from the University of California at Irvine, and a B.Tech.in Electrical Engineering from the Indian Institute of Technology at Kanpur. In his spare time, he enjoys reading non-fiction, coaching kids for the Math Olympiad, and traveling with his family to other countries to learn about their ways.
Russell is the global leader of the Advanced Technologies team specializing in Data Center Virtualization and Automation. Russell leads the team that provides Juniper’s major customers with solutions to provide the network underpinnings for highly virtualized and automated data centers.
Stephen is currently a Partner Acccount manager at Juniper Networks, and has held this role for 3 years. Prior to Juniper, he worked at Extreme Networks for 11 years in a variety of roles.
Stephen is a Father of 3 children, a keen cricket fan and enjoys cooking, reading and theatre in his free time.
Stephen Liu is Senior Director of Product Marketing for Juniper Networks. In this role, he leads product marketing for Juniper’s industry-leading service provider portfolio of high-performance routing and switching products. These products include Juniper PTX Series, T Series, MX Series, and ACX Series platforms along with software and security.
Prior to joining Juniper in 2013, Stephen served as Director of Service Provider Marketing at Cisco Systems. In that role, he led product and solution marketing worldwide for the service provider routing, switching, optical, and software portfolio. Products included NCS, CRS, ASR, and ONS platforms.
Stephen attended the University of California, San Diego, where he received a bachelor’s of science degree electrical engineering – communication systems.
Hobbies include restoring old Volkswagens and coaching competitive youth soccer. He is based in Sunnyvale, California.
About Stuart Borgman, Business Systems Architect
Having spent many years in the telecommunications and networking industry, I understand just how complex networking technology can be, and equally, just how important it is for today’s fast-moving business.
Making the right IT choice for any organization is paramount, especially when it is helping drive business strategy. In my role at Juniper, I’m committed to helping all organizations plan and design their IT systems to make sure that each part works together to fully meet the needs of the business. Together with my colleagues in Professional Services, our aim is to ensure that all you need focus on is your business strategy, not the technology.
I'm currently working on a number of Service Provider projects focusing on Identity Management. These range from Mobile Operator WiFi offload projects & 3G SCADA device management to broadband authentication encompassing quota and service management for P2P and video traffic control.
I have over 15 years progressive experience designing complex RADIUS platforms to meet the demands of the most multifaceted businesses. One of the most successful projects focused on the consolidation of 22 separate RADIUS platforms spread over a large estate onto a single pair of RADIUS servers, offering the same functionality and business logic as the prior estate.
In addition I have spent a number of years observing and implementing solutions for the enterprise space in the BYOD and NAC market. It's a keen area of interest for me as it combines the whole concept of identity management and business needs together. My largest project in this space was for a UK company with global offices providing a NAC solution for over 200 sites, with over 150,000 staff.
Over the last 10 years Netutils have invested heavily in developing a technical team to support me and the business on these key areas. I strongly believe that a solution designed by Network Utilities should be the right solution technically and commercially for the customer, so my over-riding focus is on customer satisfaction. This follows on in the technical support service the Netutils team offer post implementation.
Not making tea, NAC, RADIUS, Quota Management, Diameter, full life cycle of the subscriber management. Working with large organisations taking a concept through to delivery around identity management whether authentication or Quality of Service.
An accomplished network engineer with 14+ years’ experience, and a Juniper employee since 2004, Tony leads the IT team focused on deploying “Juniper on Juniper”, using Juniper technology to run the business and deliver core business services across the enterprise. Tony holds a double JNCIS certification in Enterprise Routing, Security (JNCIS-ER, JNCIS-SEC) and a BS degree from California Polytechnic State University. Outside of work, Tony serves on a School Advisory Council, loves biking and good coffee.
Zoe Sands is Head of Digital Marketing at Juniper Networks and is responsible for digital marketing and social media across EMEA. She is an experienced Digital Marketer since 1997 with PRINCE2 practitioner status, during this period Zoe has successfully launched many new online innovations for Juniper Networks, Cisco, Dialogic, the Chartered Institute of Marketing (CIM) and Hyundai, including content managed and e-commerce based websites to integrated social media programmes. She has International exposure running projects globally, regionally and at a country level.
Zoe’s approach is to create an environment where those around her can share her passion for the Internet and the opportunities it presents. She says sharing knowledge, championing and communicating the benefits of digital capabilities enhances both the user experience and offers additional online communication channels and business opportunities. Zoe has a blog ‘Learning and sharing...’ to share her experience of all things online marketing, social media, chat online, SEO, SEM and mobile related content. You connect with Zoe via LinkedIn or find her on Twitter: @zoe9 and @ZoeSands.