Data Center Technologists
Showing results for 
Search instead for 
Do you mean 

Network-aware placement of OpenStack workloads

by Juniper Employee ‎05-18-2015 10:14 PM - edited ‎05-19-2015 08:36 AM

What happens when you launch an instance in an OpenStack based cloud and the virtual machine is started on host that does not have sufficient CPU capacity or storage capacity? You will notice that Nova skips that specific host as part of its scheduling algorithm. On the other hand, if the hypervisor had sufficient CPU capacity but no network connectivity – what would have happened? This time you will see that Nova continues to schedule the virtual machine instance on that host and the VM does not get network connectivity.

 

While compute, storage and networking are considered as the three pillars of OpenStack, the default algorithm for scheduling a VM instance does not consider Network health. This blog shows how Nova scheduler can be enhanced to leverage the Juniper Neutron plugin and ensure that VM placement is Network-aware.

 

Adding Network Filter to Nova Scheduler

 

The default scheduler in Nova is Filter-Scheduler. The Filter-Scheduler takes a list of hypervisors and passes them through a series of filter criteria, e.g. Available RAM, CPU, Storage etc. If a hypervisor does not meet the necessary criteria to host a VM, Nova eliminates that hypervisor.

 

The scheduler then assigns weights to the filtered list of hypervisors according to the free resources on the hypervisor. Finally the hypervisor with the highest weight is selected to host the virtual machine instance.

 

Slide2.jpg 

We will add Filter extensions to the default Nova scheduler so that physical network health can be taken into account while selecting or eliminating a hypervisor.

 

Here is a simple Network Health Filter class that filters out hypervisors if the link to the top-of-the-rack (TOR) switch is down. Nova Scheduler calls the host_passes method to determine if a hypervisor meets the criteria to host a VM or not.

 

 

Network-filter.py

=================

 

from oslo_config import cfg

from oslo_log import log as logging

 

from nova.i18n import _LW

from nova.scheduler import filters

from nova.scheduler.filters import utils

 

LOG = logging.getLogger(__name__)

 

 

class NetworkHealthFilter(filters.BaseHostFilter):

   """Network Filter for scheduling VM on Hypervisors with

   appropriate networking resources and capabilities"""

 

   def host_passes(self, host_state, filter_properties):

       host_ip = host_state.host_ip

       request_spec = filter_properties['request_spec']

       instance_uuid = request_spec['instance_properties']['uuid']

 

       # Get the list of virtual networks requested for this instance.

       network_ids = self.get_requested_network(instance_uuid)

 

       for network_id in network_ids:           

           # Get the physical network device and port associated

           # with each virtual network.

           nic = self.get_host_nic(host_ip, network_id)

           switch, port = self.get_switch_port(host_ip, nic)

 

           # Get the physical network link status and filter out

           # hypervisors where physical link is down.

           if not self.get_network_link_state(switch, port):

               return False

       return True

 

   def get_network_link_state(self, switch, port):

       """Get link status for a switch port."""

 

       link_state_up = False

 

       # (1) Get the network link status.

       ....

       ....

 

       return link_state_up

 

   def get_switch_port(self, host_ip, nic):

       """Get the network switch and port a host is connecte.

       This information can be obtained from neutron server

       with juniper plugin"""

 

       switch = None

       port = None

 

       # (2) Get hypervisor to switch port connectivity from physical

       # topology information

       ....

       ....

 

       return switch, port

 

   def get_requested_network(self, instance_id):

       """return the networks requested by this instance"""

 

       network_ids = []

       # (3) Get requested networks for this instance from nova

       ....

       ....

 

       return network_ids

 

   def get_host_nic(self, host_ip, network_id):

       """Return the NIC for a virtual network"""

 

       nic = None

       # (4) Get Hypervisor NIC for a virtual network from Neutron

       ....

       ....

 

       return nic

 

The above code showed a simple use case where Port status was used to ensure that virtual machines are placed on hypervisor that has network connectivity. The get_host_nic and get_switch_port methods rely on the Network topology information available in the Juniper Networks Neutron plugin database.

 

Resilient to failures

 

Enhancing the Nova scheduler can help in solving some complex use cases. Here is an example where two virtual machines are required to provide fault tolerance. With the knowledge of Physical Network Topology, the enhanced Nova scheduler can place the VMs in two different Network paths making them resilient to edge switch failure.

 

Slide1.jpg

Leverage the intelligent network

 

Network aware placement of workloads can use the advanced capabilities supported by Juniper Networks platforms. Simple metrics like bandwidth utilization can be taken into account while selecting a host. More intelligent placement of VM is possible by leveraging the Cloud Analytics Engine (CAE) and Network Director (ND). With CAE and ND, the Network Health Filter can use network latency, faults and the full datacenter topology information. This will ensure that the end-to-end connectivity between virtual machines is also optimal.

 

Conclusion

 

As shown in this blog it is possible to customize OpenStack Nova scheduler to make use of Network health and place the virtual machine instances so that it has the right networking resources in addition to compute and storage. In the next blog of the series we will discuss an elegant way by which Nova can access the physical network topology that Neutron discovers.

Comments
by Silvia
on ‎07-14-2015 08:32 AM

Hi Chandan,

I read your post and I'm interested on your filter. I have few questions for you:

first of all I would like to know how you get the link status. Do you perform a test and register the result somewhere?

to get the information about the physical network you used the juniper plugin. What about using an SDN controller?

I would like to implement a similar filter catching the network state information by OpenDayLight.

Can you give me some hints to better understand how to do? Thanks

Silvia

 

Announcements
Juniper Networks Technical Books
About the Author
  • Anil Lohiya is a Principal Engineer in the Campus and Data Center Business unit in Juniper Networks. In his current role, he is leading some of the SDN and Network Virtualization initiatives.
  • I am an Engineer with expertise in Data Packet Forwarding, Software Design & Programming with major domain expertise in QoS (Quality of Services). I have worked across the domains in Data communications field. I love water and am a good swimmer too.
  • Remarkably organized stardust. https://google.com/+JamesKelly
  • I have been in the networking industry for over 35 years: PBXs, SNA, Muxes, ATM, routers, switches, optical - I've seen it all. Twelve years in the US, over 25 in Europe, at companies like AT&T, IBM, Bay Networks, Nortel Networks and Dimension Data. Since 2007 I have been at Juniper, focusing on solutions and services: solving business problems via products and projects. Our market is characterized by amazing technological innovations, but technology is no use if you cannot get it to work and keep it working. That is why services are so exciting: this is where the technology moves out of the glossy brochures and into the real world! Follow me on Twitter: @JoeAtJuniper For more about me, go to my LinkedIn profile: http://fr.linkedin.com/pub/joe-robertson/0/4a/34a
  • Ken Briley is Data Center TME at Juniper Networks focused on Juniper switching product lines. Prior to Juniper Networks, Ken worked at Cumulus Networks as a TME supporting the dis-aggregation movement and before that he spent 15 years at Cisco Systems working in various roles: Technical Support, Technical Marketing Engineer, Network Consulting Engineer and Product Management. Ken has an MS in Electrical Engineering and is CCIE # 9754.
  • Michael Pergament, JNCIE-SP #510, JNCIE-ENT #23, JNCIP-SEC
  • Raj is a Sr. Cloud Technology Architect with Juniper Networks and focuses on technologies such as VMware, SDN, and OpenStack etc.
  • Rakesh Dubey is the engineering head for Campus and Data Center business unit at Juniper Networks. He has been with Juniper for past six years leading multiple switching products.
  • Sarath Chandra Mekala is a staff engineer with Juniper networks and focuses on implementing Juniper's Openstack Neutron plugins in the areas of Switching, Routing, Firewall and VPN. He is an official contributor to Openstack Neutron FWaaS v2.
  • Sriram is a Sr. Manager in the Campus and Datacenter Business Unit. He is part of the Network Director team and focuses on technologies such as VMware integration, OpenStack etc.