Automation & Programmability
Showing results for 
Search instead for 
Do you mean 

Continuous Integration for OpenNTI Collector without any Junos Device

by Distinguished Expert Distinguished Expert ‎03-14-2016 01:56 AM - edited ‎03-15-2016 11:18 AM

Continuous integration (CI) is a software engineering practice in which discrete, isolated changes are immediately tested, logged and added to a larger code base, providing a detailed record of changes so that, if a defect is introduced into the code base, it can be quickly identified and corrected.

 

This web site provides a nice overview of major CI benefits, which include:

  • Eliminating long and tense integration cycles
  • Increasing visibility to enable greater communication
  • Quickly detecting issues to **bleep** them in the bud
  • Spending less time debugging and more time adding features
  • Confidently building a solid foundation
  • Rapidly determining whether code is going to work
  • Reducing integration problems, resulting in faster software delivery

 

In this blog, I will describe the various methods Juniper uses as part of our CI effort for the OpenNTI project. There are many solutions available to create a continuous integration server. As our project is public and hosted on Github, we decided to use Travis CI (https://travis-ci.org). GitHub is very well integrated with Travis CI. Each time an event happening on Github (commit, pull request), Github will automatically notify Travis-CI.

 

GitHubServices.jpg

Once a change is committed, Travis CI will follow a set of instructions defined in the .travis.yml file, running the defined actions on compute resources hosted by Travis CI. In our case, it will do following:

  • Setup Ubuntu OS with Docker running on it
  • Download and build OpenNTI container
  • Download dgarros/tcpreplay container
  • Install the required Python modules
  • Run pre-defined tests 

All tests in the projects are located in the tests/ directory and are leveraging the pytest framework (http://pytest.org/latest/).

Pytest represents a collection of pre-defined tests, each of which returns either a PASS or FAIL result. The following tests are executed to verify each and every component of the OpenNTI collector is fully operational:

  1. Verify Docker is running
  2. Verify that OpenNTI Docker container can be started
  3. Verify connection to InfluxDB can be established
  4. Verify a pre-defined “juniper” database exists on InfluxDB
  5. Verify the Data Collection Agent (Netconf over SSH) can gather data from emulated Junos device (mocked device) and write them into InfluxDB
  6. Verify that correct entries were written in step 5 by querying the database
  7. Verify tcpreplay Docker container can be started and stream JTI binary data from tcpreplay Docker container to OpenNTI container
  8. Verify that correct entries were written in step 7 by querying the database
  9. Verify an HTTP connection can be established to Grafana service and upload Grafana dashboard using RESTful API
  10. Tear down all Docker containers

It’s possible to run all tests locally as well by running the script “build-and-test.sh” located on the root directory of the project. This test infrastructure and the integration with Travis-CI enable us to make changes to the project without a high risk of breaking something. It also empowers everyone to contribute to the project by giving them the ability to test their changes. As we discover new defect, more tests will be created to improve the test coverage making the test infrastructure more robust overtime.

 

To be able to test OpenNTI without devices, neither physical nor virtual, we had to find a solution to emulate all communications to/from Junos devices. We selected 2 solutions, one for each type of communication:

  • Tcpreplay to simulate data streamed by devices (https://github.com/dgarros/docker-tcpreplay): Packets captured from a real Junos-based device are used; the destination IP is rewritten as a broadcast IP while the destination MAC is rewritten as a multicast MAC. Anything streamed by the Tcpreplay container will be received by an OpenNTI Data Streaming Agent. Provided the FluentD parser works as expected, the contents of all packets can be inspected by checking data in InfluxDB.
  • Mocked Junos Device (https://github.com/GIC-de/Juniper-PyEZ-Unit-Testing): Without going into the details of how it’s done, this mocked Junos device, which behaves just like a real one, allows you to use the same code as the Data Collection Agent and will return same XML tree back to the caller as a real Junos device.

TravisOpenNTI.jpg

As an example let us do a major upgrade of the database. Following steps are invloved to accomplish it:

  1. Update the Dockerfile with the new version in a dedicated branch
  2. Create a pull request on Github to propose this change to the main branch

Once the pull request is created Travis-CI will be notified and will automatically start to build OpenNTI with the new version of the database and will execute all tests. It is possible to follow Travis-CI progress directly from the pull request on Github. At first the pull request will show  progress status in Yellow to indicate that testing is running.

Screen Shot 2016-03-12 at 7.06.49 PM.png

 

After few minutes, test results will be reported on Github:

Screen Shot 2016-03-12 at 7.14.30 PM.png

Once the pull request is merged into the main branch, Docker Hub will be notified and will automatically rebuild OpenNTI and make it available to everyone.

Screen Shot 2016-03-12 at 7.21.04 PM.png

 

You will get a summary of the build process history on Travis CI:

TravisOutput.jpg

Multiple actions can be associated with the result of the build process. One of these can be sending an email to pre-defined list of recipients:

TravisEmail.jpg

Continuous integration is a key in delivering great customer experience. Through reliable, low-risk releases, Continuous Delivery makes it possible to continuously adapt software in line with user feedback, shifts in the market and changes to business strategy. Test, support, development and operations work together as one delivery team to automate and streamline the build, test and release process.

 

Read more on OpenNTI: http://forums.juniper.net/t5/Analytics/Open-Source-Universal-Telemetry-Collector-for-Junos/ba-p/2886...

Comments
by Jose_S
on ‎09-13-2016 09:20 AM

Hello Michael,

we would like to install and test this suite in order to monitor 5 QFX switches.

 

I'm interesting on getting information about server HW requirements to install and run NTI. Is there any minimum HW requirements to be used?. What about virtualizing such a monitoring server?

 

What's your experience in terms of server needs?

 

Thank in advance and best regards, Jose

Announcements

Juniper Design & Architecture Center - Mobile Cloud
About the Author
  • Ben has been working with service providers around the world for the last 15 years developing business cases for a variety of product concepts and new ventures. Ben holds an MBA from MIT and a BS & MS in Mechanical Engineering from Johns Hopkins University.
  • Part of Juniper PS EMEA since 2005 Primarily interested in making technology do the boring repetitive work so I can do fun new work.
  • Donyel Jones-Williams is the Director of Service Provider Product Marketing Management overseeing all of Juniper's Service Provider Products for Juniper Networks. In this role, he leads all of the internal and external marketing activities for Juniper with respect to routing, automation, SDN and NFV. Prior to joining Juniper Networks in January 2014, Donyel was a Senior Product Line Manager for Cisco Systems with in the High End Optical Routing Group managing product lifecycle for multiple products lines helping telecom providers operate efficiently and effectively including; ONS 155xx Product Family, ONS 15216, ONS 15454 MSTP, Carrier Packet Transport Product Family, ME 2600x, & ASR 9000v. He also negotiated favorable agreements with 3rd-party vendors furnishing components and parts and conducted both outbound and inbound marketing (webinars, case study-development, developed and delivered both business & technical at Cisco Live 2005-2012). Donyel graduated from California Polytechnic State University-San Luis Obispo with a Bachelor of Science in Computer Science. While attending Cal Poly SLO he was a collegiate student athlete playing football as a wide receiver and a key member of the National Society of Black Engineers. Donyel is now an active volunteer for V Foundation.
  • Ebben Aries is a Principal Engineer for Junos Manageability in the Juniper Development and Innovation Division.
  • Michael Pergament, JNCIE-SP #510, JNCIE-ENT #23, JNCIP-SEC
  • Pallavi Mahajan is Vice President Engineering, Junos Engineering, and leads the Junos Programmability & Automation teams
  • Product Manager, JUNOS Automation