Automation & Programmability
Automation & Programmability
Continuous Integration for OpenNTI Collector without any Junos Device
09.13.16

Continuous integration (CI) is a software engineering practice in which discrete, isolated changes are immediately tested, logged and added to a larger code base, providing a detailed record of changes so that, if a defect is introduced into the code base, it can be quickly identified and corrected.

 

This web site provides a nice overview of major CI benefits, which include:

  • Eliminating long and tense integration cycles
  • Increasing visibility to enable greater communication
  • Quickly detecting issues to **bleep** them in the bud
  • Spending less time debugging and more time adding features
  • Confidently building a solid foundation
  • Rapidly determining whether code is going to work
  • Reducing integration problems, resulting in faster software delivery

 

In this blog, I will describe the various methods Juniper uses as part of our CI effort for the OpenNTI project. There are many solutions available to create a continuous integration server. As our project is public and hosted on Github, we decided to use Travis CI (https://travis-ci.org). GitHub is very well integrated with Travis CI. Each time an event happening on Github (commit, pull request), Github will automatically notify Travis-CI.

 

GitHubServices.jpg

Once a change is committed, Travis CI will follow a set of instructions defined in the .travis.yml file, running the defined actions on compute resources hosted by Travis CI. In our case, it will do following:

  • Setup Ubuntu OS with Docker running on it
  • Download and build OpenNTI container
  • Download dgarros/tcpreplay container
  • Install the required Python modules
  • Run pre-defined tests 

All tests in the projects are located in the tests/ directory and are leveraging the pytest framework (http://pytest.org/latest/).

Pytest represents a collection of pre-defined tests, each of which returns either a PASS or FAIL result. The following tests are executed to verify each and every component of the OpenNTI collector is fully operational:

  1. Verify Docker is running
  2. Verify that OpenNTI Docker container can be started
  3. Verify connection to InfluxDB can be established
  4. Verify a pre-defined “juniper” database exists on InfluxDB
  5. Verify the Data Collection Agent (Netconf over SSH) can gather data from emulated Junos device (mocked device) and write them into InfluxDB
  6. Verify that correct entries were written in step 5 by querying the database
  7. Verify tcpreplay Docker container can be started and stream JTI binary data from tcpreplay Docker container to OpenNTI container
  8. Verify that correct entries were written in step 7 by querying the database
  9. Verify an HTTP connection can be established to Grafana service and upload Grafana dashboard using RESTful API
  10. Tear down all Docker containers

It’s possible to run all tests locally as well by running the script “build-and-test.sh” located on the root directory of the project. This test infrastructure and the integration with Travis-CI enable us to make changes to the project without a high risk of breaking something. It also empowers everyone to contribute to the project by giving them the ability to test their changes. As we discover new defect, more tests will be created to improve the test coverage making the test infrastructure more robust overtime.

 

To be able to test OpenNTI without devices, neither physical nor virtual, we had to find a solution to emulate all communications to/from Junos devices. We selected 2 solutions, one for each type of communication:

  • Tcpreplay to simulate data streamed by devices (https://github.com/dgarros/docker-tcpreplay): Packets captured from a real Junos-based device are used; the destination IP is rewritten as a broadcast IP while the destination MAC is rewritten as a multicast MAC. Anything streamed by the Tcpreplay container will be received by an OpenNTI Data Streaming Agent. Provided the FluentD parser works as expected, the contents of all packets can be inspected by checking data in InfluxDB.
  • Mocked Junos Device (https://github.com/GIC-de/Juniper-PyEZ-Unit-Testing): Without going into the details of how it’s done, this mocked Junos device, which behaves just like a real one, allows you to use the same code as the Data Collection Agent and will return same XML tree back to the caller as a real Junos device.

TravisOpenNTI.jpg

As an example let us do a major upgrade of the database. Following steps are invloved to accomplish it:

  1. Update the Dockerfile with the new version in a dedicated branch
  2. Create a pull request on Github to propose this change to the main branch

Once the pull request is created Travis-CI will be notified and will automatically start to build OpenNTI with the new version of the database and will execute all tests. It is possible to follow Travis-CI progress directly from the pull request on Github. At first the pull request will show  progress status in Yellow to indicate that testing is running.

Screen Shot 2016-03-12 at 7.06.49 PM.png

 

After few minutes, test results will be reported on Github:

Screen Shot 2016-03-12 at 7.14.30 PM.png

Once the pull request is merged into the main branch, Docker Hub will be notified and will automatically rebuild OpenNTI and make it available to everyone.

Screen Shot 2016-03-12 at 7.21.04 PM.png

 

You will get a summary of the build process history on Travis CI:

TravisOutput.jpg

Multiple actions can be associated with the result of the build process. One of these can be sending an email to pre-defined list of recipients:

TravisEmail.jpg

Continuous integration is a key in delivering great customer experience. Through reliable, low-risk releases, Continuous Delivery makes it possible to continuously adapt software in line with user feedback, shifts in the market and changes to business strategy. Test, support, development and operations work together as one delivery team to automate and streamline the build, test and release process.

 

Read more on OpenNTI: http://forums.juniper.net/t5/Analytics/Open-Source-Universal-Telemetry-Collector-for-Junos/ba-p/2886...

0 Kudos
09.13.16
Jose_S

Hello Michael,

we would like to install and test this suite in order to monitor 5 QFX switches.

 

I'm interesting on getting information about server HW requirements to install and run NTI. Is there any minimum HW requirements to be used?. What about virtualizing such a monitoring server?

 

What's your experience in terms of server needs?

 

Thank in advance and best regards, Jose

Top Kudoed Authors