This is a guest blog post. Views expressed in this post are original thoughts posted by Martin Brown, Network Engineer from Morrisons PLC. These views are his own and in no way do they represent the views of the company he works for.
There comes a time in every network engineer’s career when, after a possible issue is diagnosed, that engineer knows he needs to test his hypothesis—even if a slight chance exists that said test could have a disastrous effect on the network.
To follow the ITIL framework, he knows he must raise a change request before he even starts. Of course, this is knowing full well that if things were to go very wrong, it could possibly affect his career. Or worse—give his colleagues reason to ridicule him for the next week.
Let’s face it. You really don't want to be that guy.
A Great Option: The Test Lab
Happily, there is another option. One that will allow network engineers to play without the risks. In our little world, we call these "Test Labs." The only issue with these is they can take up valuable real estate in a rack “somewhere” and they do require you to sacrifice kit that could be used for the live network.
In addition, sometimes kit is borrowed and using it for the sole purpose of seeing if you can break it is no longer an option. Those lucky enough to have an organization that recognizes the importance of this environment will understand the important part these labs play in the day-to-day running of the network. And I, for one, realize how fortunate I am to not only have such an environment, but to also to have the opportunity to be able to expand it.
When Building Your Test Lab
Let it be known that the test lab I use was, in fact, created by persons better than I. My role has just been to resurrect certain
parts and then add some new equipment. The story about the lab’s creation, though, is certainly worth telling as it’s about
how the lab is built and why that’s important.
The first issue you may face when you build a lab is how to separate it from the live network so that if you, say, advertised conflicting routes using OSPF, it wouldn’t accidentally be advertised into the live network.
At the same time, however, you might wonder how to access the lab if it’s installed in a comm. room some considerable distance from where you would be accessing it. Like us, perhaps at first you’d consider something like a firewall to protect the network whilst you accessed your firewalls. But, no. As we realized, that really didn’t make sense and we knew we needed another solution.
Striking “Gold” with Spare Parts
As luck would have it (at least in our case) a little while previously the VPN appliance had been upgraded, so we happened to have a spare SA-2500. This was perfect as it meant we could not only test VPN connections, but we could segment the network and be able to remotely access the lab without fear of causing any issues on the live network. As it happened, we also had a spare IDP-75, so we decided we may as well throw this in there also, just because, well . . . because we could.
As the weeks passed, more and more kit was added, including an SSG5, an EX4200 Virtual Chassis, a few routers, WLCs, switches and firewalls from other vendors. Then we struck “gold.” Someone found an unused SRX240 in a closet somewhere. We decided to seize upon this opportunity and no sooner had we done so and gold struck twice. Someone else found another one in storage, again forgotten about and unused.
What do two SRX firewalls make? A pair, and a pair can make a cluster. Sure, these were only SRX240H and not the newer H2, but as this wouldn’t be coping with thousands of sessions, that didn’t matter.
This new addition to the lab was perfect as, if you recall from my previous blog, I’d just created an SRX cluster for a new network project and using the knowledge I’d gained from building those, I quickly had this cluster up and running. This meant we could now test configurations in a safe environment before pushing it out to the live site without bringing the remote site down. This is what’s known as perfect timing.
Once this was racked up, I added the necessary configuration and connected the “trusted” zone to the Virtual Chassis. I
was going to create a separate VLAN on the EX4200s and connect this to the “untrusted” zone, but I really wanted a separate physical switch. This lab had mainly been built using forgotten and unused equipment and I didn’t really want to use a switch that was a spare for somewhere else. As luck would have it, a colleague found an EX2200 labelled as “faulty.” We thought it better to find out why it wasn’t working and see if we could repair it.
Powering up the EX2200 soon revealed, firstly, that the power supply was giving the switch power and, secondly, that the issue was that the Junos installation had simply become corrupted. This wasn’t exactly “faulty” and, in reality, it just needed some TLC. Searching through our software library I found the backup of Junos for the EX2200 and all I needed to do was put it onto my USB stick, place it in the back of the switch and perform the recovery procedure. Perfect.
We now had an expanded lab and, more importantly, it only cost the time and effort in putting it together. In addition, there was no danger of any of it being borrowed as it had been superseded. That said, even though the SRX wasn’t the H2 and the EX2200 probably had seen better days, the tests we could perform on them were still very valid. It is, after all, still One Junos.