01-04-2012 10:50 AM
My company bought 7 Juniper EX3200 series switches to be deployed at our new DR site. Our company policy is to completely isolate management traffic from regular data traffic. In this regard, a dedicated out-of-band management interface is the preferred way to remotely mange any device.
My ordeal began as soon as I started setting up the switches. The advertised "out-of-band management" interface was simply not working as it should. In fact, there was no way to set up a separate default-gateway to the management interface. As I started looking into it, I was taken back when I realized that the management interface was in fact sharing the same routing instance as the regular copper interfaces.
JTAC was of no support in this regard. They kept telling me that it is not possible to route between the management interface and the copper interfaces. I don't have a problem with it. In fact you should NEVER be able to pass traffic between the management interface and the copper interfaces. In this regard, the routing instance should NOT be shared between the management interface and the copper interfaces as it negates the whole framework. I spoke with two JTAC personnel on this issue, who did not seem to understand the concept of out-of-band management in the first place.
So, has anyone successfully implemented true out-of-band management on EX series switches? Can the Juniper switches support this feature at all?
01-04-2012 03:41 PM - edited 01-04-2012 03:42 PM
I have never used the EX switches but I have used many SRXs which support full out-of-band management (which is sometimes a pain with clusters). I would be surprised if the EX didn't support it too.
01-04-2012 06:39 PM
I do believe it is true that traffic can't go from a front-facing port and be routed / switched to the me0 port, but it's sharing the same default routing instance is a real show-stopper.
If the EX is functioning solely as a dumb L2 switch then its not that big of a deal (no routing occurring in inet.0), but if you're doing routing on it, it's a serious pain.
all they need to do is let us put me0 / fxp0 into a routing-instance and then its a non-issue. I can't really see how this could be hard to implement
Anyway, I think you have two options here for out of band management with 'correct' functionality --
1) Borrow a front-facing port, and place it into its own routing instance by itself.
the routing tables would be separate and can manage it through that single port.
2) use me0, but place all other L3 routing interfaces into separate routing instance(s). that way inet.0 only has the default route out me0 for management purposes, and then the VR(s) have the regular routing configurations.
downside to option 2 is that you could (in theory) run into some corner case where a feature is supported in the default instance, but not in a VR -- I can't think of any such feature off-hand for the EX, but you never know.
01-05-2012 07:28 AM
The EX switch is indeed doing routing. Infant, the switches are participating in OSPF.
A little background - From our HQ we have two IPSec VPNs to the DR site. One VPN tunnel carries all data traffic and the other VPN tunnel is used to access the OOB network. There are lot of devices on the OOB network including other network hardware and servers accessible via IPMI (iLOMs, iDRACs etc.). We can access all the other devices EXCEPT the EX3200 switches.
Because the routing instance is being shared, and the switch is participating in OSPF, the front routing interfaces know the path back to the HQ via the copper ports. So, I try to SSH into the switch's Management interface, the switch receives my initial SYN packet, and responds with a SYN, ACK but sends this packet out via the front copper interfaces with a source address of the management interface.
I have verified this behavior with wireshark packet captures and my firewall logs. (firewall logs this behavior as suspicious and drops the SYN, ACK as the initial SYN packet was not handled by it)
Contacting JTAC for assistance on this issue was a complete waste of time. The person I spoke with had no idea about OOB framework and why it is important. In fact he was down right condescending and told me I did not understand the concept of a dedicated management interface. He wouldn't listen to me and everytime I tried to explain what I was trying to do, he just responded back with canned responses. I have a feeling he was just reading back from some sort of technical document about what the management interface can and cannot be used for.
At this point the best solution seems to be from "wimclend", that is to assign one of the copper interfaces to a separate routing instance and use that to manage the switch. But is is surprising that Juniper is not supporting this feature which has been around for quite some time now.
Having a dedicated OOB interface was one of the reasons we choose the EX3200. As of writing this post, I am using a policy based NAT to manage the EX3200 to make it seem like I am managing them from the same subnet as the OOB network.
This has been a setback for us and we had to push back our live dates by a couple of days. However, I don't think Juniper is going to be on the table the next time we make a purchase or an upgrade.
01-05-2012 11:28 AM
My name is Alper and I'm with the EX support team. I would like to take a look at the cases you opened with JTAC. Can you please let me know the case numbers?
01-07-2012 07:16 AM
Don't forget there still is a big difference between using the management port and using a regular switch port for management. Using one of the regular switch ports you'll have to explicitly disable it every time you configure a protocol (like STP or whatever). We tried simulating out-of-band that way on another vendors switch which didn't even have a management port, it was a major PITA.
I'd just stick with NAT - its the easiest solution and prevents a lot of other potential issues.
Sure, everyone would like a separate management routing instance, thats probably one of the oldest feature requests. But they do have a point, the reason we run into this issue is because our OOB networks are not truly separate.
01-12-2012 02:49 AM
Yes I agree that the me interface been to be by default in a seperate routing instance.
We have also had to deploy NAT on the gateway routers facing the me interfaces.
As we have these boxes dotted all over the place this a pain in the ***.
Specifically as you dont want you internal mgmt network routes inside the inet.0 table.
the only other thing we do is set static routes to point to the management subnet out of the me interface and mark these statics with the no-advertise flag so they are not propogated to neighbors.