03-29-2012 10:23 AM
I am put with the task to find out the best way to load balance IP traffic across 2 wireless connections and am looking for information on setting it up. Our customer requested a full 300mbps connection into a rural area that we can get to with a wireless link. The problem is that the wireless gear we are going to use can only run at 174mbps each way. So to get a full 300mbps we need to run 2 separate wireless links and then join them with a load balancing device at each end. I am a little unsure if we should use a layer2 or a layer3 device or something that can do both? I am pretty sure the customer is going to connect to it with a WAN connection so 1 IP on each side and only 1 MAC address on each side, so I would need the load balancing device to utilize both links evenly without looking at MAC or IP addresses for doing it. Just at the packet level so the customer can have a full 300mbps connection without knowing that it goes across 2 wireless links.
Any information about what devices I should be looking at and possibly some info on configuring this set up would be a great help.
03-30-2012 09:27 AM
We use Cambium (used to be motorola) PTP 800's that are layer 3 devices and do have vlan support. But we tend to just run them in bridge mode so they act as a transparent device to the switches on each end just like a cat 5 cable would. If doing a Vlan across the links or IP routing to do the load balancing I could look into that?
03-30-2012 10:01 AM - edited 03-30-2012 10:03 AM
It's important to carefully consider your needs here. You already mentioned 1 IP and 1 MAC, so the next question is ... 1 Port?
With an EX switch, agreegation on L2 with LACP will give you load balancing based on a hash determined out of values up to L4 (port numbers) for IP traffic, and L3 load balancing happens at most on per-flow basis (which is more or less the same thing). So, if you have, for example, a single connection to download a file or something, it will never be faster than the speed of one of your wireless links.
So, if you're looking for true per-packet load balancing, then an EX switch won't do the job. At the same time, though, please consider that with true per-packet load balancing you may start getting out of order packets on the other end which may or may not be a small or huge problem for some or many applications. I know it'd be a problem if your customer tries to run a VPN (which generally frowns at out-of-order packets).
Looking at the PTP800 specs, though, it seems like you can assign asymmtric bandwidth for upload and download. So, if the device can do 170mbps full duplex, then you can configure it to do, say, 300mbps download and 40mbps upload on one link and the reverse on the other. Then, all you'd need to do is split inbound and outbound traffic and assign each to a link. I'm not sure if the EX switch would be able to do that on L2, but ... that would actually be something interesting to try out. The downside of this set up, however, is that if one of the two wireless links fails, you're left with 40mbps link instead of 170mbps link or, worse, with no link if you have L2 one-way port that isn't smart enough to switch to full duplex.
03-30-2012 02:10 PM
Thanks for the updates Nikolay,
It would be 1 port connecting in at the customer premise for there connection then 2 ports (1 for each PTP 800) for the wireless shot. Then on our site we need to connect the 2 microwaves back into 1 connections so from the 2 microwave Ethernet ports to the 1 WAN Ethernet port to send to the customers head end. So using 3 ports at each side.
I am aware of the issues with LACP and if there is only 1 IP at each end and 1 MAC at each end and only 1 generated session then they we would never get to use the 2nd microwave link unless we had the first one fail. Then it would act as a failover but we would never see more then 174 Mbps.
I am thinking we are going to have to go ECMP per packet with a router? Possibly a J2320 at each end to do ECMP but I know that because it is a Cable ISP that we are connecting that they will for sure be running multiple VPN's and IP Phones and other services so having issues with packets out of order could be a major issue and a deal breaker.
If we are running 2 J-series routers across equal paths how big of an issue is the packets out of sequence? I know that the PTP 800 tend to be super low latency for 3 to 10 MS but that wouldn't be exacly the same always on both links so the possibility of out of sequence could be a deal breaker? What are your thoughts on this?
As far as the last part unfortunately the PTP 800 series only has symmetrical bandwidth settings so we don't have the option of skewing the links that way. The older PTP 600's do have asymmetric bandwidth but not really configurable to how you are saying just 1-1 2-1 or 1-2 so 50%-50% or 66%-33% or 33%-66% and then they will auto adjust bandwidth depending on how much data is going in each direction at any givin time.
03-31-2012 06:01 PM
I'm not 100% sure, but I don't think the J series will do true per-packet ECMP either. Please double-check. Perhaps you would need to look at routers with ... slightly more green-ish faceplates?
As far as packet order is concerned, I came across an interesting paper (http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.33.1557). So, ideally you'd get a specialzed inverse multiplexer on each side of the microwaves that will do proper load balancing and will handle the associated issues. (See, for example, Mushroom Networks, although their product maxes out at 250mbps).
How bad out-of-order traffic is really depends on the applications that are using the links. Some may be more sensitive than others. Anything that's buffered (e.g. TCP) will survive the occasional out-of-order packet without complaining. On the other hand, I've had an instance where out-of-order packets on a 100mbps link were killing VPNs until we disabled replay protection, after which there were no problems (either the ISP changed something, or applications were able to just handle it).
04-01-2012 05:29 AM
The J-series does support ECMP.
Senior IP Engineer - DQE Communications Pittsburgh, PA (Metro Ethernet & ISP)
JNCIA-Junos JNCIS-SEC JNCIP-SEC JNCSP-SEC
JNCDA JNCDS-DC JNCDS-SEC
ACE PanOS 6 ACE PanOS 7