Getting ESP to work in a load balanced Active/Active pair

[vbroadwater]

Good morning, experts! I've got a weird one today and I don't know if it's on the SSL VPN side or the load balancer side. Basically, we've got a pair of SA4000s in Active/Active mode being load balanced by a Cisco CSS. For users using Network Connect, ESP mode has never worked and it's always fallen back to SSL. I finally started looking into it recently.

I've got content rules on the load balancer for tcp 443 and tcp 80 (in case a user forgets to use https), and one for udp 4500.

For the tcp rules, everything works as it should, no problem. Traffic coming back from the IVEs is sourced as the load balanced VIP as it should. But then it gets weird.

For the udp rule, it seems to direct traffic traffic to one of the devices just fine, however the return traffic is sourced from the device itself instead of the VIP. This breaks any attempt to use ESP since the end user workstation is trying to talk to 1.1.1.2 and is hearing back from 1.1.2.2 instead.

This ONLY happens for the udp traffic... Anything I might have missed? Maybe even a setting on the IVEs themselves?



Thanks!

[drf]

Hey did you ever resolve this? We are having a similar problem with our F5 BigIP. NC always uses SSL instead of ESP. Without the load balancer it works fine with ESP.

[vbroadwater]

Nope, no resolution yet.  It's interesting though that you're having the same issue with a different load balancer.  Surely there's someone out there with active/active setup that works as it should, right? 

[Tiger56]

what kind of stickyness you used? is ssl terminated on CSS? if not i do not think you can make L7 decisions.  did you try src NAT for stickyness?

[drf]

I think we have fixed it now. It was a configuration problem with our persistence (sticky) settings with is based on source IP address. I'll have to run some more tests to make sure.

[vbroadwater]

Were you able to confirm if your sticky setting change fixed the problem?

Here's how we currently have the load balancing config set up.  Any thoughts on what may be wrong here would be greatly appreciated! 

  content http://vpn.blahblah.com

   vip address 1.1.1.1

   add service SSLVPN1_2.1

   add service SSLVPN2_2.2

   advanced-balance sticky-srcip
    port 80
    protocol tcp
    active

  content https://vpn.blahblah.com
    vip address 1.1.1.1
    port 443
    protocol tcp
    add service SSLVPN1_2.1_SSL
    add service SSLVPN2_2.2_SSL
    advanced-balance sticky-srcip
    active

  content vpn.blahblah.com_UDP4500
    vip address 1.1.1.1

   add service SSLVPN1_2.1_UDP4500
    add service SSLVPN2_2.2_UDP4500
    port 4500
    protocol udp
    advanced-balance sticky-srcip-dstport
    balance srcip
    sticky-mask 255.255.255.0
    active
 

And the service configs are a real basic:

service SSLVPNx_x.x
  ip address 1.1.2.1  protocol tcp
  port 80
  keepalive type tcp
  keepalive port 80
  active

or

service SSLVPNx_x.x_SSL
  ip address 1.1.2.1
  protocol tcp
  port 443
  keepalive type ssl
  keepalive port 443
  active

or

service SSLVPNx_x.x_UDP4500
  ip address 1.1.2.1

  protocol udp
  port 4500
  active
 

[looking4-2help]

Had the same issue when trying with our CSS, switched to A/P cluster.  UDP is not intercepted and processed by the CSS the same as TCP because it is a stateless and connection less protocol.  We could not get out CSS to maintain a UDP "connection" table for the UDP connections like it does with the TCP conncection.  The CSS documentation describes a way to do this but we could not get it to work with our configuration.  If you use a service group on the CSS, then the IP of all requests to the SA will be the CSS IP becuase it NATs the client IP to itself and forwards the packet onto the selected SA.  Maybe then the UDP packets will come back to the CSS and then be returned to the client instead of the SA trying to communicate directly back to the client.  The key is that the CSS must register and monitor client connection mappings for UDP.

[Nereida]

Hello,

the problem in F% can be colved configuring this option:

Local Traffic -> Virtual Servers -> Profiles -> Persistence -> Source Address -> Match Across Virtual Servers

This matches across TCP and UDP Virtual Servers. Remember to configure UDP 4500 server as UDP protocol... this is sometimes forgotten....

Bye.

[MattS]

If the response is sourced from the SA IP (1.1.2.2) rather than the VIP address (1.1.12) then it sounds like the load balancer is doing half-NAT (preserving the client's source IP) and the SA is responding back to the client, by-passing the load balancer.   If you set the SA to use the load balancer as the default gateway does it then work as expected?   The other option is to enable full NAT for the UDP port 4500 traffic so the SAs see the source IP as the load balancer.

The binding of the persistence across the port 443 and port 4500 VIP is necessary to make sure the same SA processes all the connections from the same client, otherwise the NC connection could be sent to the other SA which has no record of the user session and would reject it, leading to the client falling back to SSL (port 443) which would go through the port 443 VIP and match the sticky entry previously created when the client originally logged in, allowing the SSL NC to be accepted as it is on the SA which has the authorized user session.