SRX Services Gateway
Highlighted
SRX Services Gateway

GRE tunnel implementation to Zscaler using multi tunnels and failover between primary and secondary gateway - (gre tunnel loadbalancing and failover)

‎06-05-2018 08:43 AM

Hi

 

In our environment we are using Zscaler as our Cloud Proxy provider.

To connect to Zscaler we use a Juniper SRX1500 Cluster having 4xGRE tunnel in the Primary group and 4xGRE tunnel in the Backup group.

The Primary group is pointing to the Zscaler ZEN in Zurich where the secondary group points to Frankfurt.

 

Important here, is that the primary tunnel group (Zürich) remains active and the secondary tunnel group only is enabled when the Primary goes down.

In our actual configuration we implemented the primary 4xGRE tunnels as well as the secondary 4x GRE tunnels in that manner that Zscaler explains.

 

The problem that we have is like a "loadbalancing" between the primary and secondary tunnels instead of a "failover" in case that the primary goes down.

Zscaler's example on their support page is only using 1 tunnel for the primary as well as 1 for the secondary site.

Because of the limitation of 800MB/sec on a GRE tunnel, we need to bond 4xGRE tunnel to reach more throughput. And there is the problem that we are facing.

We need a loadbalancing between the each 4GRE tunnel to ensure that expected throughput.

 

In the actual setup we have some proxy requests going to Zurich as well as others going to Frankfurt.

One of the main problems is that some of our Websystems have like an GeoIP filter and those can only be accessed from Switzerland (Zscaler Zurich).

Once a connection goes to Frankfurt, the Web contents cannot be seen anymore.

 

Zscaler config example:

https://help.zscaler.com/zia/5.5/gre-configuration-example-juniper-srx

 

 

IP Setup

In our setup we are using 4 public IP's for primary and secondary group:

144.155.100.20_GRE 144.155.100.20/32

144.155.100.21_GRE 144.155.100.21/32

144.155.100.22_GRE 144.155.100.22/32

144.155.100.23_GRE 144.155.100.23/32

 

Tunnel ip's for primary group:

set interfaces gr-0/0/0 unit 1 family inet address 172.17.81.97/30

set interfaces gr-0/0/0 unit 3 family inet address 172.17.21.225/30

set interfaces gr-0/0/0 unit 5 family inet address 172.17.87.97/30

set interfaces gr-0/0/0 unit 7 family inet address 172.17.87.105/30

 

Tunnel ip's for secondary group:

set interfaces gr-0/0/0 unit 0 family inet address 172.17.81.101/30

set interfaces gr-0/0/0 unit 2 family inet address 172.17.21.229/30

set interfaces gr-0/0/0 unit 4 family inet address 172.17.87.101/30

set interfaces gr-0/0/0 unit 6 family inet address 172.17.87.109/30

 

 

The destination IP (Zscaler Gateway's) are as follows:

185.46.100.50 Gateway Zürich (Primary)

165.225.100.50 Gateway Frankfurt (Secondary)

 

set interfaces gr-0/0/0 unit 0 tunnel destination 165.225.100.50

set interfaces gr-0/0/0 unit 2 tunnel destination 165.225.100.50

set interfaces gr-0/0/0 unit 4 tunnel destination 165.225.100.50

set interfaces gr-0/0/0 unit 6 tunnel destination 165.225.100.50

 

set interfaces gr-0/0/0 unit 1 tunnel destination 185.46.100.50

set interfaces gr-0/0/0 unit 3 tunnel destination 185.46.100.50

set interfaces gr-0/0/0 unit 5 tunnel destination 185.46.100.50

set interfaces gr-0/0/0 unit 7 tunnel destination 185.46.100.50

 

Logical interfaces

reth0 - Public side (internet)

reth1 - internal side

reth9 - logging (syslog)

 

 

 

 

Actual Setup on JunOS 15.1X49-D75.5 :

Here the important part of the setup to understand better our issue.

 

Security Zones

set security zones security-zone internet interfaces reth0.2613

set security zones security-zone internal screen global-screen

set security zones security-zone internal host-inbound-traffic system-services ping

set security zones security-zone internal host-inbound-traffic system-services traceroute

set security zones security-zone internal interfaces reth1.2614

set security zones security-zone gre-data address-book address 144.155.100.20_GRE 144.155.100.20/32

set security zones security-zone gre-data address-book address 144.155.100.21_GRE 144.155.100.21/32

set security zones security-zone gre-data address-book address 144.155.100.22_GRE 144.155.100.22/32

set security zones security-zone gre-data address-book address 144.155.100.23_GRE 144.155.100.23/32

set security zones security-zone gre-data screen global-screen

set security zones security-zone gre-data host-inbound-traffic system-services traceroute

set security zones security-zone gre-data host-inbound-traffic system-services ping

set security zones security-zone gre-data interfaces gr-0/0/0.0

set security zones security-zone gre-data interfaces gr-0/0/0.1

set security zones security-zone gre-data interfaces gr-0/0/0.2

set security zones security-zone gre-data interfaces gr-0/0/0.3

set security zones security-zone gre-data interfaces gr-0/0/0.4

set security zones security-zone gre-data interfaces gr-0/0/0.5

set security zones security-zone gre-data interfaces gr-0/0/0.6

set security zones security-zone gre-data interfaces gr-0/0/0.7

 

Policies:

set security policies from-zone internal to-zone gre-data policy internal-to-gre-data_0001 match source-address any

set security policies from-zone internal to-zone gre-data policy internal-to-gre-data_0001 match destination-address any

set security policies from-zone internal to-zone gre-data policy internal-to-gre-data_0001 match application any

set security policies from-zone internal to-zone gre-data policy internal-to-gre-data_0001 then permit

set security policies from-zone internal to-zone gre-data policy internal-to-gre-data_0001 then log session-init

set security policies from-zone internal to-zone gre-data policy internal-to-gre-data_0001 then log session-close

set security policies from-zone internal to-zone internet policy default-allow match source-address any

set security policies from-zone internal to-zone internet policy default-allow match destination-address any

set security policies from-zone internal to-zone internet policy default-allow match application any

set security policies from-zone internal to-zone internet policy default-allow then permit

set security policies from-zone internal to-zone internet policy default-allow then log session-init

 

 

Screening Options:

set security flow tcp-mss all-tcp mss 1350

set security screen ids-option global-screen alarm-without-drop

set security screen ids-option global-screen icmp ip-sweep

set security screen ids-option global-screen icmp fragment

set security screen ids-option global-screen icmp flood threshold 2000

set security screen ids-option global-screen icmp ping-death

set security screen ids-option global-screen ip bad-option

set security screen ids-option global-screen ip spoofing

set security screen ids-option global-screen ip unknown-protocol

set security screen ids-option global-screen ip tear-drop

set security screen ids-option global-screen tcp syn-fin

set security screen ids-option global-screen tcp fin-no-ack

set security screen ids-option global-screen tcp tcp-no-flag

set security screen ids-option global-screen tcp syn-frag

set security screen ids-option global-screen tcp port-scan

set security screen ids-option global-screen tcp syn-flood

set security screen ids-option global-screen tcp land

set security screen ids-option global-screen tcp winnuke

set security screen ids-option global-screen tcp tcp-sweep

 

ALG's are fully disabled.


Interfaces

set interfaces gr-0/0/0 unit 0 description backup-tunnel

set interfaces gr-0/0/0 unit 0 bandwidth 10g

set interfaces gr-0/0/0 unit 0 tunnel source 144.155.100.20

set interfaces gr-0/0/0 unit 0 tunnel destination 165.225.100.50

set interfaces gr-0/0/0 unit 0 family inet mtu 1446

set interfaces gr-0/0/0 unit 0 family inet address 172.17.81.101/30

set interfaces gr-0/0/0 unit 1 description primary-tunnel

set interfaces gr-0/0/0 unit 1 bandwidth 10g

set interfaces gr-0/0/0 unit 1 tunnel source 144.155.100.20

set interfaces gr-0/0/0 unit 1 tunnel destination 185.46.100.50

set interfaces gr-0/0/0 unit 1 family inet mtu 1446

set interfaces gr-0/0/0 unit 1 family inet address 172.17.81.97/30

set interfaces gr-0/0/0 unit 2 description backup-tunnel

set interfaces gr-0/0/0 unit 2 bandwidth 10g

set interfaces gr-0/0/0 unit 2 tunnel source 144.155.100.21

set interfaces gr-0/0/0 unit 2 tunnel destination 165.225.100.50

set interfaces gr-0/0/0 unit 2 family inet mtu 1446

set interfaces gr-0/0/0 unit 2 family inet address 172.17.21.229/30

set interfaces gr-0/0/0 unit 3 description primary-tunnel

set interfaces gr-0/0/0 unit 3 bandwidth 10g

set interfaces gr-0/0/0 unit 3 tunnel source 144.155.100.21

set interfaces gr-0/0/0 unit 3 tunnel destination 185.46.100.50

set interfaces gr-0/0/0 unit 3 family inet mtu 1446

set interfaces gr-0/0/0 unit 3 family inet address 172.17.21.225/30

set interfaces gr-0/0/0 unit 4 description backup-tunnel

set interfaces gr-0/0/0 unit 4 bandwidth 10g

set interfaces gr-0/0/0 unit 4 tunnel source 144.155.100.22

set interfaces gr-0/0/0 unit 4 tunnel destination 165.225.100.50

set interfaces gr-0/0/0 unit 4 family inet mtu 1446

set interfaces gr-0/0/0 unit 4 family inet address 172.17.87.101/30

set interfaces gr-0/0/0 unit 5 description primary-tunnel

set interfaces gr-0/0/0 unit 5 bandwidth 10g

set interfaces gr-0/0/0 unit 5 tunnel source 144.155.100.22

set interfaces gr-0/0/0 unit 5 tunnel destination 185.46.100.50

set interfaces gr-0/0/0 unit 5 family inet mtu 1446

set interfaces gr-0/0/0 unit 5 family inet address 172.17.87.97/30

set interfaces gr-0/0/0 unit 6 description backup-tunnel

set interfaces gr-0/0/0 unit 6 bandwidth 10g

set interfaces gr-0/0/0 unit 6 tunnel source 144.155.100.23

set interfaces gr-0/0/0 unit 6 tunnel destination 165.225.100.50

set interfaces gr-0/0/0 unit 6 family inet mtu 1446

set interfaces gr-0/0/0 unit 6 family inet address 172.17.87.109/30

set interfaces gr-0/0/0 unit 7 description primary-tunnel

set interfaces gr-0/0/0 unit 7 bandwidth 10g

set interfaces gr-0/0/0 unit 7 tunnel source 144.155.100.23

set interfaces gr-0/0/0 unit 7 tunnel destination 185.46.100.50

set interfaces gr-0/0/0 unit 7 family inet mtu 1446

set interfaces gr-0/0/0 unit 7 family inet address 172.17.87.105/30

 

set interfaces ge-0/0/9 gigether-options redundant-parent reth9

set interfaces xe-0/0/18 gigether-options redundant-parent reth0

set interfaces xe-0/0/19 gigether-options redundant-parent reth1

set interfaces ge-7/0/9 gigether-options redundant-parent reth9

set interfaces xe-7/0/18 gigether-options redundant-parent reth0

set interfaces xe-7/0/19 gigether-options redundant-parent reth1

set interfaces fab0 fabric-options member-interfaces ge-0/0/10

set interfaces fab0 fabric-options member-interfaces ge-0/0/11

set interfaces fab1 fabric-options member-interfaces ge-7/0/10

set interfaces fab1 fabric-options member-interfaces ge-7/0/11

set interfaces reth0 vlan-tagging

set interfaces reth0 mtu 9192

set interfaces reth0 redundant-ether-options redundancy-group 1

set interfaces reth0 unit 2613 vlan-id 2613

set interfaces reth0 unit 2613 family inet sampling input

set interfaces reth0 unit 2613 family inet sampling output

set interfaces reth0 unit 2613 family inet address 144.155.100.20/28

set interfaces reth0 unit 2613 family inet address 144.155.100.21/28

set interfaces reth0 unit 2613 family inet address 144.155.100.22/28

set interfaces reth0 unit 2613 family inet address 144.155.100.23/28

set interfaces reth1 vlan-tagging

set interfaces reth1 mtu 9192

set interfaces reth1 redundant-ether-options redundancy-group 1

set interfaces reth1 unit 2614 vlan-id 2614

set interfaces reth1 unit 2614 family inet filter input no-tunnel-gateway

set interfaces reth1 unit 2614 family inet address 10.157.199.65/28

set interfaces reth9 enable

set interfaces reth9 mtu 9192

set interfaces reth9 redundant-ether-options redundancy-group 1

set interfaces reth9 unit 0 family inet address 10.65.99.20/24

 

 

 

 

Routings

set routing-options interface-routes rib-group inet global-rib

set routing-options static route 0.0.0.0/0 next-hop 144.155.139.17

set routing-options static route 10.0.0.0/8 next-hop 10.157.246.209

set routing-options static route 10.157.246.0/28 next-table log-vr.inet.0

set routing-options rib-groups global-rib import-rib inet.0

set routing-options rib-groups global-rib import-rib traffic_tunnel.inet.0

set routing-options forwarding-table export LOAD-BALANCE

set policy-options policy-statement LOAD-BALANCE then load-balance per-packet

 

set routing-instances internal-vr instance-type virtual-router

set routing-instances internal-vr interface reth1.2614

set routing-instances internal-vr routing-options static route 0.0.0.0/0 next-table traffic_tunnel.inet.0

set routing-instances internal-vr routing-options static route 10.0.0.0/8 next-hop 10.157.199.76

set routing-instances internal-vr routing-options static route 144.155.0.0/16 next-hop 10.157.199.76

set routing-instances internal-vr routing-options static route 172.16.0.0/16 next-hop 10.157.199.76

set routing-instances internal-vr routing-options static route 192.168.0.0/16 next-hop 10.157.199.76

 

set routing-instances log-vr instance-type virtual-router

set routing-instances log-vr interface reth9.0

set routing-instances log-vr routing-options static route 10.157.246.0/28 next-hop 10.65.99.1

set routing-instances log-vr routing-options static route 10.157.16.192/27 next-hop 10.65.99.1

 

set routing-instances no-tunnel-gateway instance-type forwarding

set routing-instances no-tunnel-gateway routing-options static route 0.0.0.0/0 next-hop 144.155.139.17

 

set routing-instances traffic_tunnel instance-type forwarding

set routing-instances traffic_tunnel routing-options static route 185.46.100.0/23 qualified-next-hop gr-0/0/0.1

set routing-instances traffic_tunnel routing-options static route 185.46.100.0/23 qualified-next-hop gr-0/0/0.3

set routing-instances traffic_tunnel routing-options static route 185.46.100.0/23 qualified-next-hop gr-0/0/0.5

set routing-instances traffic_tunnel routing-options static route 185.46.100.0/23 qualified-next-hop gr-0/0/0.7

set routing-instances traffic_tunnel routing-options static route 165.225.100.0/22 qualified-next-hop gr-0/0/0.0 preference 200

set routing-instances traffic_tunnel routing-options static route 165.225.100.0/22 qualified-next-hop gr-0/0/0.2 preference 200

set routing-instances traffic_tunnel routing-options static route 165.225.100.0/22 qualified-next-hop gr-0/0/0.4 preference 200

set routing-instances traffic_tunnel routing-options static route 165.225.100.0/22 qualified-next-hop gr-0/0/0.6 preference 200

 

 

 

Our Question:

Can anyone of you explan us how to configure the primary and backup tunnel groups to define a failover between those?

Meaning, that the primary tunnel group must retain as primary and only failover when Zscaler Zurich is not more available (ping monitoring for example). The Example of Zscaler is using a icmp monitoring between the tunnel ip's, which works in a single tunnel configuration, but not in a multitunnel setup where we only need to ping the tunnel group. 

 

Example how it should work:

 

Primary group:

  • Always first group enabled
  • 4xGRE tunnel to Zscaler Zurich
  • loadbalance between 4 tunnel
  • NO traffic to Zscaler Frankfurt

 

Secorndary group:

  • Always backup group and only enabled when Zurich offline (icmp)
  • 4xGRE tunnel to Zscaler Frankfurt
  • loadbalance between 4 tunnel
  • NO traffic to Zscaler Zurich
  • Automatic Failover to primary (Zurich) - Intervall check all 5 minutes (prevent flapping!)

 

Thank's for your help and we hope to find asap a solution.

 

Best Regards

Patrick Vanreck, Mubedin Osmanoski, Peter Howes

3 REPLIES 3
Highlighted
SRX Services Gateway

Re: GRE tunnel implementation to Zscaler using multi tunnels and failover between primary and secondary gateway - (gre tunnel loadbalancing and failover)

‎06-05-2018 07:05 PM
I think enabling Ip-Monitoring for all these 8 IPs will help. Below KB explains How to track two IP addresses in IP monitoring on SRX Branch Series


https://kb.juniper.net/InfoCenter/index?page=content&id=KB24362
Thanks,
Suraj
Please Mark My Solution Accepted if it Helped, Kudos are Appreciated too
Highlighted
SRX Services Gateway

Re: GRE tunnel implementation to Zscaler using multi tunnels and failover between primary and secondary gateway - (gre tunnel loadbalancing and failover)

[ Edited ]
‎06-05-2018 10:30 PM

 

IMHO it should/could work via routing, the only strange thing is that the primary route is typically defined with next-hop and not with qualified next-hop, For For a detailed analysis verification of my assumption please

Can you please do a "show route table  traffic-tunnel.inet.0" and show it to us

and also display teh forwarding table for that routing instance

 

regards

alexander

Highlighted
SRX Services Gateway

Re: GRE tunnel implementation to Zscaler using multi tunnels and failover between primary and secondary gateway - (gre tunnel loadbalancing and failover)

‎06-20-2018 02:02 AM

Hi Alexander

 

Thanks for your reply. Here the routing table on traffic_tunnel.inet.0.

 

user@junipersrx1500> show route table traffic_tunnel.inet.0

traffic_tunnel.inet.0: 23 destinations, 26 routes (23 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

144.155.100.16/28 *[Direct/0] 36w5d 22:26:40
> via reth0.2613
[Direct/0] 36w5d 22:26:40
> via reth0.2613
[Direct/0] 36w5d 22:26:40
> via reth0.2613
[Direct/0] 36w5d 22:26:40
> via reth0.2613
144.155.100.20/32 *[Local/0] 36w5d 22:26:40
Local via reth0.2613
144.155.100.21/32 *[Local/0] 36w5d 22:26:40
Local via reth0.2613
144.155.100.22/32 *[Local/0] 36w5d 22:26:40
Local via reth0.2613
144.155.100.23/32 *[Local/0] 36w5d 22:26:40
Local via reth0.2613
165.225.100.0/22 *[Static/200] 13w0d 17:31:37
via gr-0/0/0.2
via gr-0/0/0.4
via gr-0/0/0.6
> via gr-0/0/0.0
172.17.21.224/30 *[Direct/0] 36w5d 22:26:40
> via gr-0/0/0.3
172.17.21.225/32 *[Local/0] 36w5d 22:26:40
Local via gr-0/0/0.3
172.17.21.228/30 *[Direct/0] 36w5d 22:26:40
> via gr-0/0/0.2
172.17.21.229/32 *[Local/0] 36w5d 22:26:40
Local via gr-0/0/0.2
172.17.81.96/30 *[Direct/0] 36w5d 22:26:40
> via gr-0/0/0.1
172.17.81.97/32 *[Local/0] 36w5d 22:26:40
Local via gr-0/0/0.1
172.17.81.100/30 *[Direct/0] 36w5d 22:26:40
> via gr-0/0/0.0
172.17.81.101/32 *[Local/0] 36w5d 22:26:40
Local via gr-0/0/0.0
172.17.87.96/30 *[Direct/0] 36w5d 22:26:40
> via gr-0/0/0.5
172.17.87.97/32 *[Local/0] 36w5d 22:26:40
Local via gr-0/0/0.5
172.17.87.100/30 *[Direct/0] 36w5d 22:26:40
> via gr-0/0/0.4
172.17.87.101/32 *[Local/0] 36w5d 22:26:40
Local via gr-0/0/0.4
172.17.87.104/30 *[Direct/0] 36w5d 22:26:40
> via gr-0/0/0.7
172.17.87.105/32 *[Local/0] 36w5d 22:26:40
Local via gr-0/0/0.7
172.17.87.108/30 *[Direct/0] 36w5d 22:26:40
> via gr-0/0/0.6
172.17.87.109/32 *[Local/0] 36w5d 22:26:40
Local via gr-0/0/0.6
185.46.100.0/23 *[Static/5] 12w5d 16:31:20
> via gr-0/0/0.3
via gr-0/0/0.5
via gr-0/0/0.7
via gr-0/0/0.1

 

How did you mean that the primary route is typically defined with next-hop and not with qualified next-hop?
Sorry to ask you that, but we implemented the GRE as Zscaler told us to do. Strange is that I was more or less thinking the same like you.

 

BTW: we implemented in the LAB an test environment using 4 tunnel (two primary and two secondary) and using the last implementation setup they explain in https://help.zscaler.com/zia/gre-configuration-example-juniper-srx we also set a tunnel monitor. But the issue stays the same as before...

 

Thanks in advance 

BR Patrick

Feedback