SRX

I have implemented a dual-WAN with failover configuration for my SRX 300.

Each egress interface (Fiber and Cable) has a separate default route, so they are placed in distinct routing instances.

The default route associated with the primary egress interface is advertised to inet.0 via a RIB group.

Everything appears to be working well, but my destination NAT policies are broken.

Nothing gets through to my hosts behind the SRX.

Before the latest changes, I had a simple trust/untrust setup.

Now I have 3 zones: trust, untrust-Fiber and untrust-Cable.

Can anyone spot an obvious mistake?

Below is an excerpt of my config.

services {
rpm {
probe Probe-Fiber {
test test-GoogleDNS {
probe-type icmp-ping;
target address 8.8.8.8;
probe-count 5;
probe-interval 1;
test-interval 2;
thresholds {
successive-loss 2;
total-loss 4;
}
destination-interface reth0.0;
next-hop 1.2.3.9;
}
}
probe Probe-Cable {
test test-GoogleDNS {
probe-type icmp-ping;
target address 8.8.8.8;
probe-count 5;
probe-interval 1;
test-interval 2;
thresholds {
successive-loss 2;
total-loss 4;
}
destination-interface reth1.0;
next-hop 10.20.30.9;
}
}
}
ip-monitoring {
policy Comcast-Fiber-monitor {
match {
rpm-probe Probe-Fiber;
}
then {
preferred-route {
routing-instances Comcast-Fiber {
route 0.0.0.0/0 {
next-hop 1.2.3.9;
}
}
}
}
}
policy Comcast-Cable-monitor {
match {
rpm-probe Probe-Cable;
}
then {
preferred-route {
routing-instances Comcast-Cable {
route 0.0.0.0/0 {
next-hop 10.20.30.9;
}
}
}
}
}
}
}
security {
address-book {
global {
address Unifi-Controller 192.168.1.141/32;
}
}
nat {
source {
rule-set trust-to-untrust-Fiber {
from zone trust;
to zone untrust-Fiber;
rule source-nat-rule-Fiber {
match {
source-address 0.0.0.0/0;
destination-address 0.0.0.0/0;
}
then {
source-nat {
interface;
}
}
}
}
rule-set trust-to-untrust-Cable {
from zone trust;
to zone untrust-Cable;
rule source-nat-rule-Cable {
match {
source-address 0.0.0.0/0;
destination-address 0.0.0.0/0;
}
then {
source-nat {
interface;
}
}
}
}
}
destination {
pool dnat-Unifi-8080 {
address 192.168.1.141/32 port 8080;
}
rule-set dest-nat-Fiber {
from zone untrust-Fiber;
rule rule-Unifi-8080-Fiber {
match {
destination-address 1.2.3.10/32;
destination-port {
8080;
}
}
then {
destination-nat {
pool {
dnat-Unifi-8080;
}
}
}
}
}
rule-set dest-nat-Cable {
from zone untrust-Cable;
rule rule-Unifi-8080-Cable {
match {
destination-address 10.20.30.10/32;
destination-port {
8080;
}
}
then {
destination-nat {
pool {
dnat-Unifi-8080;
}
}
}
}
}
}
}
policies {
from-zone trust to-zone untrust-Fiber {
policy trust-to-untrust-Fiber {
match {
source-address any;
destination-address any;
application any;
}
then {
permit;
}
}
}
from-zone untrust-Fiber to-zone trust {
policy unt-to-tr-Unifi-8080 {
match {
source-address any;
destination-address Unifi-Controller;
application Unifi-HTTP;
}
then {
permit;
}
}
}
from-zone trust to-zone untrust-Cable {
policy trust-to-untrust-Cable {
match {
source-address any;
destination-address any;
application any;
}
then {
permit;
}
}
}
from-zone untrust-Cable to-zone trust {
policy unt-to-tr-Unifi-8080 {
match {
source-address any;
destination-address Unifi-Controller;
application Unifi-HTTP;
}
then {
permit;
}
}
}
}
zones {
security-zone trust {
host-inbound-traffic {
system-services {
all;
}
protocols {
all;
}
}
interfaces {
irb.0 {
host-inbound-traffic {
system-services {
ping;
http;
https;
ssh;
telnet;
dhcp;
}
}
}
}
}
security-zone untrust-Fiber {
interfaces {
reth0.0 {
host-inbound-traffic {
system-services {
ping;
ssh;
snmp;
ike;
}
}
}
}
}
security-zone untrust-Cable {
interfaces {
reth1.0 {
host-inbound-traffic {
system-services {
ping;
ssh;
snmp;
ike;
}
}
}
}
}
}
}
interfaces {
irb {
unit 0 {
family inet {
address 192.168.1.1/24;
}
}
}
reth0 {
redundant-ether-options {
redundancy-group 1;
}
unit 0 {
family inet {
address 1.2.3.10/30;
}
}
}
reth1 {
redundant-ether-options {
redundancy-group 1;
}
unit 0 {
family inet {
address 10.20.30.10/30;
}
}
}
}
routing-options {
static {
route 0.0.0.0/0 next-table Comcast-Fiber.inet.0;
}
rib-groups {
Fiber-to-Cable {
import-rib [ Comcast-Fiber.inet.0 Comcast-Cable.inet.0 ];
}
Cable-to-Fiber {
import-rib [ Comcast-Cable.inet.0 Comcast-Fiber.inet.0 ];
}
}
}
routing-instances {
Comcast-Cable {
instance-type virtual-router;
interface reth1.0;
routing-options {
interface-routes {
rib-group inet Fiber-to-Cable;
}
static {
route 0.0.0.0/0 next-hop 10.20.30.9;
}
}
}
Comcast-Fiber {
instance-type virtual-router;
interface reth0.0;
routing-options {
interface-routes {
rib-group inet Cable-to-Fiber;
}
static {
route 0.0.0.0/0 next-hop 1.2.3.9;
}
}
}
}
applications {
application Unifi-HTTP {
protocol tcp;
destination-port 8080;
}
}

Hello , 

DNAT configuration looks fine here . But the concern is that , to which interface does the traffic to Unifi-Controller comes in ? Untrust fiber or cable ? because if it comes through one and the route is pointing to another untrust zone , it will fail in re-route since you are sharing the routes between routing tables . So make sure if the traffic is coming VIA untrust fiber , the return traffic is via same zone . 

Try capturing the traffic drops using flow trace so that It will be clear where its dropping :  https://kb.juniper.net/InfoCenter/index?page=content&id=KB21757&actp=METADATA

Currently I'm testing DNAT through the Fiber-untrust interface (1.2.3.10).

But I do believe you are correct: the Comcast-Fiber routing instance is unaware of the LAN routes, and redirects the incoming traffic out to the Internet (see trace log at bottom).

I added the inet.0 routing tables to the import-rib statement, but that does not seem to have an effect according to the routes showing.

#show config routing-instance

Comcast-Cable {

    instance-type virtual-router;

    interface reth1.0;

    routing-options {

        interface-routes {

            rib-group inet Fiber-to-Cable;

        }

        static {

            route 0.0.0.0/0 next-hop 10.20.30.9;

        }

    }

}

Comcast-Fiber {

    instance-type virtual-router;

    interface reth0.0;

    routing-options {

        interface-routes {

            rib-group inet Cable-to-Fiber;

        }

        static {

            route 0.0.0.0/0 next-hop 1.2.3.9;

        }

    }

}

#show routing-options

rib-groups {

    Fiber-to-Cable {

        import-rib [ Comcast-Fiber.inet.0 Comcast-Cable.inet.0 inet.0 ];

    }

    Cable-to-Fiber {

        import-rib [ Comcast-Cable.inet.0 Comcast-Fiber.inet.0 inet.0 ];

    }

}

>show route table Comcast-Fiber.inet.0

Comcast-Fiber.inet.0: 5 destinations, 5 routes (5 active, 0 holddown, 0 hidden)

+ = Active Route, - = Last Active, * = Both

0.0.0.0/0          *[Static/5] 16:01:30

                    > to 1.2.3.9 via reth0.0

10.20.30.9/30     *[Direct/0] 16:01:30

                    > via reth1.0

10.20.30.10/32     *[Local/0] 16:01:30

                      Local via reth1.0

1.2.3.8/30    *[Direct/0] 16:01:30

                    > via reth0.0

1.2.3.10/32   *[Local/0] 16:01:30

                      Local via reth0.0

>show log debug.log

Jul 15 10:47:27 10:47:27.300439:CID-1:RT:  reth0.0:96.66.219.90/25332->1.2.3.10/8080, tcp, flag 2 syn

Jul 15 10:47:27 10:47:27.300439:CID-1:RT: find flow: table 0x536c2b8, hash 18510(0xffff), sa 96.66.219.90, da 1.2.3.10, sp 25332, dp 8080, proto 6, tok 32779, conn-tag 0x00000000

Jul 15 10:47:27 10:47:27.300439:CID-1:RT:  no session found, start first path. in_tunnel - 0x0, from_cp_flag - 0

Jul 15 10:47:27 10:47:27.300439:CID-1:RT:search gate for untrust-Fiber:96.66.219.90/25332->1.2.3.10/8080,6

Jul 15 10:47:27 10:47:27.300439:CID-1:RT:gate_search_specific_bucket: no gate found

Jul 15 10:47:27 10:47:27.300439:CID-1:RT:search gate for untrust-Fiber:96.66.219.90/25332->1.2.3.10/8080,6

Jul 15 10:47:27 10:47:27.300439:CID-1:RT:gate_search_specific_bucket: no gate found

Jul 15 10:47:27 10:47:27.300439:CID-1:RT:search widecast gate for untrust-Fiber:96.66.219.90/25332->1.2.3.10/8080,6

Jul 15 10:47:27 10:47:27.300439:CID-1:RT:gate_search_widecast_bucket: no gate found

Jul 15 10:47:27 10:47:27.300439:CID-1:RT:check self-traffic on reth0.0, in_tunnel 0x0

Jul 15 10:47:27 10:47:27.300439:CID-1:RT:retcode: 0x1

Jul 15 10:47:27 10:47:27.300439:CID-1:RT:pak_for_self : proto 6, dst port 8080, action 0x0

Jul 15 10:47:27 10:47:27.300439:CID-1:RT:flow_first_create_session

Jul 15 10:47:27 10:47:27.300439:CID-1:RT:Save init hash spu id 0 to nsp and nsp2!

Jul 15 10:47:27 10:47:27.300439:CID-1:RT:First path alloc and instl pending session, natp=0x8b87270, id=36780

Jul 15 10:47:27 10:47:27.300439:CID-1:RT:  flow_first_in_dst_nat: in <reth0.0>, out <N/A> dst_adr 1.2.310, sp 25332, dp 8080

Jul 15 10:47:27 10:47:27.300439:CID-1:RT:  chose interface reth0.0 as incoming nat if.

Jul 15 10:47:27 10:47:27.300439:CID-1:RT:flow_first_rule_dst_xlate: DST xlate: 1.2.3.10(8080) to 192.168.1.141(8080), rule/pool id 21/7.

Jul 15 10:47:27 10:47:27.300439:CID-1:RT:[JSF] Do ingress interest check. regd ingress plugins(1)

Jul 15 10:47:27 10:47:27.300439:CID-1:RT:[JSF][0]plugins(0x0) enabled for session = 12884938668  implicit mask(0x0), service request(0x0)

Jul 15 10:47:27 10:47:27.300439:CID-1:RT:-jsf : no plugin ingress interested for session 12884938668

Jul 15 10:47:27 10:47:27.300439:CID-1:RT:flow_first_routing: vr_id 8, call flow_route_lookup(): src_ip 96.66.219.90, x_dst_ip 192.168.1.141, in ifp reth0.0, out ifp N/A sp 25332, dp 8080, ip_proto 6, tos 50

Jul 15 10:47:27 10:47:27.300439:CID-1:RT:Doing DESTINATION addr route-lookup

Jul 15 10:47:27 10:47:27.300439:CID-1:RT:flow_ipv4_rt_lkup success 192.168.1.141, iifl 0x6b, oifl 0x6

Jul 15 10:47:27 10:47:27.300439:CID-1:RT:  routed (x_dst_ip 192.168.1.141) from untrust-Fiber (reth0.0 in 1) to reth0.0, Next-hop: 1.2.3.9

Jul 15 10:47:27 10:47:27.300439:CID-1:RT:flow_first_policy_search: policy search from zone untrust-Fiber-> zone untrust-Fiber (0x110,0x62f41f90,0x1f90)

Jul 15 10:47:27 10:47:27.300439:CID-1:RT:Policy lkup: vsys 0 zone(11:untrust-Fiber) -> zone(11:untrust-Fiber) scope:0

Jul 15 10:47:27 10:47:27.300439:CID-1:RT:             96.66.219.90/25332 -> 192.168.1.141/8080 proto 6

Jul 15 10:47:27 10:47:27.300439:CID-1:RT:Policy lkup: vsys 0 zone(5:global) -> zone(5:global) scope:0

Jul 15 10:47:27 10:47:27.300439:CID-1:RT:             96.66.219.90/25332 -> 192.168.1.141/8080 proto 6

Jul 15 10:47:27 10:4727.300439:CID-1:RT:flow_first_policy_search: dynapp_none_policy: 1? is_final: 0x0, is_explicit: 0x0, policy_meta_data: 0x0

Jul 15 10:47:27 10:47:27.300439:CID-1:RT:  app 0, timeout 1800s, curr ageout 20s

Jul 15 10:47:27 10:47:27.300439:CID-1:RT:  packet dropped, denied by policy

Jul 15 10:47:27 10:47:27.300439:CID-1:RT:  denied by policy default-policy-logical-system-00(2), dropping pkt

Jul 15 10:47:27 10:47:27.300439:CID-1:RT:  packet dropped,  policy deny.

Jul 15 10:47:27 10:47:27.300439:CID-1:RT:flow_initiate_first_path: first pak no session

Jul 15 10:47:27 10:47:27.300439:CID-1:RT:  flow find session returns error.

Jul 15 10:47:27 10:47:27.300439:CID-1:RT:flow_proc_rc: -1.

Jul 15 10:47:27 10:47:27.300439:CID-1:RT: ----- flow_process_pkt rc 0x7 (fp rc -1)

Hi Arenault,

I checked the traces and I believe the reason for the drop is there is no route for the translated IP address - 192.168.1.141.

As per the SRX flow, when the packet comes in, Destination NAT translates the Interface IP address into the translated IP address(1.2.3.10 -> 192.168.1.1); post that route lookup and policy lookup happens for the translated IP address.

So, during the route lookup, the SRX was unable to find the direct route or static route hence, it takes the default route 0.0.0.0/0 which is exiting via reth0.0 under the table Comcast-Fiber.inet.0

Also looking at the table Comcast-Fiber.inet.0, there is no route configured for the IP address 192.168.1.141 which is suppose to be behind the SRX.

>show route table Comcast-Fiber.inet.0

Comcast-Fiber.inet.0: 5 destinations, 5 routes (5 active, 0 holddown, 0 hidden)

+ = Active Route, - = Last Active, * = Both

0.0.0.0/0          *[Static/5] 16:01:30

                    > to 1.2.3.9 via reth0.0

10.20.30.9/30     *[Direct/0] 16:01:30

                    > via reth1.0

10.20.30.10/32     *[Local/0] 16:01:30

                      Local via reth1.0

1.2.3.8/30    *[Direct/0] 16:01:30

                    > via reth0.0

1.2.3.10/32   *[Local/0] 16:01:30

                      Local via reth0.0

So, give a static route under this table for the destination 192.168.1.141/32 and I think it should work. BTW, behind which interface this IP address - 192.168.1.141 is connected?

Hi NoobMaster,

Yes, I agree with your idea, but I'm not sure how to implement a solution.

I tried to add a static route in the routing instance as follows, but that didn't help.

Comcast-Fiber {

    instance-type virtual-router;

    interface reth0.0;

    routing-options {

        interface-routes {

            rib-group inet Cable-to-Fiber;

        }

        static {

            route 0.0.0.0/0 next-hop 1.2.3.9;

            route 192.168.1.0/24 next-hop 192.168.1.1;

        }

    }

}

Hi Arenault,

Provide me the output of "show route table Comcast-Fiber.inet.0" and also, the output of flow traces.

Let's see where it is blocked this time.

Same thing. The static route doesn't seem to make it to the routing instance table.

#show routing-instances

Comcast-Fiber {

    instance-type virtual-router;

    interface reth0.0;

    routing-options {

        interface-routes {

            rib-group inet Cable-to-Fiber;

        }

        static {

            route 0.0.0.0/0 next-hop 50.228.248.9;

            route 192.168.1.0/24 next-hop 192.168.1.1;

        }

    }

}

>show route table Comcast-Fiber.inet.0   

Comcast-Fiber.inet.0: 5 destinations, 5 routes (5 active, 0 holddown, 0 hidden)

+ = Active Route, - = Last Active, * = Both

0.0.0.0/0          *[Static/5] 1d 14:39:24

                    > to 50.228.248.9 via reth0.0

50.78.16.16/29     *[Direct/0] 1d 14:39:24

                    > via reth1.0

50.78.16.21/32     *[Local/0] 1d 14:39:24

                      Local via reth1.0

50.228.248.8/30    *[Direct/0] 1d 14:39:24

                    > via reth0.0

50.228.248.10/32   *[Local/0] 1d 14:39:24

                      Local via reth0.0

>show log debug.log

Jul 16 09:38:20 09:38:20.752613:CID-1:RT:[JSF][0]plugins(0x0) enabled for session = 68719528944  implicit mask(0x0), service request(0x0)

Jul 16 09:38:20 09:38:20.752613:CID-1:RT:-jsf : no plugin ingress interested for session 68719528944

Jul 16 09:38:20 09:38:20.752613:CID-1:RT:flow_first_routing: vr_id 8, call flow_route_lookup(): src_ip 96.66.219.90, x_dst_ip 192.168.1.141, in ifp reth0.0, out ifp N/A sp 31760, dp 8080, ip_proto 6, tos 50

Jul 16 09:38:20 09:38:20.752613:CID-1:RT:Doing DESTINATION addr route-lookup                                 

Jul 16 09:38:20 09:38:20.752613:CID-1:RT:flow_ipv4_rt_lkup success 192.168.1.141, iifl 0x6b, oifl 0x6b

Jul 16 09:38:20 09:38:20.752613:CID-1:RT:  routed (x_dst_ip 192.168.1.141) from untrust-Fiber (reth0.0 in 1) to reth0.0, Next-hop: 1.2.3.9

Hi Arenault,

Can you tell me behind which interface the destination(192.168.1.141) resides?

If 192.168.1.1 is your next-hop for the destination 192.168.1.141, can you see it in the output of "show arp no-resolve"? If you can't see it then 192.168.1.1 isn't your next-hop. I guess once you answer my first question I can give me some insight.

So I fixed my destination NAT issue with exporting the interface routes to my new routing instances in routing-options (see below).

Adding a static route to the routing instance as previously mentioned was ineffective.

Now I have one final issue:

I used to poll the SRX via offsite SNMP on port 1610 translated to 161.

This DNAT rule instead of passing traffic to a host on LAN behind SRX, is routed to the loopback address.

Traces show the traffic properly translated and routed to loopback.

Yet no SNMP data makes it back to our offsite server.

Any suggestion?

#show routing-options

interface-routes {

    rib-group inet inet.0-to-RI;

}

rib-groups {

    inet.0-to-RI {

        import-rib [ inet.0 Comcast-Fiber.inet.0 Comcast-Cable.inet.0 ];

    }

    Fiber-to-Cable {

        import-rib [ Comcast-Fiber.inet.0 Comcast-Cable.inet.0 ];

    }

    Cable-to-Fiber {

        import-rib [ Comcast-Cable.inet.0 Comcast-Fiber.inet.0 ];

    }

}

Hey Arenault,

I was under the impression that you have already imported the interface routes in your instance; that's the reason I asked behind which interface the destination resides because it clearly isn't in one of the interfaces which were there earlier. I'm glad the issue  is sorted :smileyhappy:

Now, regarding your SNMP issue can we continue it in a new thread? Because this will be useful for the community members to track the answers in an effective way and besides, I think SNMP traffic is processed properly but it can be an issue with the SNMP process itself.