SRX Services Gateway
SRX Services Gateway

SRX650 won't route packets and I have no idea why

‎02-02-2019 06:49 AM

I've recently spent way too much time trying to get an SRX650 to properly route packets between two networks without any success. Hopefully someone here can point me in the right direction.  

 

The basic setup is as follows: I have an SRX650 that's conneted via ethernet to an upstream router. Here, I'm using 198.51.100.0/24 instead of my real assigned network address. I've split off a /29 network (198.51.100.248/29) to be used for addressing between the SRX650 and the upstream routers. The uplink is connected to an SFP on ge-2/0/20 which is assigned 198.51.100.253/29. The default upstream router is as 198.51.100.254.

 

ge-2/0/0 is meant to be connected to clients on my network and is assigned 198.51.100.1/25. For testing purposes, I've also connected a device to ge-2/0/6 and assigned 10.0.0.1/16 to that port.

 

Packets are properly routed between clients on the 198.51.100.0/25 and 10.0.0.0/16 networks. However, nothing seems to work between 198.51.100.0/25 and 198.51.100.248/29. If I try to ping an external IP (e.g. 8.8.8.8) from the router CLI, everything works as expected and I can run

monitor traffic interface ge-2/0/20 no-resolve size 1500

to see the ICMP echo requests and replies leave and arrive on the interface. However, if I instead run 

ping 8.8.8.8 source 198.51.100.1

nothing works anymore. The same applies when I try to send traffic to and from clients on that network (e.g. 198.51.100.10). Running the same monitor traffic command as above now shows ICMP echo requests leaving the interface, but none coming back. Initially, I suspected that the upstream router was dropping incoming or outgoing packets, but I checked with the administrator of the upstream router and he confirmed (and showed me) that it's indeed configured to route the entire 198.51.100.0/24 network to 198.51.100.253.

 

Another curuous circumstance is that disabling ge-2/0/0 in the configuration or physically removing the network cable will cause the SRX to return "destination unreachable" for external ping requests to the 198.51.100.0/25. With the interface active, the same requests will just time out. My interpretation of this is that the upstream router is correctly routing packets to my SRX. None of those packets are displayed when using the monitor traffic command above, however.

 

I've also tried to set forwarding mode to packet based without any luck. One thing that does work is NAT-ing the 198.51.100.0/25 network, so that all traffic is address translated to 198.51.100.253, but that not really what I want.

 

I'm pasting the router configuration below. Thanks in advance for any and all help!

 

version 12.3X48-D75.4;
system {
    host-name r1;
    time-zone UTC;
    root-authentication {
        encrypted-password ## REDACTED
    }
    name-server {
        8.8.8.8;
        8.8.4.4;
        208.67.222.222;
        208.67.220.220;
    }
    name-resolution {
        no-resolve-on-input;
    }
    services {
        ssh;
        web-management {
            http {
                interface ge-2/0/0.0;
            }
            https {
                system-generated-certificate;
                interface ge-2/0/0.0;
            }
            session {
                idle-timeout 60;
            }
        }
        dhcp {
            pool 198.51.100.0/25 {
                address-range low 198.51.100.10 high 198.51.100.126;
                name-server {
                    8.8.8.8;
                    8.8.4.4;
                }
                router {
                    198.51.100.1;
                }
            }
        }
    }
    syslog {
        archive size 100k files 3;
        user * {
            any emergency;
        }
        file messages {
            any critical;
            authorization info;
        }
        file interactive-commands {
            interactive-commands error;
        }
    }
    max-configurations-on-flash 5;
    max-configuration-rollbacks 5;
    license {
        autoupdate {
            url https://ae1.juniper.net/junos/key_retrieval;
        }
    }
    ntp {
        server ntp.se;
    }
}
security {
    screen {
        ids-option untrust-screen {
            ip {
                source-route-option;
                tear-drop;
            }
            tcp {
                syn-flood {
                    alarm-threshold 1024;
                    attack-threshold 200;
                    source-threshold 1024;
                    destination-threshold 2048;
                    timeout 20;
                }
                land;
            }
        }
    }
    policies {
        from-zone clients to-zone Internet {
            policy All_clients_Internet {
                match {
                    source-address any;
                    destination-address any;
                    application any;
                }
                then {
                    permit;
                }
            }
        }
        from-zone clients to-zone clients {
            policy client_to_client {
                match {
                    source-address any;
                    destination-address any;
                    application any;
                }
                then {
                    permit;
                }
            }
        }
        from-zone Internet to-zone clients {
            policy internet_to_clients {
                match {
                    source-address any;
                    destination-address any;
                    application any;
                }
                then {
                    permit;
                }
            }
        }
    }
    zones {
        security-zone clients {
            host-inbound-traffic {
                system-services {
                    all;
                }
                protocols {
                    all;
                }
            }
            interfaces {
                ge-2/0/0.0 {
                    host-inbound-traffic {
                        system-services {
                            ping;
                            dhcp;
                            http;
                            https;
                            ssh;
                        }
                    }
                }
                ge-2/0/6.0 {
                    host-inbound-traffic {
                        system-services {
                            ping;
                            dhcp;
                            http;
                            https;
                            ssh;
                        }
                    }
                }
            }
        }
        security-zone Internet {
            interfaces {
                ge-2/0/20.0 {
                    host-inbound-traffic {
                        system-services {
                            ping;
                            ssh;
                        }
                    }
                }
            }
        }
    }
}
interfaces {
    ge-2/0/0 {
        unit 0 {
            family inet {
                address 198.51.100.1/25;
            }
        }
    }
    ge-2/0/6 {
        unit 0 {
            family inet {
                address 10.0.0.1/16;
            }
        }
    }
    ge-2/0/20 {
        unit 0 {
            family inet {
                address 198.51.100.253/29;
            }
        }
    }
}
routing-options {
    static {
        route 0.0.0.0/0 next-hop 198.51.100.254;
    }
}
7 REPLIES 7
SRX Services Gateway

Re: SRX650 won't route packets and I have no idea why

[ Edited ]
‎02-02-2019 07:52 AM

None of the devices in 198.51.100.248/29 (or beyond) have routes back to 198.51.100.0/25 -- this is why NATting to 198.51.100.253 works. You need to either announce 198.51.100.0/25 from the srx via some routing protocol or install a static route (and redistribute) on 198.51.100.254.

SRX Services Gateway

Re: SRX650 won't route packets and I have no idea why

‎02-02-2019 08:41 AM

Thanks for the reply!

The upstream router (198.51.100.254) has a static route for 198.51.100.0/24 to 198.51.100.253. Also, when I disable ge-2/0/0, I get "destination unreachable" replies from my SRX650 to pings for e.g. 198.51.100.1, so the packets seem to be routed correctly.

That said, this really does look like an routing issue external to the SRX650.

 

SRX Services Gateway

Re: SRX650 won't route packets and I have no idea why

‎02-02-2019 03:29 PM

Hi wittypotato,

 

Lets confirm if the packets are received/sent by the SRX when pinging from an external address to an internal address in subnet 198.51.100.0/25:

 

set firewall filter EXTERNAL-INTERFACE-IN term INCOMING-TRAFFIC from destination-prefix [IP-from-internal-host]
set firewall filter EXTERNAL-INTERFACE-IN term INCOMING-TRAFFIC from source-prefix [IP-from-external-host]
set firewall filter EXTERNAL-INTERFACE-IN term INCOMING-TRAFFIC from protocol icmp
set firewall filter EXTERNAL-INTERFACE-IN term INCOMING-TRAFFIC then count INCOMING-TRAFFIC
set firewall filter EXTERNAL-INTERFACE-IN term INCOMING-TRAFFIC then accept
set firewall filter EXTERNAL-INTERFACE-IN term ALLOW-ALL-ELSE then accept

set firewall filter EXTERNAL-INTERFACE-OUT term OUTGOING-TRAFFIC from source-prefix [IP-from-internal-host]
set firewall filter EXTERNAL-INTERFACE-OUT term OUTGOING-TRAFFIC from destination-prefix [IP-from-external-host]
set firewall filter EXTERNAL-INTERFACE-OUT term OUTGOING-TRAFFIC from protocol icmp
set firewall filter EXTERNAL-INTERFACE-OUT term OUTGOING-TRAFFIC then count OUTGOING-TRAFFIC
set firewall filter EXTERNAL-INTERFACE-OUT term OUTGOING-TRAFFIC then accept
set firewall filter EXTERNAL-INTERFACE-OUT term ALLOW-ALL-ELSE then accept

set interfaces ge-2/0/20.0 family inet filter input EXTERNAL-INTERFACE-IN
set interfaces ge-2/0/20.0 family inet filter output EXTERNAL-INTERFACE-OUT

set firewall filter INTERNAL-INTERFACE-IN term INCOMING-TRAFFIC from destination-prefix [IP-from-external-host]
set firewall filter INTERNAL-INTERFACE-IN term INCOMING-TRAFFIC from source-prefix [IP-from-internal-host]
set firewall filter INTERNAL-INTERFACE-IN term INCOMING-TRAFFIC from protocol icmp
set firewall filter INTERNAL-INTERFACE-IN term INCOMING-TRAFFIC then count INCOMING-TRAFFIC
set firewall filter INTERNAL-INTERFACE-IN term INCOMING-TRAFFIC then accept
set firewall filter INTERNAL-INTERFACE-IN term ALLOW-ALL-ELSE then accept

set firewall filter INTERNAL-INTERFACE-OUT term INCOMING-TRAFFIC from destination-prefix [IP-from-internal-host]
set firewall filter INTERNAL-INTERFACE-OUT term INCOMING-TRAFFIC from source-prefix [IP-from-external-host]
set firewall filter INTERNAL-INTERFACE-OUT term INCOMING-TRAFFIC from protocol icmp
set firewall filter INTERNAL-INTERFACE-OUT term INCOMING-TRAFFIC then count OUTGOING-TRAFFIC
set firewall filter INTERNAL-INTERFACE-OUT term INCOMING-TRAFFIC then accept
set firewall filter INTERNAL-INTERFACE-OUT term ALLOW-ALL-ELSE then accept

set interfaces ge-2/0/0.0 family inet filter input INTERNAL-INTERFACE-IN
set interfaces ge-2/0/0.0 family inet filter output INTERNAL-INTERFACE-OUT

After committing the above commands try an ping from an external host and verify the counters with "show firewall" command.

 

Ref: https://kb.juniper.net/InfoCenter/index?page=content&id=KB21872

 

Please also configure flow traceoptions so we can see whats the SRX doing with the packets received:

 

set security flow traceoptions file TEST
set security flow traceoptions flag basic-datapath
set security flow traceoptions packet-filter TEST source-address [IP-from-external-host]
set security flow traceoptions packet-filter TEST destination-address [IP-from-internal-host]
set security flow traceoptions packet-filter TEST protocol icmp

 

After commiting the above config and tried the ping test, check the result with the command "show log TEST"

 

Pura Vida from Costa Rica - Mark as Resolved if it applies.
Kudos are appreciated too!
SRX Services Gateway

Re: SRX650 won't route packets and I have no idea why

‎02-03-2019 03:32 AM

Thank you! I had to make a few modifications to the config to get it to load. It seems that the router is performing as it should and that a router upstream is dropping outgoing packets from 198.51.100.0/25.

 

From external computer:

--- 198.51.100.10 ping statistics ---
85 packets transmitted, 0 received, 100% packet loss, time 112ms

From router:

root@r1> show firewall    

Filter: __default_bpdu_filter__                                

Filter: EXTERNAL-INTERFACE-IN                                  
Counters:
Name                                                Bytes              Packets
INCOMING-TRAFFIC                                     7140                   85

Filter: EXTERNAL-INTERFACE-OUT                                 
Counters:
Name                                                Bytes              Packets
OUTGOING-TRAFFIC                                     7140                   85

Filter: INTERNAL-INTERFACE-IN                                  
Counters:
Name                                                Bytes              Packets
INCOMING-TRAFFIC                                     7140                   85

Filter: INTERNAL-INTERFACE-OUT                                 
Counters:
Name                                                Bytes              Packets
OUTGOING-TRAFFIC                                     7140                   85

Everything looks normal in the datapath logging as well. I also recorded a pcap file and inspected the packets. Nothing out of the ordinary there either.

 

I'll talk with the administrator of the upstream router on Monday.

SRX Services Gateway

Re: SRX650 won't route packets and I have no idea why

[ Edited ]
‎02-04-2019 03:14 AM

My bet would also be an upstream (upstream or internal to your ISP) that is not getting the /24.

I've seen it happen before, especially when blocks are reassigned.

 

You could verify by doing some traceroutes, from within and from the outside. I can't reach any of the IPs so they're probably redacted....

 

Try comparing a regular traceroute and one sourced from the assigned block. If you get beyond the first hop (which I bet you will) you can 'prove' it's an upstream problem, which could be caused by either bad route distribution on your ISP's CPE or something else higher up. Simply having the route on the first router is not enough; it should also be announced upstream by your ISP.

SRX Services Gateway

Re: SRX650 won't route packets and I have no idea why

‎02-04-2019 11:37 AM

Hi, wittypotato

 

The results talk by themselves, I am glad you have isolated where the issue is.

 

Let us know if you need further help or please mark the post as Resolved if it applies.

 

Pura Vida from Costa Rica - Mark as Resolved if it applies.
Kudos are appreciated too!
SRX Services Gateway

Re: SRX650 won't route packets and I have no idea why

‎02-11-2019 03:25 PM

Hi wittypotato,

 

I can see that the post is still not marked as resolved, were you able to dig further and confirm if the problem was outside the SRX? Let us know if you need further help.

 

Pura Vida from Costa Rica - Mark as Resolved if it applies.
Kudos are appreciated too!