DNS and NTP traffic to the Routing Engine is being blocked [ Edited ]

[mawr]

Hello everyone,

While implementing a new firewall filter for the routing engine (routing-engine-incoming-policy) I noticed that incoming DNS and NTP traffic was being dropped from hosts the system is configured to use.  I tried adjusting the appropriate terms in the filter to allow all traffic, adding ntp to the host-inbound-traffic system-services for the interface, and disabling the dns ALG but it still isn't working.  I've been testing it by pinging a DNS name that works when pinged via its IP.  Any ideas?

Firewall configuration:

## Last changed: 2010-06-12 23:57:54 CDT
version 10.1R2.8;
system {
    host-name SRX100;
    time-zone America/xxxxxxxx;
    root-authentication {
        encrypted-password "xxxxxxxx."; ## SECRET-DATA
    }
    name-server {
        208.67.222.222;
        208.67.220.220;
    }
    login {
        user xxxxxxxx {
            full-name "xxxxxxxx xxxxxxxx";
            uid 2000;
            class super-user;
            authentication {
                encrypted-password "xxxxxxxx."; ## SECRET-DATA
            }
        }
    }
    services {
        ssh {
            root-login deny;
            protocol-version v2;
            connection-limit 3;
        }
        web-management {
            https {
                system-generated-certificate;
                interface [ vlan.0 fe-0/0/0.0 ];
            }
        }
        dhcp {
            name-server {
                208.67.222.222;
                208.67.220.220;
            }
            wins-server {
                192.168.168.165;
            }
            router {
                192.168.168.168;
            }
            pool 192.168.168.0/24 {
                address-range low 192.168.168.50 high 192.168.168.100;
            }
        }
    }
    syslog {
        archive size 100k files 3;
        user * {
            any emergency;
        }
        file messages {
            any critical;
            authorization info;
        }
        file interactive-commands {
            interactive-commands error;
        }
        file accepted-traffic {
            any any;
            match RT_FLOW_SESSION_CLOSE;
        }
        file blocked-traffic {
            any any;
            match RT_FLOW_SESSION_DENY;
        }
    }
    max-configurations-on-flash 5;
    max-configuration-rollbacks 10;
    license {
        autoupdate {
            url https://ae1.juniper.net/junos/key_retrieval;
        }
    }
    ntp {
        server 129.6.15.28 version 4 prefer;
    }
}
interfaces {
    interface-range interfaces-trust {
        member fe-0/0/1;
        member fe-0/0/2;
        member fe-0/0/3;
        member fe-0/0/4;
        member fe-0/0/5;
        member fe-0/0/6;
        member fe-0/0/7;
        unit 0 {
            family ethernet-switching {
                vlan {
                    members vlan-trust;
                }
            }
        }
    }
    fe-0/0/0 {
        unit 0 {
            family inet {
                filter {
                    input internet-incoming-policy;
                    output internet-outgoing-policy;
                }
                sampling {
                    output;
                }
                address xxx.xxx.xxx.xxx/29;
            }
        }
    }
    lo0 {
        unit 0 {
            family inet {
                filter {
                    input routing-engine-incoming-policy;
                }
                address 127.0.0.1/32;
            }
        }
    }
    vlan {
        unit 0 {
            family inet {
                address 192.168.168.168/24;
            }
        }
    }
}
forwarding-options {
    sampling {
        input {
            family inet {
                rate 1;
                run-length 0;
                max-packets-per-second 1000;
            }
        }
        output {
            flow-server 192.168.168.166 {
                port 9996;
                version 5;
            }
        }
    }
}
snmp {
    community xxxxxxxx {
        authorization read-only;
        clients {
            192.168.168.166/32;
        }
    }
}
routing-options {
    static {
        route 0.0.0.0/0 next-hop xxx.xxx.xxx.xxx;
    }
}
security {
    nat {
        source {
            rule-set trust-to-untrust {
                from zone trust;
                to zone untrust;
                rule source-nat-rule {
                    match {
                        source-address 192.168.168.0/24;
                    }
                    then {
                        source-nat {
                            interface;
                        }
                    }
                }
            }
        }
    }
    screen {
        ids-option untrust-screen {
            icmp {
                ping-death;
            }
            ip {
                source-route-option;
                tear-drop;
            }
            tcp {
                syn-flood {
                    alarm-threshold 1024;
                    attack-threshold 200;
                    source-threshold 1024;
                    destination-threshold 2048;
                    timeout 20;
                }
                land;
            }
        }
    }
    zones {
        security-zone trust {
            address-book {
                address internal-network 192.168.168.0/24;
                address SRX100 192.168.168.168/32;
                address pos-till-1 192.168.168.201/32;
                address pos-till-2 192.168.168.202/32;
                address pos-till-3 192.168.168.203/32;
                address pos-term-1 192.168.168.204/32;
                address pos-term-2 192.168.168.205/32;
                address pos-term-3 192.168.168.206/32;
            }
            host-inbound-traffic {
                system-services {
                    all;
                }
                protocols {
                    all;
                }
            }
            interfaces {
                vlan.0;
            }
        }
        security-zone untrust {
            address-book {
                address resolver1.opendns.com 208.67.222.222/32;
                address resolver2.opendns.com 208.67.220.220/32;
                address time-a.nist.gov 129.6.15.28/32;
            }
            screen untrust-screen;
            interfaces {
                fe-0/0/0.0 {
                    host-inbound-traffic {
                        system-services {
                            ssh;
                            https;
                        }
                    }
                }
            }
        }
    }
    policies {
        from-zone trust to-zone untrust {
            policy dns {
                match {
                    source-address internal-network;
                    destination-address [ resolver1.opendns.com resolver2.opendns.com ];
                    application [ junos-dns-tcp junos-dns-udp ];
                }
                then {
                    permit;
                    log {
                        session-close;
                    }
                }
            }
            policy heartland-credit-processing {
                match {
                    source-address [ pos-till-1 pos-till-2 pos-till-3 ];
                    destination-address any;
                    application heartland-credit-processing;
                }
                then {
                    permit;
                    log {
                        session-close;
                    }
                }
            }
            policy heartland-gift-processing {
                match {
                    source-address [ pos-term-1 pos-term-2 pos-term-3 ];
                    destination-address any;
                    application heartland-gift-processing;
                }
                then {
                    permit;
                    log {
                        session-close;
                    }
                }
            }
            policy http {
                match {
                    source-address internal-network;
                    destination-address any;
                    application [ junos-http junos-http-ext ];
                }
                then {
                    permit;
                    log {
                        session-close;
                    }
                }
            }
            policy https {
                match {
                    source-address internal-network;
                    destination-address any;
                    application junos-https;
                }
                then {
                    permit;
                    log {
                        session-close;
                    }
                }
            }
            policy email {
                match {
                    source-address internal-network;
                    destination-address any;
                    application gmail;
                }
                then {
                    permit;
                    log {
                        session-close;
                    }
                }
            }
            policy ftp {
                match {
                    source-address internal-network;
                    destination-address any;
                    application junos-ftp;
                }
                then {
                    permit;
                    log {
                        session-close;
                    }
                }
            }
            policy ntp {
                match {
                    source-address internal-network;
                    destination-address time-a.nist.gov;
                    application junos-ntp;
                }
                then {
                    permit;
                    log {
                        session-close;
                    }
                }
            }
            policy ping {
                match {
                    source-address internal-network;
                    destination-address any;
                    application junos-ping;
                }
                then {
                    permit;
                    log {
                        session-close;
                    }
                }
            }
            inactive: policy emergency {
                match {
                    source-address internal-network;
                    destination-address any;
                    application any;
                }
                then {
                    permit;
                    log {
                        session-close;
                    }
                }
            }
            policy deny {
                match {
                    source-address any;
                    destination-address any;
                    application any;
                }
                then {
                    deny;
                    log {
                        session-init;
                    }
                }
            }
        }
        from-zone untrust to-zone trust {
            policy deny {
                match {
                    source-address any;
                    destination-address any;
                    application any;
                }
                then {
                    deny;
                    log {
                        session-init;
                    }
                }
            }
        }
    }
    flow {
        aging {
            low-watermark 50;
            high-watermark 80;
        }
    }
}
firewall {
    family inet {
        filter internet-outgoing-policy {
            term heartland-processing-prioritization {
                from {
                    source-address {
                        192.168.168.201/32;
                        192.168.168.202/32;
                        192.168.168.203/32;
                    }
                    protocol tcp;
                    source-port 1024-65535;
                    destination-port 9000;
                }
                then {
                    loss-priority low;
                    forwarding-class assured-forwarding;
                    accept;
                }
            }
            term forward-to-internet {
                then accept;
            }
        }
        filter internet-incoming-policy {
            term filter-management-traffic {
                from {
                    source-address {
                        0.0.0.0/0;
                        xxx.xxx.xxx.xxx/32 except;
                        xxx.xxx.xxx.xxx/32 except;
                    }
                    protocol tcp;
                    source-port 1024-65535;
                    destination-port [ ssh https ];
                }
                then {
                    discard;
                }
            }
            term forward-to-flow-processing {
                then accept;
            }
        }
        filter routing-engine-incoming-policy {
            term tcp-fragment-protection {
                from {
                    fragment-offset 1-5;
                }
                then {
                    syslog;
                    discard;
                }
            }
            term tcp-denial-of-service-protection {
                from {
                    protocol tcp;
                    tcp-flags "(syn & !ack) | fin | rst";
                }
                then {
                    policer routing-engine-tcp-policer;
                    accept;
                }
            }
            term icmp-denial-of-service-protection {
                from {
                    protocol icmp;
                    icmp-type [ echo-request echo-reply unreachable time-exceeded ];
                }
                then {
                    policer routing-engine-icmp-policer;
                    count icmp-counter;
                    accept;
                }
            }
            term allow-dns-traffic {
                from {
                    source-address {
                        208.67.222.222/32;
                        208.67.220.220/32;
                    }
                    protocol [ tcp udp ];
                    source-port 1024-65535;
                    destination-port domain;
                }
                then accept;
            }
            term allow-ntp-traffic {
                from {
                    source-address {
                        129.6.15.28/32;
                    }
                    protocol udp;
                    source-port 1024-65535;
                    destination-port ntp;
                }
                then accept;
            }
            term allow-management-traffic {
                from {
                    source-address {
                        192.168.168.0/24;
                        xxx.xxx.xxx.xxx/32;
                        xxx.xxx.xxx.xxx/32;
                    }
                    protocol tcp;
                    source-port 1024-65535;
                    destination-port [ ssh https ];
                }
                then accept;
            }
            term discard-remaining-traffic {
                then {
                    log;
                    syslog;
                    discard;
                }
            }
        }
    }
    policer routing-engine-tcp-policer {
        filter-specific;
        if-exceeding {
            bandwidth-limit 500k;
            burst-size-limit 15k;
        }
        then discard;
    }
    policer routing-engine-icmp-policer {
        filter-specific;
        if-exceeding {
            bandwidth-limit 1m;
            burst-size-limit 15k;
        }
        then discard;
    }
}
applications {
    application gmail {
        term imap-ssl protocol tcp source-port 1024-65535 destination-port 993;
        term pop3-ssl protocol tcp source-port 1024-65535 destination-port 995;
        term smtp-ssl protocol tcp source-port 1024-65535 destination-port 465;
        term smtp-tls protocol tcp source-port 1024-65535 destination-port 587;
    }
    application heartland-credit-processing {
        term credit protocol tcp source-port 1024-65535 destination-port 9000;
    }
    application heartland-gift-processing {
        term gift-1 protocol tcp source-port 1024-65535 destination-port 9402-9404;
        term gift-2 protocol tcp source-port 1024-65535 destination-port 8230;
    }
}
vlans {
    vlan-trust {
        vlan-id 3;
        l3-interface vlan.0;
    }
}

Thanks in advance,

mawr

[mawr]

Firewall log:

Log :
Time      Filter    Action Interface     Protocol        Src Addr                         Dest Addr
00:16:04  pfe       D      fe-0/0/0.0    UDP             208.67.222.222                   xxx.xxx.xxx.xxx
00:13:38  pfe       D      fe-0/0/0.0    UDP             208.67.222.222                   xxx.xxx.xxx.xxx
00:13:15  pfe       D      fe-0/0/0.0    UDP             208.67.222.222                   xxx.xxx.xxx.xxx
00:13:10  pfe       D      fe-0/0/0.0    UDP             208.67.220.220                   xxx.xxx.xxx.xxx
00:13:05  pfe       D      fe-0/0/0.0    UDP             208.67.222.222                   xxx.xxx.xxx.xxx
00:13:00  pfe       D      fe-0/0/0.0    UDP             208.67.220.220                   xxx.xxx.xxx.xxx
00:12:55  pfe       D      fe-0/0/0.0    UDP             208.67.222.222                   xxx.xxx.xxx.xxx
00:12:50  pfe       D      fe-0/0/0.0    UDP             208.67.220.220                   xxx.xxx.xxx.xxx
00:12:45  pfe       D      fe-0/0/0.0    UDP             208.67.222.222                   xxx.xxx.xxx.xxx
00:12:37  pfe       D      fe-0/0/0.0    UDP             208.67.222.222                   xxx.xxx.xxx.xxx
00:12:17  pfe       D      fe-0/0/0.0    UDP             129.6.15.28                      xxx.xxx.xxx.xxx
00:08:18  pfe       D      fe-0/0/0.0    UDP             208.67.220.220                   xxx.xxx.xxx.xxx
00:08:13  pfe       D      fe-0/0/0.0    UDP             208.67.222.222                   xxx.xxx.xxx.xxx
00:01:50  pfe       D      fe-0/0/0.0    UDP             208.67.222.222                   xxx.xxx.xxx.xxx
23:58:15  pfe       D      fe-0/0/0.0    UDP             208.67.220.220                   xxx.xxx.xxx.xxx
23:58:10  pfe       D      fe-0/0/0.0    UDP             208.67.222.222                   xxx.xxx.xxx.xxx
23:55:38  pfe       D      fe-0/0/0.0    UDP             208.67.220.220                   xxx.xxx.xxx.xxx
23:55:33  pfe       D      fe-0/0/0.0    UDP             208.67.222.222                   xxx.xxx.xxx.xxx
23:55:28  pfe       D      fe-0/0/0.0    UDP             208.67.220.220                   xxx.xxx.xxx.xxx
23:55:23  pfe       D      fe-0/0/0.0    UDP             208.67.222.222                   xxx.xxx.xxx.xxx
23:55:12  pfe       D      fe-0/0/0.0    UDP             129.6.15.28                      xxx.xxx.xxx.xxx
23:38:06  pfe       D      fe-0/0/0.0    UDP             129.6.15.28                      xxx.xxx.xxx.xxx
23:21:21  pfe       D      fe-0/0/0.0    UDP             208.67.220.220                   xxx.xxx.xxx.xxx
23:21:16  pfe       D      fe-0/0/0.0    UDP             208.67.222.222                   xxx.xxx.xxx.xxx
23:21:10  pfe       D      fe-0/0/0.0    UDP             208.67.220.220                   xxx.xxx.xxx.xxx
23:21:05  pfe       D      fe-0/0/0.0    UDP             208.67.222.222                   xxx.xxx.xxx.xxx
23:21:02  pfe       D      fe-0/0/0.0    UDP             129.6.15.28                      xxx.xxx.xxx.xxx
23:03:58  pfe       D      fe-0/0/0.0    UDP             129.6.15.28                      xxx.xxx.xxx.xxx
22:46:52  pfe       D      fe-0/0/0.0    UDP             129.6.15.28                      xxx.xxx.xxx.xxx
22:29:47  pfe       D      fe-0/0/0.0    UDP             129.6.15.28                      xxx.xxx.xxx.xxx

[aarseniev · Accepted solution]

Hello,

You have this terms in lo0.0 filter:

          term allow-dns-traffic {
                from {
                    source-address {
                        208.67.222.222/32;
                        208.67.220.220/32;
                    }
                    protocol [ tcp udp ];
                    source-port 1024-65535;
                    destination-port domain;
                }
                then accept;
            }
            term allow-ntp-traffic {
                from {
                    source-address {
                        129.6.15.28/32;
                    }
                    protocol udp;
                    source-port 1024-65535;
                    destination-port ntp;
                }
                then accept;
            }

FW filter on lo0.0 processes traffic _inbound_ to Routing Engine.

DNS query replies from server have source port 53 and destination port of either 53 or 1024-65535.

NTP replies from NTP server have source port 123 and destination port 123 or 1024-65535 after NAT.

I think You should reverse source and destination port definitions above.

HTH

Regards

Alex

[mawr]

That did it Alex, thanks!  Do you know if there is a way to have the firewall log record the source and destination ports as well?  Having that would have helped considerably when troubleshooting this problem.

mawr

[aarseniev]

Hello there

show firewall log detail

-- will display the port and other pkt info as well

http://www.juniper.net/techpubs/software/junos/junos73/swcmdref73-protocols/html/firewall-monitor4.h...

HTH

Regards

Alex

[mawr]

Thanks again!  One more question, although somewhat unrelated, while reading the documentation I noticed a statement saying that a lot of firewall filters slows down the Routing Engine.  Do you think that ten or so terms for a filter is going to slow it down?

[aarseniev]

Hello,

Depends on the filter and also on the ingress interface.

If you are syslogging denied traffic to file on RE HDD/flash and if You have a (D)DoS attack then RE has to work hard to write all denied pkt info on disk/flash. 

Lo0.0 FW filters are implemented in

1/ hardware on hi-end SRX - if locally-terminated traffic arrives on PFE interfaces, there is no slowdown, and

2/ RE kernel - if locally-terminated traffic arrives on fxp0, obviously the RE kernel has to work/filter it. Make sure Your fxp0 is really inside trusted network to avoid (D)DoS attacks on fxp0.

In short - YMMV

HTH

Regards

Alex