ScreenOS Firewalls (NOT SRX)
Showing results for 
Search instead for 
Do you mean 
Reply
Contributor
Posts: 14
Registered: ‎01-13-2011
0 Kudos

ScreenOS 6.3 - Connection refused by the DNS server.

I recently upgraded a several SSG5's from ScreenOS 5.4 to 6.3.0r11.0.  We're using NSM 2010.3p58 to configure/deploy policies too. The 5.4 and 6.3 devices share the same policy.

 

Only on the 6.3 device, it's been failing to resolve FQDN address objects. The system event log is filled with Connection refused by the DNS server.  The 5.4 device, which uses the exact same DNS servers does not exhibit this problem. I can see in the 6.3 device, the Dynamic cache some times contains a few entries, but the bulk of the FQDN addresses remain in the unresolved cache.

 

I've already read  KB5522 - Connection refused by Domain Name Server (DNS) server. Yes, the SSG5 is able to ping the DNS servers.  PC workstations all around campus use these DNS servers too and they are not exhibiting any problems. I checked with the DNS admin and the servers are healthy.  Thus, I'm certain the DNS servers are functioning properly and the issue resides with the SSG5 and ScreenOS 6.3.

 

What's next to troubleshoot why the SSG5 is having DNS issues?

Trusted Expert
Posts: 378
Registered: ‎05-12-2012
0 Kudos

Re: ScreenOS 6.3 - Connection refused by the DNS server.

Really strange issue , upgrade shouldn't cause any such issue.

To troubleshoot this further, try running snoop detail with filters for DNS

 

cl db

snoop filter ip x.x.x.x   ( DNS server IP)

snoop detail len 1514

snoop detail

snoop

 

And simultaneously run sniffer on the server.

 

Try resolving some FQDN on firewall.

 

Turn off snoop by hitting 'Esc' Key and collect the output ( get db st) in a text file  and open it in wireshark.

Check if  the DNS queries sent by firewall were received by firewall and did DNS server respond to them. If not then check DNS servers logs for rejection reason.

Contributor
Posts: 14
Registered: ‎01-13-2011
0 Kudos

Re: ScreenOS 6.3 - Connection refused by the DNS server.

Awesome, never knew the snoop output could be fed back into Wireshark. 

 

However, it seems my capture fails to be properly parsed by wireshark 1.6.5.  It parses the first entry and then displays "the capture file appears to be damanged or corrupt (netscreen: can't parse packet-header).  The netscreen sample in http://wiki.wireshark.org/NetScreen still parses correctly though... so not sure if it's due to the suble changes between SCreenOS versions or my capture method (logging all Putty session output to file and then clean out the screenos commands present).  The sample netscreen.txt looks basically like my captured output though.  Only differences which appear to me is my capture shows the same packet flowing through the different interfaces, which possibly is messing up the wireshark parser?

 

Example:

Netscreen.txt sample

6843828.0: trust(o) len=98:00121ebbd132->00600868d659/0800
              192.168.1.1 -> 192.168.1.10/6
              vhl=45, tos=00, id=37739, frag=0000, ttl=64 tlen=84
              tcp:ports 2222->2333, seq=3452113890, ack=1540618280, flag=5018/ACK
              00 60 08 68 d6 59 00 12 1e bb d1 32 08 00 45 00     .`.h.Y.....2..E.
              00 54 93 6b 00 00 40 06 63 dd c0 a8 01 01 c0 a8     .T.k..@.c.......
              01 0a 08 ae 09 1d cd c3 13 e2 5b d3 f8 28 50 18     ..........[..(P.
              1f d4 79 21 00 00 e7 76 89 64 16 e2 19 0a 80 09     ..y!...v.d......
              31 e7 04 28 04 58 f3 d9 b1 9f 3d 65 1a db d8 61     1..(.X....=e...a
              2c 21 b6 d3 20 60 0c 8c 35 98 88 cf 20 91 0e a9     ,!...`..5.......
              1d 0b                                               ..              

6843828.0: trust(i) len=102:00600868d659->00121ebbd132/0800
              192.168.1.10 -> 213.84.244.33/6
              vhl=45, tos=10, id=13195, frag=4000, ttl=64 tlen=88
              tcp:ports 22->4340, seq=1624766759, ack=4143560096, flag=5018/ACK
              00 12 1e bb d1 32 00 60 08 68 d6 59 08 00 45 10     .....2.`.h.Y..E.
              00 58 33 8b 40 00 40 06 7b dc c0 a8 01 0a d5 54     .X3.@.@.{......T
              f4 21 00 16 10 f4 60 d7 f9 27 f6 f9 b5 a0 50 18     .!....`..'....P.
              e4 20 6a ad 00 00 4d 2a 89 d2 72 85 98 75 ca 25     ..j...M*..r..u.%
              35 2e 32 73 fd b8 3a 50 11 c0 51 27 9d cf fa fa     5.2s..:P..Q'....
              c6 e1 0e 39 49 52 97 14 40 d4 f5 d1 16 2d 6a c8     ...9IR..@....-j.
              7e 90 a6 a4 e4 34                                   ~....4          

 

My capture sample:

88994.0: bgroup0(i) len=93:000d0505304a->00005e000128/0800
              172.22.227.36 -> 140.142.5.198/17
              vhl=45, tos=00, id=34271, frag=0000, ttl=128 tlen=79
              udp:ports 58041->53, len=59
              00 00 5e 00 01 28 00 0d 05 05 30 4a 08 00 45 00     ..^..(....0J..E.
              00 4f 85 df 00 00 80 11 93 2f ac 16 e3 24 8c 8e     .O......./...$..
              05 c6 e2 b9 00 35 00 3b cc 3a 00 e3 01 00 00 01     .....5.;.:......
              00 00 00 00 00 00 06 70 31 70 72 6f 64 16 63 61     .......p1prod.ca
              72 64 69 6e 61 6c 73 75 70 70 6f 72 74 63 6f 6e     rdinalsupportcon
              6e 65 63 74 03 63 6f 6d 00 00 01 00 01              nect.com.....   

88994.0: v1-untrust(o) len=93:000d0505304a->00005e000128/0800
              172.22.227.36 -> 140.142.5.198/17
              vhl=45, tos=00, id=34271, frag=0000, ttl=128 tlen=79
              udp:ports 58041->53, len=59
              00 00 5e 00 01 28 00 0d 05 05 30 4a 08 00 45 00     ..^..(....0J..E.
              00 4f 85 df 00 00 80 11 93 2f ac 16 e3 24 8c 8e     .O......./...$..
              05 c6 e2 b9 00 35 00 3b cc 3a 00 e3 01 00 00 01     .....5.;.:......
              00 00 00 00 00 00 06 70 31 70 72 6f 64 16 63 61     .......p1prod.ca
              72 64 69 6e 61 6c 73 75 70 70 6f 72 74 63 6f 6e     rdinalsupportcon
              6e 65 63 74 03 63 6f 6d 00 00 01 00 01              nect.com.....   

88994.0: ethernet0/0(o) len=93:000d0505304a->00005e000128/0800
              172.22.227.36 -> 140.142.5.198/17
              vhl=45, tos=00, id=34271, frag=0000, ttl=128 tlen=79
              udp:ports 58041->53, len=59
              00 00 5e 00 01 28 00 0d 05 05 30 4a 08 00 45 00     ..^..(....0J..E.
              00 4f 85 df 00 00 80 11 93 2f ac 16 e3 24 8c 8e     .O......./...$..
              05 c6 e2 b9 00 35 00 3b cc 3a 00 e3 01 00 00 01     .....5.;.:......
              00 00 00 00 00 00 06 70 31 70 72 6f 64 16 63 61     .......p1prod.ca
              72 64 69 6e 61 6c 73 75 70 70 6f 72 74 63 6f 6e     rdinalsupportcon
              6e 65 63 74 03 63 6f 6d 00 00 01 00 01              nect.com.....   

88994.0: ethernet0/0(i) len=172:0014f2b3b06d->000d0505304a/0800
              140.142.5.198 -> 172.22.227.36/17
              vhl=45, tos=00, id=31296, frag=0000, ttl=61 tlen=158
              udp:ports 53->58041, len=138
              00 0d 05 05 30 4a 00 14 f2 b3 b0 6d 08 00 45 00     ....0J.....m..E.
              00 9e 7a 40 00 00 3d 11 e1 7f 8c 8e 05 c6 ac 16     ..z@..=.........
              e3 24 00 35 e2 b9 00 8a c1 2c 00 e3 81 80 00 01     .$.5.....,......
              00 01 00 02 00 01 06 70 31 70 72 6f 64 16 63 61     .......p1prod.ca
              72 64 69 6e 61 6c 73 75 70 70 6f 72 74 63 6f 6e     rdinalsupportcon
              6e 65 63 74 03 63 6f 6d 00 00 01 00 01 c0 0c 00     nect.com........
              01 00 01 00 00 09 5b 00 04 d1 ca 9a 97 c0 13 00     ......[.........
              02 00 01 00 01 7a 1a 00 10 04 6e 73 33 36 08 77     .....z....ns36.w
              6f 72 6c 64 6e 69 63 c0 2a c0 13 00 02 00 01 00     orldnic.*.......
              01 7a 1a 00 07 04 6e 73 33 35 c0 54 c0 6b 00 01     .z....ns35.T.k..
              00 01 00 00 0d e3 00 04 cd b2 be 12                 ............    

88994.0: v1-trust(o) len=172:0014f2b3b06d->000d0505304a/0800
              140.142.5.198 -> 172.22.227.36/17
              vhl=45, tos=00, id=31296, frag=0000, ttl=61 tlen=158
              udp:ports 53->58041, len=138
              00 0d 05 05 30 4a 00 14 f2 b3 b0 6d 08 00 45 00     ....0J.....m..E.
              00 9e 7a 40 00 00 3d 11 e1 7f 8c 8e 05 c6 ac 16     ..z@..=.........
              e3 24 00 35 e2 b9 00 8a c1 2c 00 e3 81 80 00 01     .$.5.....,......
              00 01 00 02 00 01 06 70 31 70 72 6f 64 16 63 61     .......p1prod.ca
              72 64 69 6e 61 6c 73 75 70 70 6f 72 74 63 6f 6e     rdinalsupportcon
              6e 65 63 74 03 63 6f 6d 00 00 01 00 01 c0 0c 00     nect.com........
              01 00 01 00 00 09 5b 00 04 d1 ca 9a 97 c0 13 00     ......[.........
              02 00 01 00 01 7a 1a 00 10 04 6e 73 33 36 08 77     .....z....ns36.w
              6f 72 6c 64 6e 69 63 c0 2a c0 13 00 02 00 01 00     orldnic.*.......
              01 7a 1a 00 07 04 6e 73 33 35 c0 54 c0 6b 00 01     .z....ns35.T.k..
              00 01 00 00 0d e3 00 04 cd b2 be 12                 ............    

88994.0: bgroup0(o) len=172:0014f2b3b06d->000d0505304a/0800
              140.142.5.198 -> 172.22.227.36/17
              vhl=45, tos=00, id=31296, frag=0000, ttl=61 tlen=158
              udp:ports 53->58041, len=138
              00 0d 05 05 30 4a 00 14 f2 b3 b0 6d 08 00 45 00     ....0J.....m..E.
              00 9e 7a 40 00 00 3d 11 e1 7f 8c 8e 05 c6 ac 16     ..z@..=.........
              e3 24 00 35 e2 b9 00 8a c1 2c 00 e3 81 80 00 01     .$.5.....,......
              00 01 00 02 00 01 06 70 31 70 72 6f 64 16 63 61     .......p1prod.ca
              72 64 69 6e 61 6c 73 75 70 70 6f 72 74 63 6f 6e     rdinalsupportcon
              6e 65 63 74 03 63 6f 6d 00 00 01 00 01 c0 0c 00     nect.com........
              01 00 01 00 00 09 5b 00 04 d1 ca 9a 97 c0 13 00     ......[.........
              02 00 01 00 01 7a 1a 00 10 04 6e 73 33 36 08 77     .....z....ns36.w
              6f 72 6c 64 6e 69 63 c0 2a c0 13 00 02 00 01 00     orldnic.*.......
              01 7a 1a 00 07 04 6e 73 33 35 c0 54 c0 6b 00 01     .z....ns35.T.k..
              00 01 00 00 0d e3 00 04 cd b2 be 12                 ............    

89000.0: bgroup0(i) len=93:001517d86705->00005e000128/0800
              172.22.227.34 -> 140.142.5.198/17
              vhl=45, tos=00, id=22484, frag=0000, ttl=128 tlen=79
              udp:ports 54775->53, len=59
              00 00 5e 00 01 28 00 15 17 d8 67 05 08 00 45 00     ..^..(....g...E.
              00 4f 57 d4 00 00 80 11 c1 3c ac 16 e3 22 8c 8e     .OW......<..."..
              05 c6 d5 f7 00 35 00 3b 3b 58 9e 89 01 00 00 01     .....5.;;X......
              00 00 00 00 00 00 06 70 31 70 72 6f 64 16 63 61     .......p1prod.ca
              72 64 69 6e 61 6c 73 75 70 70 6f 72 74 63 6f 6e     rdinalsupportcon
              6e 65 63 74 03 63 6f 6d 00 00 01 00 01              nect.com.....   

 

If I take each entry, put it into a separate file and then feed those into Wireshark I can get an better idea of what's occurring.  Anyways, it appears the DNS server IS respoding with the correct address, but for some reason the SSG5 isn't handling it and then issues yet another request again for the address.

 

Looking like a ScreenOS bug?

 

Trusted Expert
Posts: 378
Registered: ‎05-12-2012
0 Kudos

Re: ScreenOS 6.3 - Connection refused by the DNS server.

I am seeing same packet going through three interfaces in this snoop, could you please explain the topology briefly.

Also I could see V1-trust which is a layer 2 interface and bgroup0 is a layer 3 interface.

So how come both are visible in snoop captuered from single firewall ?

 

As per the below output its confirmed that we are receiving a reply from DNS server, could you please run 'debug dns all' while FW is doing the lookup for some FQDN configured on it.

Trusted Expert
Posts: 378
Registered: ‎05-12-2012
0 Kudos

Re: ScreenOS 6.3 - Connection refused by the DNS server.

Also please share

get dns host cache

Contributor
Posts: 14
Registered: ‎01-13-2011
0 Kudos

Re: ScreenOS 6.3 - Connection refused by the DNS server.

SSG5 is in transparent mode. The topology:

A - Active, I - Inactive, U - Up, D - Down, R - Ready

Interfaces in vsys Root:
Name           IP Address                        Zone        MAC            VLAN State VSD
serial0/0      0.0.0.0/0                         Null        N/A               -   D   -
eth0/0         0.0.0.0/0                         V1-Untrust  001b.c0c3.e300    -   U   -
bgroup0        0.0.0.0/0                         V1-Trust    001b.c0c3.e30b    -   U   -
  eth0/1       N/A                               N/A         N/A               -   D   -
  eth0/2       N/A                               N/A         N/A               -   D   -
  eth0/3       N/A                               N/A         N/A               -   U   -
  eth0/4       N/A                               N/A         N/A               -   D   -
  eth0/5       N/A                               N/A         N/A               -   U   -
  eth0/6       N/A                               N/A         N/A               -   U   -
bgroup1        0.0.0.0/0                         Null        001b.c0c3.e30c    -   D   -
bgroup2        0.0.0.0/0                         Null        001b.c0c3.e30d    -   D   -
bgroup3        0.0.0.0/0                         Null        001b.c0c3.e30e    -   D   -
vlan1          172.22.227.31/24                  VLAN        001b.c0c3.e30f    1   U   -
null           0.0.0.0/0                         Null        N/A               -   U   -

 

DNS cache:

DNS Server:
  Primary  :        140.142.5.198, Src Interface: Null
  Secondary:        140.142.5.214, Src Interface: Null
  Tertiary :      128.208.169.203, Src Interface: Null
DNS Cache (Static):
DNS Cache (Dynamic):
DNS Cache (Unresolved):
Host name: tick.cac.washington.edu last IP: 0.0.0.0
Host name: p1qa.cardinalsupportconnect.com last IP: 0.0.0.0
Host name: ca1.CardinalSupportConnect.com last IP: 0.0.0.0
Host name: bo2.CardinalSupportConnect.com last IP: 0.0.0.0
Host name: bo1qa.cardinalsupportconnect.com last IP: 0.0.0.0
Host name: sj1.CardinalSupportConnect.com last IP: 0.0.0.0
Host name: its-njb-rx1.MYDOMAIN last IP: 0.0.0.0
Host name: bo1.CardinalSupportConnect.com last IP: 0.0.0.0
Host name: sj2.CardinalSupportConnect.com last IP: 0.0.0.0
Host name: ca2.CardinalSupportConnect.com last IP: 0.0.0.0
Host name: its-sp-rx1.MYDOMAIN last IP: 0.0.0.0
Host name: P1prod.CardinalSupportConnect.com last IP: 0.0.0.0
Host name: itsvm-rx-prn.MYDOMAIN last IP: 0.0.0.0
Host name: tock.cac.washington.edu last IP: 0.0.0.0

 The current settings don’t prohibit DNS lookups from functioning in the CLI.

SSG5-> ping www.juniper.net
Type escape sequence to abort

Sending 5, 100-byte ICMP Echos to www.juniper.net [69.192.199.148], timeout is 1 seconds
!!!!!
Success Rate is 100 percent (5/5), round-trip time min/avg/max=3/3/3 ms
SSG5-> ping its-sp-rxfs.MYDOMAIN
Type escape sequence to abort

Sending 5, 100-byte ICMP Echos to its-sp-rxfs.MYDOMAIN [140.142.xxx.xxx], timeout is 1 seconds
!!!!!
Success Rate is 100 percent (5/5), round-trip time min/avg/max=3/3/4 ms
SSG5->

 

 

Trusted Expert
Posts: 378
Registered: ‎05-12-2012
0 Kudos

Re: ScreenOS 6.3 - Connection refused by the DNS server.

I see normal DNS resolution works just fine. So you are only facing issue where FQDNs configured in policies are not getting resolved. One last thing,  try disabling DNS Alg.

 

If it doesn't then It could be some OS issue only and I would suggest opening a case with JTAC.

Contributor
Posts: 14
Registered: ‎01-13-2011
0 Kudos

Re: ScreenOS 6.3 - Connection refused by the DNS server.

That statement is correct.

 

I disabled DNS ALG globally, but it does not appear to have any immedate affect in resolving the problem.

 

I just opened a JTAC case yesterday. Progress is slow so far.  Was hoping this was an easy problem/oversight which could be quickly resolved.  It's appearing to not be the case.

Visitor
Posts: 3
Registered: ‎02-07-2009
0 Kudos

Re: ScreenOS 6.3 - Connection refused by the DNS server.

Any news from JTAC?

Contributor
Posts: 81
Registered: ‎08-09-2010
0 Kudos

Re: ScreenOS 6.3 - Connection refused by the DNS server.

Don't know if this is of any help or not, or if it is related:-

 

After upgrading a cluster of SRX 5500's recently, we noticed that the DNS service ceased functioning.

 

Upon investigation we found that it was disabled and needed re-including in the Policies with Juniper-dns. Once we enabled this allowing through the correct port it all worked fine.

 

Might help with this but I dont know.

Contributor
Posts: 14
Registered: ‎01-13-2011
0 Kudos

Re: ScreenOS 6.3 - Connection refused by the DNS server.


sinu42 wrote:

Any news from JTAC?


Still attempting to proove there is a problem...

 

 

adgwytc:

Do you use NSM to deploy policies?  At least according to NSM, there are no policy differnces between the ScreenOS 5.4 and 6.3 devices.

Super Contributor
Posts: 146
Registered: ‎02-08-2008
0 Kudos

Re: ScreenOS 6.3 - Connection refused by the DNS server.

I noticed that the DNS queries in your snoop are coming from the IPs 172.22.227.36 and .34, but they don't appear to come from the firewall itself (.31), so these packets may be coming from your LAN hosts.

What happens if you try to set the source interface in the DNS config to vlan1?

Contributor
Posts: 14
Registered: ‎01-13-2011
0 Kudos

Re: ScreenOS 6.3 - Connection refused by the DNS server.

Changing the DNS src-interface to vlan1 doesn't help.

 

Actually, rebooting the firewall temporarily fixes the problem.  A co-worker rebooted it almost 2 days ago and the problem vanished.  No policy changes have been made.  Now I wait again for the problem to surface...

Visitor
Posts: 3
Registered: ‎02-07-2009
0 Kudos

Re: ScreenOS 6.3 - Connection refused by the DNS server.

I am facing a similiar isssue. From time to time no dns resolution is possible "Connection refused by the DNS server". But I f I change only one dns servers ip address (out of three) and apply the changes, it is working again.

I am running 6.3R10

Contributor
Posts: 14
Registered: ‎01-13-2011
0 Kudos

Re: ScreenOS 6.3 - Connection refused by the DNS server.

[ Edited ]

Finally, Juniper issued a patch (ssg5ssg20.6.3.0r11-cpb1.0) for this issue which I'm evaluating now.  At this point looks like it can make it into 6.3.0r13.

 

Root Cause per TSE:

if arp task failed to update session outgoing interface when receive arp responce, box will keep dropping packets match this session. Solution: in this scenario, we need invalid this session and let create a new session again.

Highlighted
Trusted Expert
Posts: 378
Registered: ‎05-12-2012
0 Kudos

Re: ScreenOS 6.3 - Connection refused by the DNS server.

Great, al last we have solution for this issue Smiley Happy Please evaluate this patch and let the Forum know how it worked.
Contributor
Posts: 14
Registered: ‎01-13-2011
0 Kudos

Re: ScreenOS 6.3 - Connection refused by the DNS server.

Received confirmation from enginnering that this problem (JUNOS Defect: 797786), will be fixed in 6.3.0r13.

Contributor
Posts: 192
Registered: ‎06-17-2008
0 Kudos

Re: ScreenOS 6.3 - Connection refused by the DNS server.

So did 6.3.0r13 solve the issue ?

 


Best Regards

Tom Roholm
JNCIS-ENT, FWV, SEC, SA, WLAN
Contributor
Posts: 14
Registered: ‎01-13-2011

Re: ScreenOS 6.3 - Connection refused by the DNS server.


TRK-NKA wrote:

So did 6.3.0r13 solve the issue ?

 


Yes, it has.

 

I noticed the JunOS defect # wasnt listed in the r13 release notes. Called JTAC back and had them validate it truly was included, but missed the release notes.

Visitor
Posts: 6
Registered: ‎04-15-2014
0 Kudos

Re: ScreenOS 6.3 - Connection refused by the DNS server.

I have three SSG5 firewalls running 6.3.0r16a-dfj1.0 that are also having this problem. Clearing the sessions to the DNS servers solves the problem. Anybody else having this with 6.3.0r16+?