Switching

I found the NTP process is stuck on EX.

I have deactivated NTP by issuing a “ deactivate system ntp “ in the JunOS CLI, which shut down the xntpd process, but there is a tnp.sntpd or tnp.sntpc that is used by NTP, remains stuck:

root@SW01:RE:0% ps aux | grep -i tnp
root    1289  0.0  0.0  2264   992  ??  I    10Oct14   0:02.59 /usr/sbin/tnp.sn    <————————— This process is stuck, refuses to quit by itself
root   46764  0.0  0.1  2392  1064  u0  S+    9:10PM   0:00.01 grep -i tnp
root@SW01:RE:0%

Which would explain these log messages:

Oct 29 21:05:21  SW01 xntpd[46408]: bind() fd 5, family 2, port 123, addr 0.0.0.0, in_classd=0 flags=8 fails: Address already in use
Oct 29 21:05:22  SW01 xntpd[46408]: setsockopt IP_MULTICAST_TTL fails on address 0.0.0.0: Bad file descriptor
Oct 29 21:05:22  SW01 xntpd[46408]: sendto(10.1.21.254): Bad file descriptor


Normally, like a Unix box, you can issue a “ kill 1289 “ to kill off the process. But again, its a switch in production use. I’m uneasy issuing that command…

So,does someone can help me with it. Is there any impact if I use kill command on the EX.

Thanks.

Why do you think the logs are related to that process? The logs are generated by process 46408.

tnp.sntp is not going away even after a reboot after ntp was completely disabled:

{master:0}[edit]
lab@ex4200# show system | display set | match ntp | except server
set system processes ntp disable
deactivate system ntp

{master:0}[edit]
lab@ex4200#

After reboot:

root@ex4200:RE:0% ps axu | grep ntp
root  1305  0.0  0.1  2264   992  ??  S     8:55AM   0:00.01 /usr/sbin/tnp.sntp
root  1400  0.0  0.1  2388  1056  u0  S+    8:56AM   0:00.01 grep ntp
root@ex4200:RE:0%

=====

If this worked for you please flag my post as an "Accepted Solution" so others can benefit. A kudo would be cool if you think I earned it.

I still have this odd issue. Please see my output from switch. The 10.1.21.254 is firewall VIP. I already have rule permit port 123. Do you have any ideals? Thank you.

root@SW01> show configuration system ntp | display set
set system ntp server 10.1.21.254

{master:0}
root@SW01> show configuration forwarding-options | display set
set forwarding-options helpers port 123 server 10.1.1.254
set forwarding-options helpers port 123 interface vlan.256

{master:0}
root@SW01> show ntp status                                        
localhost: timed out, nothing received
***Request timed out

{master:0}
root@SW01> ping 10.1.21.254
PING 10.1.21.254 (10.1.21.254): 56 data bytes
64 bytes from 10.1.21.254: icmp_seq=0 ttl=64 time=1.234 ms
64 bytes from 10.1.21.254: icmp_seq=1 ttl=64 time=1.240 ms
64 bytes from 10.1.21.254: icmp_seq=2 ttl=64 time=1.463 ms
^C
--- 10.1.21.254 ping statistics ---
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max/stddev = 1.234/1.312/1.463/0.107 ms

Hi there,

---

@eric.song wrote:

 

I still have this odd issue. 

 

---

No, You are having a different one.

---

@eric.song wrote:

 

 The 10.1.21.254 is firewall VIP. I already have rule permit port 123. Do you have any ideals? Thank you.

 

 

 

root@SW01> show configuration system ntp | display set
set system ntp server 10.1.21.254

---

Did You check this FW VIP actually can serve time via NTP? If it cannot, please choose a different time server. If You are labbing this up and don't have internet access/don't want your lab gear to go out to internet, You can make any Windows PC to become an NTP server, there are plenty of instructions on google.

---

@eric.song wrote:

 

 

{master:0}
root@SW01> show configuration forwarding-options | display set
set forwarding-options helpers port 123 server 10.1.1.254
set forwarding-options helpers port 123 interface vlan.256

 

---

This one is for BROADCAST NTP packets from OTHER machines. Locally-generated traffic, broadcast or not, is not captured by IP helper.

---

@eric.song wrote:

 

 

{master:0}
root@SW01> show ntp status                                        
localhost: timed out, nothing received
***Request timed out

 

---

Please try:

1/ disable Your lo0.0 filter if You have any, and

2/ add this line to the config 

set system static-host-mapping localhost inet 127.0.0.1

3/ run "show ntp status no-resolve"

- and then report back.

HTH

Thanks
Alex

Thank you so much for your help.

The firewall VIP is working for sure. I've tested it with my laptop, I can get the time from firewall VIP. Also I've configured other EX switch using NTP server as 10.1.21.254 (VIP). It is getting time from there. I had NTP attacks from ouside router before. The attacks were stoped when I disabled the interface on EX.

I confirmed with Juniper J-TAC, it is means NTP not configured if I get the output as below.

root@SW01> show ntp status                                        
localhost: timed out, nothing received
***Request timed out

So I don't have any filter on the EX, this is our producation environment. We have l3-interface routing enabled and many things on it. I don't know I will have some risk if I add the command line on the EX.

"set system static-host-mapping localhost inet 127.0.0.1"

If the xntp session is ok for Juniper EX, I have no ideal what is cause and how to solve it.

Thanks again.

Eric

Hello,

---

@eric.song wrote:

 

 

I confirmed with Juniper J-TAC, it is means NTP not configured if I get the output as below.

root@SW01> show ntp status                                        
localhost: timed out, nothing received
***Request timed out

 

 

 

Eric

 

 

---

This is only half of the story. The other half is that this command produces same output when UDP port 123 is blocked for: (a) NTP source-address and (b) 127.0.0.1 if no NTP source-address is configured, in lo0.0 filter.

That's why I asked You to disable lo0 filter.

These two situations can be discerned with another command:

1/ when NTP is disabled, the following printout applies:

aarseniev@m7i> set date ntp 
Nov 01 18:13:47
 1 Nov 18:13:47 ntpdate[32415]: no servers can be used, exiting

 2/ when UDP port 123 is blocked in lo0.0 filter

aarseniev@rm7i> set date ntp    
Nov 01 18:16:19
 1 Nov 18:16:19 ntpdate[32761]: no server suitable for synchronization found

---

@eric.song wrote:

 

 I don't know I will have some risk if I add the command line on the EX.

 

"set system static-host-mapping localhost inet 127.0.0.1"

 

 

 

 

 

---

The "risk" is that when any JUNOS CLI command referencing "localhost" is run (which is almost any "show ntp..." command), instead of querying Your DNS server for 127.0.0.1, the resolution will use this static mapping. Your DNS server may not have any idea how to resolve "localhost", hence commands previously hanging before will mysteriously start working.

HTH

Thanks
Alex

Hi, 

Thanks for your inputs in this discussion.

I am a bit struggling to understand - "This one is for BROADCAST NTP packets from OTHER machines. Locally-generated traffic, broadcast or not, is not captured by IP helper."

Do you mean locally-generated traffic as within the same subnet? OR device specific internal generated traffic? What do you mean, OTHER machines here?

Thank you. 

Hello,

---

@eric.song wrote:

I found the NTP process is stuck on EX.

I have deactivated NTP by issuing a “ deactivate system ntp “ in the JunOS CLI, which shut down the xntpd process, but there is a tnp.sntpd or tnp.sntpc that is used by NTP, remains stuck:

 

 

 

---

This process is not stuck, it is used by EX cluster master to serve time to the other cluster members not via IP but via internal Juniper-proprietary transport protocol called TNP.

---

@eric.song wrote:

Normally, like a Unix box, you can issue a “ kill 1289 “ to kill off the process. But again, its a switch in production use. I’m uneasy issuing that command…

 

So,does someone can help me with it. Is there any impact if I use kill command on the EX.

 

Thanks.

 

 

 

---

If You kill this process, it will be re-spawned again. But it has nothing to do with Your syslog messages, as already pointed out by others.

HTH

Thanks
Alex