[ntp:questions] Isolated Network Drift Problem
Unruh
unruh-spam at physics.ubc.ca
Fri Nov 21 02:48:42 UTC 2008
cwebster at ec.rr.com (Cal Webster) writes:
>Our NTP servers are slowly loosing time. All are in nearly perfect sync
>but collectively drift backwards over time. Is there a way to apply a
>bias to the drift calculations?
>We had to disconnect from the Internet several months ago. Since then we
>have had serious drift problems. Shortly after the disconnect I
>discovered that we were predictably loosing 10 minutes every 15 days. I
>tried several things but not until I zeroed out the
>"driftfile" (/var/lib/ntp/drift) 10 days ago [Mon Nov 10 18:10:00 2008]
>did this large drift abate.
You might want to try using chrony assuming your machines are Linux
machines. One of the features of chrony is the ability to put in "watch"
time-- ie manually set the time. Now this will NOT give you ms accuracy,
but it will use those times to discipline the drift of the clocks just as
it would with regular ntp servers. 10min/15days is about 1000PPM which
ntp would not be able to fix even if it were running.
alternatively you can do the drift yourself. Look at the adjtimex console
command. You can alter the tick rate to try to bring the main server
(jato?) rate into reasonable agreemetn with the truth.
>Although it is much improved, we are still steadily loosing time. Three
>days after I zeroed the drift file [Thu Nov 13 15:04:00 EST 2008] we
>were 32 seconds behind. Today, 10 days later [Thu Nov 20 09:05:00 2008]
>we are 1 min 54 secs behind. This works out to roughly 12 secs per day -
>not bad I guess but still requires regular monitoring.
That is about 25PPM.
>Any suggestions would be welcome. I've outlined our NTP/Network topology
>and configuration below, as well as the output of some time reference
>tools that might be relevant.
>Let me know if you need more info. Thank you in advance!
>Cal Webster
>##############
>## Topology ##
>##############
>Notes:
>1. Links between routers are busy ADSL lines with approx 512 Mbps
>throughput.
>2. Host names are shown in parenthesis for cross-referencing data.
>[router]---[LAN A]
> | |
> | +--[NTP 1 (axl)]
> |
>[router]---[LAN B]---[LAN B.1]
> | |
> | +--[NTP 2 (pegasus)]
> |
>[router]---[LAN C]
> |
> +--[NTP 3 (jato)]
> +--[NTP 4 (fluid)]
>###################
>## Configuration ##
>###################
>[root at jato ~]# egrep -v '^[[:space:]]*$|^#' /etc/ntp.conf
>---------------------------------------------------------
>restrict default notrust nomodify notrap noquery
>restrict 127.0.0.1
>restrict 192.168.1.0 mask 255.255.255.0 nomodify notrap
>restrict 192.168.2.0 mask 255.255.255.0 nomodify notrap
>restrict 192.168.3.0 mask 255.255.255.0 nomodify notrap
>restrict 10.1.150.0 mask 255.255.255.0 nomodify notrap
>restrict 192.168.1.30 mask 255.255.255.255
>restrict 192.168.2.6 mask 255.255.255.255
>restrict 192.168.3.132 mask 255.255.255.255
>peer 192.168.3.132
>peer 192.168.1.30
>peer 192.168.2.6
>server 127.127.1.0
>fudge 127.127.1.0 stratum 5
>driftfile /var/lib/ntp/drift
>broadcastdelay 0.008
>keys /etc/ntp/keys
>---------------------------------------------------------
>[root at pegasus ~]# egrep -v '^[[:space:]]*$|^#' /etc/ntp.conf
>---------------------------------------------------------
>restrict default notrust nomodify notrap noquery
>restrict 127.0.0.1
>restrict 192.168.1.0 mask 255.255.255.0 nomodify notrap
>restrict 192.168.2.0 mask 255.255.255.0 nomodify notrap
>restrict 192.168.3.0 mask 255.255.255.0 nomodify notrap
>restrict 10.1.150.0 mask 255.255.255.0 nomodify notrap
>restrict 192.168.3.9 mask 255.255.255.255
>restrict 192.168.3.132 mask 255.255.255.255
>restrict 192.168.1.30 mask 255.255.255.255
>peer 192.168.3.9
>peer 192.168.3.132
>peer 192.168.1.30
>server 127.127.1.0 # local clock
>fudge 127.127.1.0 stratum 8
>driftfile /var/lib/ntp/drift
>broadcastdelay 0.008
>keys /etc/ntp/keys
>---------------------------------------------------------
>[root at fluid root]# egrep -v '^[[:space:]]*$|^#' /etc/ntp.conf
>---------------------------------------------------------
>restrict 127.0.0.1
>restrict 192.168.1.0 mask 255.255.255.0 nomodify notrap
>restrict 192.168.2.0 mask 255.255.255.0 nomodify notrap
>restrict 192.168.3.0 mask 255.255.255.0 nomodify notrap
>restrict 10.1.150.0 mask 255.255.255.0 nomodify notrap
>restrict 192.168.3.9 mask 255.255.255.255
>restrict 192.168.2.6 mask 255.255.255.255
>restrict 192.168.1.30 mask 255.255.255.255
>peer 192.168.3.9
>peer 192.168.1.30
>peer 192.168.2.6
>server 127.127.1.0 # local clock
>fudge 127.127.1.0 stratum 5
>driftfile /etc/ntp/drift
>broadcastdelay 0.008
>keys /etc/ntp/keys
>---------------------------------------------------------
>[root at axl ~]# egrep -v '^[[:space:]]*$|^#' /etc/ntp.conf
>---------------------------------------------------------
>restrict 127.0.0.1
>restrict 192.168.1.0 mask 255.255.255.0 nomodify notrap
>restrict 192.168.2.0 mask 255.255.255.0 nomodify notrap
>restrict 192.168.3.0 mask 255.255.255.0 nomodify notrap
>restrict 10.1.150.0 mask 255.255.255.0 nomodify notrap
>restrict 192.168.3.9 mask 255.255.255.255
>restrict 192.168.3.132 mask 255.255.255.255
>restrict 192.168.2.6 mask 255.255.255.255
>peer 192.168.3.9
>peer 192.168.3.132
>peer 192.168.2.6
>server 127.127.1.0 # local clock
>fudge 127.127.1.0 stratum 5
>driftfile /var/lib/ntp/drift
>broadcastdelay 0.008
>keys /etc/ntp/keys
>---------------------------------------------------------
>##############
>## Topology ##
>##############
>Notes:
>1. Links between routers are ADSL lines with approx 512 Mbps throughput.
>2. Host names are shown in parenthesis for cross-referencing data.
>[router]---[LAN A]
> | |
> | +--[NTP 1 (axl)]
> |
>[router]---[LAN B]---[LAN B.1]
> | |
> | +--[NTP 2 (pegasus)]
> |
>[router]---[LAN C]
> |
> +--[NTP 3 (jato)]
> +--[NTP 4 (fluid)]
>###################
>## Configuration ##
>###################
>[root at jato ~]# egrep -v '^[[:space:]]*$|^#' /etc/ntp.conf
>---------------------------------------------------------
>restrict default notrust nomodify notrap noquery
>restrict 127.0.0.1
>restrict 192.168.1.0 mask 255.255.255.0 nomodify notrap
>restrict 192.168.2.0 mask 255.255.255.0 nomodify notrap
>restrict 192.168.3.0 mask 255.255.255.0 nomodify notrap
>restrict 10.1.150.0 mask 255.255.255.0 nomodify notrap
>restrict 192.168.1.30 mask 255.255.255.255
>restrict 192.168.2.6 mask 255.255.255.255
>restrict 192.168.3.132 mask 255.255.255.255
>peer 192.168.3.132
>peer 192.168.1.30
>peer 192.168.2.6
>server 127.127.1.0
>fudge 127.127.1.0 stratum 5
>driftfile /var/lib/ntp/drift
>broadcastdelay 0.008
>keys /etc/ntp/keys
>---------------------------------------------------------
>[root at pegasus ~]# egrep -v '^[[:space:]]*$|^#' /etc/ntp.conf
>---------------------------------------------------------
>restrict default notrust nomodify notrap noquery
>restrict 127.0.0.1
>restrict 192.168.1.0 mask 255.255.255.0 nomodify notrap
>restrict 192.168.2.0 mask 255.255.255.0 nomodify notrap
>restrict 192.168.3.0 mask 255.255.255.0 nomodify notrap
>restrict 10.1.150.0 mask 255.255.255.0 nomodify notrap
>restrict 192.168.3.9 mask 255.255.255.255
>restrict 192.168.3.132 mask 255.255.255.255
>restrict 192.168.1.30 mask 255.255.255.255
>peer 192.168.3.9
>peer 192.168.3.132
>peer 192.168.1.30
>server 127.127.1.0 # local clock
>fudge 127.127.1.0 stratum 8
>driftfile /var/lib/ntp/drift
>broadcastdelay 0.008
>keys /etc/ntp/keys
>---------------------------------------------------------
>[root at fluid root]# egrep -v '^[[:space:]]*$|^#' /etc/ntp.conf
>---------------------------------------------------------
>restrict 127.0.0.1
>restrict 192.168.1.0 mask 255.255.255.0 nomodify notrap
>restrict 192.168.2.0 mask 255.255.255.0 nomodify notrap
>restrict 192.168.3.0 mask 255.255.255.0 nomodify notrap
>restrict 10.1.150.0 mask 255.255.255.0 nomodify notrap
>restrict 192.168.3.9 mask 255.255.255.255
>restrict 192.168.2.6 mask 255.255.255.255
>restrict 192.168.1.30 mask 255.255.255.255
>peer 192.168.3.9
>peer 192.168.1.30
>peer 192.168.2.6
>server 127.127.1.0 # local clock
>fudge 127.127.1.0 stratum 5
>driftfile /etc/ntp/drift
>broadcastdelay 0.008
>keys /etc/ntp/keys
>---------------------------------------------------------
>[root at axl ~]# egrep -v '^[[:space:]]*$|^#' /etc/ntp.conf
>---------------------------------------------------------
>restrict 127.0.0.1
>restrict 192.168.1.0 mask 255.255.255.0 nomodify notrap
>restrict 192.168.2.0 mask 255.255.255.0 nomodify notrap
>restrict 192.168.3.0 mask 255.255.255.0 nomodify notrap
>restrict 10.1.150.0 mask 255.255.255.0 nomodify notrap
>restrict 192.168.3.9 mask 255.255.255.255
>restrict 192.168.3.132 mask 255.255.255.255
>restrict 192.168.2.6 mask 255.255.255.255
>peer 192.168.3.9
>peer 192.168.3.132
>peer 192.168.2.6
>server 127.127.1.0 # local clock
>fudge 127.127.1.0 stratum 5
>driftfile /var/lib/ntp/drift
>broadcastdelay 0.008
>keys /etc/ntp/keys
>---------------------------------------------------------
>##########################
>## Current Drift Values ##
>##########################
>[root at jato /]# cat /var/lib/ntp/drift
>-------------------------------------
>-99.534
>-------------------------------------
>[root at pegasus /]# cat /var/lib/ntp/drift
>-------------------------------------
>-26.817
>-------------------------------------
>[root at fluid /]# cat /etc/ntp/drift
>-------------------------------------
>-81.140
>-------------------------------------
>[root at axl /]# cat /var/lib/ntp/drift
>-------------------------------------
>17.653
>-------------------------------------
>##################
>## NTPQ Queries ##
>##################
>[root at jato backup]# ntpq
>------------------------
>ntpq> pe
> remote refid st t when poll reach delay offset
>jitter
>==============================================================================
> fluid 192.168.3.9 7 u 831 1024 376 0.368 -0.896
>1.275
> axl 192.168.3.9 7 u 951 1024 376 25.263 -0.776
>0.464
> pegasus 192.168.3.9 7 u 28 1024 377 24.597 -1.228
>0.298
>*LOCAL(0) LOCAL(0) 5 l 8 64 377 0.000 0.000
>0.004
>ntpq> as
>
>ind assID status conf reach auth condition last_event cnt
>===========================================================
> 1 46140 9014 yes yes none reject reachable 1
> 2 46141 9034 yes yes none reject reachable 3
> 3 46142 9054 yes yes none reject reachable 5
> 4 46143 9614 yes yes none sys.peer reachable 1
>------------------------
>[root at pegasus ECP Data]# ntpq
>------------------------
>ntpq> pe
> remote refid st t when poll reach delay offset
>jitter
>==============================================================================
>*jato LOCAL(0) 6 u 59 1024 376 24.853 1.356
>0.976
>+fluid 192.168.3.9 7 u 352 1024 377 24.737 0.568
>0.159
>+axl 192.168.3.9 7 u 194 1024 377 32.219 0.475
>0.342
> LOCAL(0) LOCAL(0) 8 l 41 64 377 0.000 0.000
>0.001
>ntpq> as
>
>ind assID status conf reach auth condition last_event cnt
>===========================================================
> 1 37660 96f4 yes yes none sys.peer reachable 15
> 2 37661 94f4 yes yes none candidat reachable 15
> 3 37662 94f4 yes yes none candidat reachable 15
> 4 37663 90b4 yes yes none reject reachable 11
>------------------------
>[root at fluid cwebster]# ntpq
>------------------------
>ntpq> pe
> remote refid st t when poll reach delay offset
>jitter
>==============================================================================
>*jato LOCAL(0) 6 u 138 1024 377 0.395 0.883
>0.494
>+axl jato 7 u 897 1024 376 27.010 -1.833
>0.974
>+pegasus jato 7 u 884 1024 376 25.082 -0.396
>0.298
> LOCAL(0) LOCAL(0) 5 l 35 64 377 0.000 0.000
>0.008
>ntpq> as
>ind assID status conf reach auth condition last_event cnt
>===========================================================
> 1 47588 9694 yes yes none sys.peer reachable 9
> 2 47589 94f4 yes yes none candidat reachable 15
> 3 47590 9494 yes yes none candidat reachable 9
> 4 47591 9074 yes yes none reject reachable 7
>------------------------
>[root at axl /]# ntpq
>------------------------
>ntpq> pe
> remote refid st t when poll reach delay offset
>jitter
>==============================================================================
>*jato LOCAL(0) 6 u 674 1024 377 24.492 1.161
>0.082
> fluid 192.168.3.9 7 u 846 1024 377 24.673 0.665
>1.095
> pegasus 192.168.3.9 7 u 523 1024 376 32.235 -0.018
>0.340
> LOCAL(0) .LOCL. 5 l 23 64 377 0.000 0.000
>0.001
>ntpq> as
>
>ind assID status conf reach auth condition last_event cnt
>===========================================================
> 1 23414 96a4 yes yes none sys.peer reachable 10
> 2 23415 9064 yes yes none reject reachable 6
> 3 23416 9084 yes yes none reject reachable 8
> 4 23417 9064 yes yes none reject reachable 6
>------------------------
>#########################
>## Hardware Clock Info ##
>#########################
>Note: There are no precision clocks. All NTP servers use standard quartz
>based clocks on system boards.
>[root at jato backup]# cat /etc/adjtime
>------------------------------------
>0.146704 1226952522 0.000000
>1226952522
>LOCAL
>------------------------------------
>[root at jato backup]# hwclock --test --debug
>------------------------------------------
>hwclock from util-linux-2.12a
>Using /dev/rtc interface to clock.
>Last drift adjustment done at 1226952522 seconds after 1969
>Last calibration done at 1226952522 seconds after 1969
>Hardware clock is on local time
>Assuming hardware clock is kept in local time.
>Waiting for clock tick...
>...got clock tick
>Time read from Hardware Clock: 2008/11/20 09:07:56
>Hw clock time : 2008/11/20 09:07:56 = 1227190076 seconds since 1969
>Thu 20 Nov 2008 09:07:56 AM EST -0.852202 seconds
>------------------------------------------
>[root at pegasus ECP Data]# cat /etc/adjtime
>------------------------------------
>27.446035 1226358312 0.000000
>1226358312
>LOCAL
>------------------------------------
>[root at pegasus ECP Data]# hwclock --test --debug
>------------------------------------------
>hwclock from util-linux-2.12a
>Using /dev/rtc interface to clock.
>Last drift adjustment done at 1226358312 seconds after 1969
>Last calibration done at 1226358312 seconds after 1969
>Hardware clock is on local time
>Assuming hardware clock is kept in local time.
>Waiting for clock tick...
>...got clock tick
>Time read from Hardware Clock: 2008/11/20 09:07:55
>Hw clock time : 2008/11/20 09:07:55 = 1227190075 seconds since 1969
>Thu 20 Nov 2008 09:07:55 AM EST -0.016559 seconds
>------------------------------------------
>[root at fluid cwebster]# cat /etc/adjtime
>------------------------------------
>-0.025420 1226360186 0.000000
>1226360186
>LOCAL
>------------------------------------
>[root at fluid cwebster]# hwclock --test --debug
>------------------------------------------
>hwclock from util-linux-2.11y
>Using /dev/rtc interface to clock.
>Last drift adjustment done at 1226360186 seconds after 1969
>Last calibration done at 1226360186 seconds after 1969
>Hardware clock is on local time
>Assuming hardware clock is kept in local time.
>Waiting for clock tick...
>...got clock tick
>Time read from Hardware Clock: 2008/11/20 09:07:56
>Hw clock time : 2008/11/20 09:07:56 = 1227190076 seconds since 1969
>Thu 20 Nov 2008 09:07:56 AM EST -0.232158 seconds
>------------------------------------------
>[root at axl /]# cat /etc/adjtime
>------------------------------------
>44.508790 1226358437 0.000000
>1226358437
>LOCAL
>------------------------------------
>[root at axl /]# hwclock --test --debug
>------------------------------------------
>hwclock from util-linux-ng 2.13
>Using /dev interface to clock.
>Last drift adjustment done at 1226358437 seconds after 1969
>Last calibration done at 1226358437 seconds after 1969
>Hardware clock is on local time
>Assuming hardware clock is kept in local time.
>Waiting for clock tick...
>...got clock tick
>Time read from Hardware Clock: 2008/11/20 09:07:56
>Hw clock time : 2008/11/20 09:07:56 = 1227190076 seconds since 1969
>Thu 20 Nov 2008 09:07:56 AM EST -0.001092 seconds
>------------------------------------------
More information about the questions
mailing list