[ntp:questions] ARRGH!!! I woke up to a 50 SECOND clock error.
Ron Frazier (NTP)
timekeepingntplist at c3energy.com
Sat Mar 17 00:25:16 UTC 2012
Hi Charles,
I cannot answer your questions about filters, or the intrinsic function
of NTP. However, I just wanted to show you this graph of my BU-353 if
you hadn't seen it.
http://dl.dropbox.com/u/9879631/drifting01-peerstats.20120312.jpg
The computer is locked to GPS time, and the internet servers are
noselected and appear to be drifting away. I cannot say for sure, but
it seems more likely that GPS NMEA sentence start time is drifting
away. David Taylor says a similar phenomenon has been observed in the
Garmin 18??. Because the GPS is the only selectable time server in my
case, for testing, it's always going to be on the zero baseline. That
would have the same appearance as if the internet servers were
drifting. This has been causing me great consternation. So, depending
on how long it's been since you cycled power to your GPS, it may have an
offset away from the internet servers. If this is intrinsic to the
device, it is useless for timekeeping over a period of several days to
an accuracy of less than 130 ms or so, even though each individual
sample from the GPS is accurate to a few ms of what the GPS THINKS is
true time. From our prior discussions, we already know your GPS has
heart attacks like the one shown in this graph. Note, you COULD keep
all the computers aligned to EACH OTHER within a few ms. They just
won't necessarily be aligned to UTC any better than the internet servers
could do.
Assuming you're running windows, if you'd like to make some test graphs
for yourself, you can do the following. You could do similar things in
Linux, but not as easy. Note that this graph took several days to
produce. A new stats file will be created at 0000 UTC each day. After
that point, you will no longer see new updates on your graph. Close the
graphing program, then drag files to it again, including the latest file
that was just created. You can graph files that are currently being
updated by NTP.
1) Put the following in your ntp.conf to get loopstats and peerstats
files. Customize the directory for your systsem.
###########################################################
# Enable statistics collection.
enable stats
statsdir "C:\NTP Service\NTP\etc\"
statistics loopstats peerstats
###########################################################
2) For testing only, set the GPS to be the sole clock source by putting
noselect on each internet server line. As an alternative, you can put
prefer on your GPS line. However, NTP may clock hop away from the GPS
when the offsets compared to internet servers get big.
2) Start the NTP service and let it run for a few minutes.
3) Get David Taylor's NTP Plotter, which has now been enhanced to plot
peerstats files.
http://www.satsignal.eu/software/NTPplotter.zip
Unzip it and put it in a folder some where. Make a shortcut to the
executable and put that on your desktop.
4) Open Windows explorer and go to the folder where your stats files
are. Select the peerstats file that has just appeared if you've never
captured them before. If you have multiple peerstats, you can select
several.
5) Drag the selected files and drop them on top of the NTP Plotter
shortcut. If the NTP Plotter program is open, you can drop files onto
it too.
6) The NTP Plotter program starts up. Click the peerstats tab and then
the charts tab and select offsets.
7) On the lower right of the chart program, click the auto update drop
down and select a time if you want the chart to auto update. I use 1
minute.
8) You should start seeing a graph for your GPS offsets. It may be
jaggy, depending on your sample interval. Most likely at a longer
interval, you should see the internet peers updating on the chart. Over
time, you can build a graph like mine, which will show the GPS's offsets
from the computer's clock as well as the internet servers' offsets from
the computer's clock. Consequently, you can also see the GPS's offsets
compared to the internet servers' offsets compared to each other,
although, if one is drifting, you can't tell which it is. I'm assuming
all the internet servers are not drifting at the same time.
It would be interesting to compare one of these graphs from your GPS to
mine. In my experience, you can end a GPS heart attack by stopping NTP,
powering the GPS off and back on after 30 seconds or so, and restarting
NTP. However, letting it ride through the heart attack gives you to
graph some things on the other side, as I did in the aforementioned
graph quite by accident.
Once I get my Sure board, I'm going to make it my primary GPS and
compare other stuff to it.
Sincerely,
Ron
On 3/16/2012 7:07 PM, Charles Elliott wrote:
> On the subject of accuracy, has anyone ever really looked at NTPD's offset
> filtering mechanism? What it does now is sort the last (about 50) offsets
> from smallest to largest and then prunes the smallest or largest, depending
> on which is further away from the average, until there are only N (I forget
> what N is) offset observations left.
>
> There may be at least two problems with this filtering mechanism. First,
> there is no apparent theory behind it; I have never seen such a crude filter
> that does not take into account any information inherent in the data. On
> the other hand, what I don't know about filters would fill all 24 volumes of
> an encyclopedia.
>
> Second, we know that each offset observation should have arrived about one
> second after the previous one, yet NTPD does not take advantage of that
> knowledge. There are filters, such as the Kalman filter that uses a
> Bayesian estimation approach to predict the next observation and adjusts it
> according to the prediction when it arrives, that do take advantage of
> previous observations. Demonstrations of the Kalman filter on the Internet
> show almost spectacular results. I used a Kalman filter in my clock
> simulation program and the results seemed pretty good. However, there are
> numerical analysis considerations to programming a Kalman filter as the sums
> and products of observations can become large in a program that runs
> infinitely long. Also, choosing the parameters of a Kalman filter is
> apparently a black art.
>
> Would it be worth it to recruit an electrical or systems engineer who
> claimed to know something about filtering data to take a serious look at
> NTPD's data filtering approach? There has to be some reason that there is a
> significant negative correlation between delay and offset in NTPD. There
> also has to be a reason that my GPS clock (BU-353, which, when it is working
> well, only has offset ±6 ms from zero) has a difference between about 0 and
> 47 ms from an NTP server on another computer that gets its time from 8 NTP
> stratum 2 servers over the Internet and has remarkably consistent offsets
> ±3.5 ms from zero. The difference between the GPS clock and the average of
> the stratum 2 servers appears to be a function of the time of day; it is
> large during the mid-part of the day, when the Internet is busy and the
> delay is large and quite variable between servers, and small late in the day
> (right now it is -0.626; 6:55 PM EST), when the delay is smaller and pretty
> uniform for all stratum 2 servers.
>
> Charles Elliott
>
>
>> -----Original Message-----
>> From: questions-bounces+elliott.ch=verizon.net at lists.ntp.org
>> [mailto:questions-bounces+elliott.ch=verizon.net at lists.ntp.org] On
>> Behalf Of Chris Albertson
>> Sent: Thursday, March 15, 2012 5:22 PM
>> To: unruh
>> Cc: questions at lists.ntp.org
>> Subject: Re: [ntp:questions] ARRGH!!! I woke up to a 50 SECOND clock
>> error.
>>
>> On Thu, Mar 15, 2012 at 2:09 PM, unruh<unruh at invalid.ca> wrote:
>>
>>
>>> Unfortunately it is not that simple. That rate changes by significan
>>> amounts. Thus the rate you get after a week may be very different
>>>
>> than
>>
>>> the rate you get after an hour. That, I submit, is the chief obstacle
>>> to having an accurate clock. And that change in rate does not fit
>>>
>> with
>>
>>> the "Allan variance" assumptions (the noise source is not of the type
>>> assumed)
>>>
>> You are right about that. I was going to add in a bit about how to
>> pick the best time to look at the clock tower. But left it out because
>> the point I was making was only that these things are not NTP
>> specific. Details after that did not contribute the the main point.
>>
>>
>> Chris Albertson
>> Redondo Beach, California
>>
--
(PS - If you email me and don't get a quick response, don't be concerned.
I get about 300 emails per day from alternate energy mailing lists and
such. I don't always see new messages very quickly. If you need a
reply and have not heard from me in 1 - 2 weeks, send your message again.)
Ron Frazier
timekeepingdude AT c3energy.com
More information about the questions
mailing list