[ntp:questions] ARRGH!!! I woke up to a 50 SECOND clock error.
Ron Frazier (NTP)
timekeepingntplist at c3energy.com
Wed Mar 14 22:07:07 UTC 2012
On 3/14/2012 5:04 PM, Ron Frazier (NTP) wrote:
> On 3/14/2012 4:00 PM, David J Taylor wrote:
>>> Hi David T,
>>> NOW .... you understand.
>>> I have 4 PC's connected to the LAN, plus my wife's work computer 3
>>> days / week, and on rare occasions my son's computer. All are
>>> connected by wifi. All do pretty mundane things: web browser,
>>> email, sometimes downloading patches, sometimes doing online
>>> backup. My 4 are running NTPD. 3 run windows most of the time and
>>> dual boot into linux. My 4th machine runs linux all the time. When
>>> my wife is here, she does a remote desktop type of thing into her
>>> work system. Two of my PC's are running Vista, one is running
>>> Windows 7. Those three dual boot into Ubuntu 11.04 and the always
>>> linux machine runs Ubuntu 11.04. Almost all the discussions I've
>>> had on this mailing list are for my Windows 7 machine.
>>> The path out of my house is:
>>> PC Wifi --> Wifi router --> wired router --> cable modem --> ISP --
>>>> - is this PC connected over wireless or wired?
>>> Wifi G
>>>> - what else is going through the router? Someone else downloading
>>>> large files or using streaming audio or video?
>>> See above. Normally, no huge data hogs.
>>>> - who else might be sharing your connection?
>>> It's cable. Who knows? I also get cable TV and telephone through
>>> the same wire.
>>>> - what type of service do you have? Presumably not dial-up! But
>>>> what speed?
>>> Comcast Cable
>>> Just tested it with speedtest.net
>>> Ping to near city: 91 ms, Download: 29.63 Mbps, Upload: 5.3 Mbps
>>>> - having checked you speed, would you describe your connection as
>>> See above for speed. Generally, it's very stable. However, for the
>>> purposes we're discussing, I think it's latencies and delays that
>>> are the problem.
>>> As I mentioned in my reply to David L, I'm not concerned over trying
>>> to get stellar performance from internet servers. I just want to
>>> get a good GPS server system running and use the internet servers as
>>> a backup.
>> Thanks for that clarification. I think you should be getting /very
>> much/ better performance from your Internet servers. That ping is
>> poor as well. Here I have 30 Mb/s down, just 1 Mb/s up, and NTP
>> delays show as 18-34 ms (most in 22-30 ms).
>> To that end, if your cable modem has multiple ports, connect one of
>> the PCs direct to the CM and run it as your local NTP server. Wi-Fi
>> doesn't help NTP. Later, you can add GPS/PPS to that PC as well. At
>> the very least, connect your timekeeping PC direct to the wired
>> router. No Wi-Fi! For my main NTP server, I got a low-powered and
>> fan-less Intel Atom system, and it runs FreeBSD. It /only/ runs NTP,
>> no interactive stuff at all.
>> You might also consider getting rid of the two routers and just using
>> the wireless one. You might also see whether you can run NTP on your
>> router - perhaps it's a model which can run the DD-WRT firmware. I
>> think some variants of DD-WRT can run NTP, but please check. I have
>> a WRT 54GL in my system where I run the DD-WRT firmware.
>> Online backup could well affect the performance of the connection for
>> timekeeping, and your imminent Sure GPS/PPS will help enormously.
>> Just some thoughts which may help you along the way.
> Hi David,
> I really appreciate all these suggestions you shared, as well as past
> ones. If I decide to revamp my network, I'll probably put some of
> them into use. However, that's not really practical right now. All
> the networking gear is in the basement and all the PC's are upstairs.
> I've already spent 2 months working on this GPS stuff and ignoring
> some other things I need to be doing. I think I'm just going to focus
> on getting the Sure board up and running on a time server when it
> comes and, perhaps, use my BU-353 as a backup time source, and use the
> internet servers as a third level backup source. Most of the time, I
> won't care what they're doing Hopefully, I can spend less time
> thinking about time and just know that my PC's clocks are right.
> Regardless, I'm going to continue to have an interest in the topic and
> have enjoyed all the discussions and learning that have occurred.
> It's just that I have to do at least a few other things besides this.
PS to my prior message.
I don't think the problem so much is the delay to the internet servers,
or even to get out of my house. NTPD is supposed to take care of that
as long as it's pretty much symmetrical. I think the problem is that
the Windows clock is like a wild tiger that doesn't want to be tamed and
which is running every which way. For whatever reason, cpu load, heat,
cosmic vibrations, whatever, the intrinsic frequency of the windows
clock is always changing. In order to avoid beating up on the internet
servers too much, I have to poll them at least every 4 minutes apart.
If you let it, NTPD will extend that out to 16 minutes or more. So,
when the clock source is polled, say the PC clock is too fast, so NTPD
slows it down. Then, when you poll the clock source again, say the PC
clock is too slow, so NTPD speeds it up. Because of the varying
intrinsic frequency of the clock, you can never find a clock speed that
just works, because then the system goes and changes, by changes in the
oscillator, how much time passes at those particular settings. It's a
battle you cannot win. By polling my GPS every 8 seconds, I can keep
the clock under control based on it's current needs which are varying
second by second. Of course, when discussing internet servers, 30 ms of
jitter doesn't help any.
I actually have an analogy from the world of old analog mechanical
clocks. I actually took a week long clock repair class just for fun
once. I thought I'd pursue it as a business, but never did. Imagine
you have an antique clock which is spring driven by a coil spring that
you wind up and regulated by a pendulum. You want to calibrate the
clock to keep the correct time. How do you do it? A logical answer
a) wind the clock
b) start the pendulum and let it stabilize
c) count the number of pendulum swings per minute with some instrument
d) look at the gear train and determine exactly how many seconds that
many pendulum swings would be
e) determine how many pendulum swings are needed in a minute
f) adjust the pendulum so it swings the proper number of times in a minute
So, lets say this particular clock needs 100 swings of the pendulum per
minute to be on time. Say, it's measuring at 87, so it's running slow.
You tweak the pendulum until it's swinging exactly 100 times per minute
and, boom, the clock should be keeping perfect time.
There's one little problem. A coil spring is a non linear power
source. The torque it puts on the shaft of the clock will vary as it
unwinds. Therefore, making up some numbers, maybe it starts out at full
torque and keeps good time. Then, a week later, maybe the torque is 97
percent, so it's going to be running 3 percent slow. Maybe after 3
weeks, maybe the torque is down to 85%, so it might be running 15%
slow. When it's 30 day cycle is up, the time shown will be
substantially behind what it should be. The only way to fix it is to
set it to run fast during the first part of its cycle then slower at the
end so it averages out.
The point is, that it's a continually moving target. The windows clock
is the same way. It never runs at the same frequency from minute to
minute. Even if you get it running right one minute, it's wrong the next.
Let's take, for example, the TAZ computer I mentioned earlier. Forget
GPS for the moment. With the default settings, NTPD will eventually be
polling the internet server every 16 minutes. The problem is not
exclusively that there is jitter in the time retrieved from the time
server. Let's also forget that for the moment. Say we poll the
internet server and it says the time is exactly 12:00:00. Rounding to
the seconds level just for simplicity, say my clock says 12:00:02, so
I'm 2 seconds fast. So, we slow down the clock by tweaking its
parameters. Then, we wait 16 more minutes. Now the time server says
12:16:00. Say my clock says 12:15:57. Now, I'm 3 seconds slow, so we
speed the clock back up. Now, theoretically, just like my pendulum
clock, I should be able to get the parameters dialed in so the clock
keeps time. However, behind the scenes, the intrinsic operational speed
changed. While I'm sure I'm butchering the internal technicalities,
let's say the clock has a speed knob, and if we set the speed knob to a
value of 100, the the clock will count exactly 1 second while exactly 1
second passes. If this stayed true, NTPD would eventually set the speed
knob at 100 and everybody would be happy. But the intrinsic speed of
the oscillator changed, so that now setting that knob at 100 now makes
the clock think 1.03 seconds has passed when, in actuality, 1 second has
passed. So the clock is running fast. So, NTPD dials the speed knob
back to 97. But now, the oscillator has changed again so that a setting
of 97 now makes the clock think that .95 seconds have passed, when, in
actuality, 1 second has passed. This is why I'm getting such wild
oscillation in the graph. No matter what NTPD does to tweak the clock
speed, and no matter how accurate that is at the time, that adjustment
never has the same effect the next time the time server is polled. Just
as setting the screw on the pendulum of my mechanical clock doesn't have
the same effect tomorrow as it does today. There will never be a
setting that just works.
I don't think it's a game you can win on a standard windows computer.
So, I'm not even inclined to play the internet NTP server game. So, by
polling my GPS every 8 seconds, the computer has much less time to
drift, no matter what the oscillator is doing, and it needs much less
correction. Then, NTPD can adapt dynamically and almost instantly to
put in whatever adjustments are needed at that moment in time. The
overall net effect, assuming the GPS is accurate, is that the computer's
clock is never more than 10 ms away from GPS time, rather than being 50
ms away from an internet time server's time. With PPS, I could get it
much better still.
(PS - If you email me and don't get a quick response, don't be concerned.
I get about 300 emails per day from alternate energy mailing lists and
such. I don't always see new messages very quickly. If you need a
reply and have not heard from me in 1 - 2 weeks, send your message again.)
timekeepingdude AT c3energy.com
More information about the questions