[ntp:questions] NTP in a Linux cluster
Richard B. Gilbert
rgilbert88 at comcast.net
Tue Sep 8 13:17:40 UTC 2009
> Many thanks to everyone who replied! I'm slowly learning
> my way around this stuff, and your help is much appreciated.
> In my original posting, I should probably have provided a litlle more
> information about the environment in which our systems are typically
> deployed. Our customers are telcos, which has some interesting
> For one thing, they take the physical security of their systems very
> seriously. The cluster nodes will quite possibly be buried in a
> bunker somewhere, with little hope of picking up a GPS signal.
> Also, I'm hoping to avoid extra hardware - not really on cost grounds,
> but because I would hope we could retro-fit the new solution to
> existing sites; tweaking config is one thing, installing new
> cabling etc. is quite another.
> On the plus side, our systems typically are hooked into a large
> corporate LAN that will already have an in-house NTP server
> (maybe more than one) that we can slave off. We already do
> that in fact, but with less-than-optimal config, I think...
> So, I'm coming to the idea that what I'm really missing may be
> appropriate use of the "peer" option in /etc/ntp.conf.
> Could anyone explain exactly how peers (as distinct from servers)
> are used? If two clients sync off the same server, and manage to
> get to within (say) 10mS of the server, I would expect that they
> could be as much as 20mS apart (if one is at +10ms, and one
> at -10ms). Would making them peers mean that they should
> remain within 10ms of each other?
> I'm leaning towards having a common ntp.conf file on all nodes.
> On a four-node cluster, with a single ntp server, it would look like
> ==== start
> driftfile /var/lib/ntp/drift
> server the_ntp_server minpoll 4 maxpoll 6
> peer node1
> peer node2
> peer node3
> peer node4
> ==== end
> Is that a reasonable starting point?
> Can the "peer" entries have minpoll and maxpoll? If so, is there
> any reason not to set those to low values (4? 6?)?
I don't know if servers or peers can be given their own minpoll and
maxpoll. There is very little reason to meddle with the default MINPOLL
and MAXPOLL! NTPD adjusts the value in use as needed.
A somewhat oversimplified explanation is that a short poll interval is
used to correct large errors quickly and the long poll interval allows
NTPD to measure and correct small errors very accurately.
> (The nodes have multiple NICs; the inter-node traffic is carried
> on a separate "internal" LAN, which has low latency and should
> have bandwidth to spare.)
> Also, I'll use the "-x" command-line flag to enforce "slewing only"
> behaviour. (If the clock on a single node is way out of sync
> for some reason we can shut it down and "jump" the clock when
> restarting it.)
> If there are multiple NTP servers available on the LAN, is it possible
> that our particular requirements (inter-node synchronisation being
> more important than accurate timekeeping) means that we're better
> off just using one of them? Or should we use several of them,
> and use the "prefer" option to encourage all nodes to attach more
> significance to the same one?
I would try to avoid using servers on the Internet. You can get much
better stability and accuracy using a hardware reference clock. This
can be a GPS receiver, a LORAN receiver, an "atomic clock" or, perhaps,
one or two other devices. Symmetricom and Meinberg Funkuhren both offer
equipment that you may find useful. It's expensive but if you need
stable and accurate time it well be worth the expense.
If you are able to site an antenna where it will have a view of most or
all of the sky, a GPS timing receiver is a very good choice. The
antenna is a little bigger than a quarter and smaller than a half
dollar. Mine sits on top of a "Leaf Guard" rain gutter on the Southern
side of the house. It works like a champ!
More information about the questions