[ntp:questions] NTP in a Linux cluster

Richard B. Gilbert rgilbert88 at comcast.net
Tue Sep 8 13:17:40 UTC 2009

Lorcan wrote:
> Many thanks to everyone who replied!  I'm slowly learning
> my way around this stuff, and your help is much appreciated.
> In my original posting, I should probably have provided a litlle more
> information about the environment in which our systems are typically
> deployed.  Our customers are telcos,  which has some interesting
> implications.
> For one thing, they take the physical security of their systems very
> seriously.  The cluster nodes will quite possibly be buried in a
> bunker somewhere, with little hope of picking up a GPS signal.
> Also, I'm hoping to avoid extra hardware - not really on cost grounds,
> but because I would hope we could retro-fit the new solution to
> existing sites;  tweaking config is one thing, installing new
> hardware,
> cabling etc. is quite another.
> On the plus side, our systems typically are hooked into a large
> corporate LAN that will already have an in-house NTP server
> (maybe more than one) that we can  slave off.  We already do
> that in fact, but with less-than-optimal config, I think...
> So, I'm coming to the idea that what I'm really missing may be
> appropriate use of the "peer" option in /etc/ntp.conf.
> Could anyone explain exactly how peers (as distinct from servers)
> are used?  If two clients sync off the same server, and manage to
> get to within (say) 10mS of the server, I would expect that they
> could be as much as 20mS apart (if one is at +10ms, and one
> at -10ms).  Would making them peers mean that they should
> remain within 10ms of each other?
> I'm leaning towards having a common ntp.conf file on all nodes.
> On a four-node cluster, with a single ntp server, it would look like
> this:
> ==== start
> driftfile /var/lib/ntp/drift
> server the_ntp_server  minpoll 4 maxpoll 6
> peer node1
> peer node2
> peer node3
> peer node4
> ==== end
> Is that a reasonable starting point?
> Can the "peer" entries have minpoll and maxpoll?  If so, is there
> any reason not to set those to low values (4? 6?)?
I don't know if servers or peers can be given their own minpoll and 
maxpoll.  There is very little reason to meddle with the default MINPOLL 
and MAXPOLL!  NTPD adjusts the value in use as needed.

A somewhat oversimplified explanation is that a short poll interval is 
used to correct large errors quickly and the long poll interval allows 
NTPD to measure and correct small errors very accurately.

> (The nodes have multiple NICs; the inter-node traffic is carried
> on a separate "internal" LAN, which has low latency and should
> have bandwidth to spare.)
> Also, I'll use the "-x" command-line flag to enforce "slewing only"
> behaviour.   (If the clock on a single node is way out of sync
> for some reason we can shut it down and "jump" the clock when
> restarting it.)
> If there are multiple NTP servers available on the LAN, is it possible
> that our particular requirements (inter-node synchronisation being
> more important than accurate timekeeping) means that we're better
> off just using one of them?  Or should we use several of them,
> and use the "prefer" option to encourage all nodes to attach more
> significance to the same one?

I would try to avoid using servers on the Internet.  You can get much 
better stability and accuracy using a hardware reference clock.  This 
can be a GPS receiver, a LORAN receiver, an "atomic clock" or, perhaps, 
one or two other devices.  Symmetricom and Meinberg Funkuhren both offer 
equipment that you may find useful.  It's expensive but if you need 
stable and accurate time it well be worth the expense.

If you are able to site an antenna where it will have a view of most or 
all of the sky, a GPS timing receiver is a very good choice.  The 
antenna is a little bigger than a quarter and smaller than a half 
dollar.  Mine sits on top of a "Leaf Guard" rain gutter on the Southern 
side of the house.  It works like a champ!


More information about the questions mailing list