[ntp:questions] Syncing and peering for a multi-continent deployment

Charles Elliott elliott.ch at verizon.net
Tue Apr 9 00:45:46 UTC 2013

I don't think you will ever achieve acceptable synchronization between the
data centers without accurate external clocks, such as GPS-based clocks,
attached to a time server in each location.  Internet traffic acts as waves
of high use and periods of quiescence.  During these periods of high traffic
your data centers will differ by 250 ms or more and will take some time to
resynchronize during quiescent periods.  This problem is greatly exacerbated
by the large distances between your data center locations.  NTPD cannot
prevent that.  Nor is it really NTPD's fault.  The ISPs have installed very
large buffers in front of their routers to prevent dropping packets and
sending choke packets to the data originators, as the Internet was
originally designed.  Hence the Internet is not automatically rerouting
packets when traffic patterns change, again as originally designed.  Money
is the root of all evil.

Charles Elliott 

> -----Original Message-----
> From: questions-bounces+elliott.ch=verizon.net at lists.ntp.org
> [mailto:questions-bounces+elliott.ch=verizon.net at lists.ntp.org] On
> Behalf Of Blair Zajac
> Sent: Saturday, April 6, 2013 9:01 PM
> To: questions at lists.ntp.org
> Subject: [ntp:questions] Syncing and peering for a multi-continent
> deployment
> I'm working on deploying a large number of servers to data centers on
> three continents, say US, Europe and India.  Because each data center
> will be writing data and timestamping the data, I want them all to
> think
> they are at the same time, even if it isn't exactly the correct UTC
> time.  The data will be synced to other data centers, so relative
> ordering of when something happened can be important.
> To not ping stratum-1 and -2 level servers from a large number of
> boxes,
> I'm setting up three stratum-3 NTP servers in each data center.
> Here's what I've read:
> 1) Each ntp server should have 4 to 7 upstream clocks [1], [2], [3].
> 2) The three stratum-3 NTP servers in each data center should be peers
> of each other [4].  So each NTP server would have at most 2+7=9
> connections.
> Questions:
> 1) Should each data center use nearby stratum-2 clocks or pick a set of
> stratum-2 clocks that are network wise in the center between all data
> centers (it may not be possible to get a true center)?
> 2) Should each stratum-3 server in a single data center use all 4-7
> upstream clocks?  Or should the 4-7 be split between the three stratum-
> 3
> servers?  The diagram at [4] suggests they should be split?  If that's
> the case, then how are falsetickers identified?
> 3) Should I have all the stratum-3 servers in each data center be peers
> of each other, so each would have 8 peers?  This would ensure that all
> my clocks think they are at the same time.  If I do this, then I would
> need to increase maxclocks to support up to 7 (upstream) + 8 (peers)=15
> clocks?
> 4) Or, should I not peer data centers to each other and trust that the
> stratum-2 clocks near each data center will be close to the other
> clocks?
> Other random questions:
> 1) It doesn't appear that its necessary to set up symmetric keys for
> peers?
> 2) Is there a way to tell which peer is the "master" peer?  Looking
> through ntpdc and ntpq I didn't see anything.
> Thanks,
> Blair
> [1]
> http://support.ntp.org/bin/view/Support/SelectingOffsiteNTPServers#Sect
> i
> [2]
> http://support.ntp.org/bin/view/Support/SelectingOffsiteNTPServers#Sect
> i
> [3]
> http://support.ntp.org/bin/view/Support/StartingNTP4#Section_7.
> [4] http://www.ntp.org/ntpfaq/NTP-s-config-adv.htm#AEN3101
> _______________________________________________
> questions mailing list
> questions at lists.ntp.org
> http://lists.ntp.org/listinfo/questions

More information about the questions mailing list