[ntp:hackers] Setting up a high-end NTP server. :-)

Poul-Henning Kamp phk at phk.freebsd.dk
Tue Jan 11 12:52:43 PST 2005


In message <41E438C9.5010503 at hda.hydro.com>, Terje Mathisen writes:
>I'm going to rework our corporate ntp structure this year, partly by 
>replacing a set of old PCs located under my desk, with a number of 
>regular 1U rack servers, located in various locations around the world.
>
>I have 4 GPS-based refclocks today, plus a Meinberg DFC-77 receiver that 
>will be installed in our German headquarter
>
>For my first attempt I borrowed a spare HP DL360, just to verify that 
>the hardware was compatible with my preferred OS (FreeBSD).

If you _really_ want a great NTP server you should go the other way.

>FreeBSD 5.3 had no problems at all with this setup, it came up with all 
>hw enabled.
>
>Connecting all six GigE cards to independent network switches would 
>allow it to handle up to 6 million ntp requests/second, assuming the 
>cpus could keep up.

No it wouldn't, you would get timestamping errors due to packet queing
long before you get there.

>It seems like the usno servers handle an average of up to 5000 
>packets/second, so there's still a lot of headroom, as long as the IP 
>links can keep up.

I have NET4501 handling several hundred packets a second and not
even touching the CPU.  This is not the metric you should worry
about anyway.  Properly configured you need 6400 clients before
you get to 100 packets/sec anyway.

The problems with using a big iron box for NTP server are legion.

For one thing you can't get rid of weird BIOS modes like "ACPI"
and SMM mode etc, so you will have significant holes in your cpu
time, that ruins your timestamps and causes instability in your
PLLs.

Don't even get me started about fans, variable speed in particular.
The more aggressive they are, the worse the temperature swings over
your Xtal.  Not to mention that they wear out and are noisy.

The DL360 is a SMP machine if I remember right, that sucks for
timekeeping because of all the interrupt routing intricacies and
the locking performed between the CPUs.  Not to mention that
you have 20 times more cpu power _per chip_ than you really need.

Instead, use a soekris NET4501.

Apart from the fact that it is cheap, has no moving parts at all,
uses passive cooling (== great temperature stability for the X-tal)
and can be run for hours on a small lead-acid battery, it has a PPS
timestamping input to kill for:

	http://phk.freebsd.dk/soekris/pps/

I know it sounds silly and a lot of people are simply not able
to wrap their heads around this, but for an NTP server, small
and simple beats any big iron.

-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.



More information about the hackers mailing list