[ntp:questions] How is the NTP build tested?

Brian Inglis Brian.Inglis at SystematicSw.ab.ca
Tue Jan 14 01:51:54 UTC 2014

On 2014-01-13 15:13, William Unruh wrote:
> On 2014-01-13, Harlan Stenn <stenn at ntp.org> wrote:
>> romain.pontida at gmail.com writes:
>>> Hello,
>>> I'm sorry if this has already been answered before, but how do I prove
>>> that the release of NTP included with your Linux distribution (using
>>> ntpd) is compliant to the NTP specification as defined in the RFC?
>> Interesting question.
>> The short answer is that the ntp.org release has been the "reference
>> implementation" for NTP since the beginning, but the IETF doesn't do
>> reference implementations.
>>> I have found a statement in v4.1 release notes (
>>> http://doc.ntp.org/4.1.0/biblio.htm ) that says "this version is fully
>>> compliant with the previous NTP Ve rsion 3 specification and
>>> implementation defined in [ref to RFC]". I'm wondering whether there's
>>> any automated test that allows the development team to ensure this
>>> statement remains true with every new release.
>> There is no current automated test framework to do this.  I'd love to
>> have such a beast, either contributed to the project or find folks who
>> want this enough to fund the effort.
> Of course then you would have to prove that that test framework was
> actually testing compliance with the standard, which seems harder than
> to see if the reference implimentation actually impliments the standard.

>>> I guess any implementation could be tested for synchronisation against
>>> the reference implementation, but how do you prove that the reference
>>> implementation is correct?
>> What happens (and some view this as a bug while others view it as a
>> feature) is that the development version of the reference implementation
>> gets to a point where folks say "Time for a new standard", and then
>> folks write the Standard based on the implementation in the code.  Then
>> more folks very carefully study both the code and the proposed standard
>> and reconcile any differences.  At some point the Standard goes as far
>> as it can (the V3 was only a *draft* standard) and we keep working on
>> the codebase.  Eventually the codebase *will* do some things differently
>> than the standard, and we do expect that these differences will become
>> part of the next version of the NTP Standard.
> Almost everything is a draft. How many of the RFcs ( which after all
> means "Request for Comment" not "This is the standard") actually
> graduate into an actual official (what official body?) standard, rather
> than becoming the defacto standard.
> And then you would have to destandardize the old standard to ever do
> anything new.
> Finally, why do you care if the reference implimentation actually
> completely and reliably impliments the RFC? And if you found a
> difference, would the bug be in the reference implimentation or in the
> RFC?
> Note also, that IMHO, the RFC is too concerned with implimentation,
> rather than setting out the requirements of the standard. Certainly the
> format of the packets being interchanged must be completely specified,
> but what the machine then does with the contents of those packets should
> be much more loosely specified. For example, chrony uses those contents
> much differently than does ntpd. One could argue that chrony thus does
> not comply with the RFC. But the experimental evidence is that in fact
> chrony disciplines the system clock much more accurately than does ntpd.
> Certainly chrony must read and send the same packets, but is it a bad
> thing that chrony then uses those packets very differently? I would say
> not.

Besides the U Del ntp-dev release, the NTP.org stable release, and chrony,
there is also the OpenBSD derived OpenNTPd implementation to choose from.

Take care. Thanks, Brian Inglis

More information about the questions mailing list