[ntp:questions] How is the NTP build tested?

Brian Inglis Brian.Inglis at SystematicSw.ab.ca
Tue Jan 14 01:51:54 UTC 2014


On 2014-01-13 15:13, William Unruh wrote:
> On 2014-01-13, Harlan Stenn <stenn at ntp.org> wrote:
>> romain.pontida at gmail.com writes:
>>> Hello,
>>>
>>> I'm sorry if this has already been answered before, but how do I prove
>>> that the release of NTP included with your Linux distribution (using
>>> ntpd) is compliant to the NTP specification as defined in the RFC?
>>
>> Interesting question.
>>
>> The short answer is that the ntp.org release has been the "reference
>> implementation" for NTP since the beginning, but the IETF doesn't do
>> reference implementations.
>>
>>> I have found a statement in v4.1 release notes (
>>> http://doc.ntp.org/4.1.0/biblio.htm ) that says "this version is fully
>>> compliant with the previous NTP Ve rsion 3 specification and
>>> implementation defined in [ref to RFC]". I'm wondering whether there's
>>> any automated test that allows the development team to ensure this
>>> statement remains true with every new release.
>>
>> There is no current automated test framework to do this.  I'd love to
>> have such a beast, either contributed to the project or find folks who
>> want this enough to fund the effort.
>
> Of course then you would have to prove that that test framework was
> actually testing compliance with the standard, which seems harder than
> to see if the reference implimentation actually impliments the standard.

>>> I guess any implementation could be tested for synchronisation against
>>> the reference implementation, but how do you prove that the reference
>>> implementation is correct?
>>
>> What happens (and some view this as a bug while others view it as a
>> feature) is that the development version of the reference implementation
>> gets to a point where folks say "Time for a new standard", and then
>> folks write the Standard based on the implementation in the code.  Then
>> more folks very carefully study both the code and the proposed standard
>> and reconcile any differences.  At some point the Standard goes as far
>> as it can (the V3 was only a *draft* standard) and we keep working on
>> the codebase.  Eventually the codebase *will* do some things differently
>> than the standard, and we do expect that these differences will become
>> part of the next version of the NTP Standard.
>
> Almost everything is a draft. How many of the RFcs ( which after all
> means "Request for Comment" not "This is the standard") actually
> graduate into an actual official (what official body?) standard, rather
> than becoming the defacto standard.
>
> And then you would have to destandardize the old standard to ever do
> anything new.
>
> Finally, why do you care if the reference implimentation actually
> completely and reliably impliments the RFC? And if you found a
> difference, would the bug be in the reference implimentation or in the
> RFC?
>
> Note also, that IMHO, the RFC is too concerned with implimentation,
> rather than setting out the requirements of the standard. Certainly the
> format of the packets being interchanged must be completely specified,
> but what the machine then does with the contents of those packets should
> be much more loosely specified. For example, chrony uses those contents
> much differently than does ntpd. One could argue that chrony thus does
> not comply with the RFC. But the experimental evidence is that in fact
> chrony disciplines the system clock much more accurately than does ntpd.
> Certainly chrony must read and send the same packets, but is it a bad
> thing that chrony then uses those packets very differently? I would say
> not.

Besides the U Del ntp-dev release, the NTP.org stable release, and chrony,
there is also the OpenBSD derived OpenNTPd implementation to choose from.

-- 
Take care. Thanks, Brian Inglis


More information about the questions mailing list