[ntp:questions] How is the NTP build tested?

William Unruh unruh at invalid.ca
Mon Jan 13 22:13:27 UTC 2014


On 2014-01-13, Harlan Stenn <stenn at ntp.org> wrote:
> romain.pontida at gmail.com writes:
>> Hello,
>> 
>> I'm sorry if this has already been answered before, but how do I prove
>> that the release of NTP included with your Linux distribution (using
>> ntpd) is compliant to the NTP specification as defined in the RFC?
>
> Interesting question.
>
> The short answer is that the ntp.org release has been the "reference
> implementation" for NTP since the beginning, but the IETF doesn't do
> reference implementations.
>
>> I have found a statement in v4.1 release notes (
>> http://doc.ntp.org/4.1.0/biblio.htm ) that says "this version is fully
>> compliant with the previous NTP Ve rsion 3 specification and
>> implementation defined in [ref to RFC]". I'm wondering whether there's
>> any automated test that allows the development team to ensure this
>> statement remains true with every new release.
>
> There is no current automated test framework to do this.  I'd love to
> have such a beast, either contributed to the project or find folks who
> want this enough to fund the effort.

Of course then you would have to prove that that test framework was
actually testing compliance with the standard, which seems harder than
to see if the reference implimentation actually impliments the standard. 


>
>> I guess any implementation could be tested for synchronisation against
>> the reference implementation, but how do you prove that the reference
>> implementation is correct?
>
> What happens (and some view this as a bug while others view it as a
> feature) is that the development version of the reference implementation
> gets to a point where folks say "Time for a new standard", and then
> folks write the Standard based on the implementation in the code.  Then
> more folks very carefully study both the code and the proposed standard
> and reconcile any differences.  At some point the Standard goes as far
> as it can (the V3 was only a *draft* standard) and we keep working on
> the codebase.  Eventually the codebase *will* do some things differently
> than the standard, and we do expect that these differences will become
> part of the next version of the NTP Standard.

Almost everything is a draft. How many of the RFcs ( which after all
means "Request for Comment" not "This is the standard") actually
graduate into an actual official (what official body?) standard, rather
than becoming the defacto standard. 

And then you would have to destandardize the old standard to ever do
anything new. 

Finally, why do you care if the reference implimentation actually
completely and reliably impliments the RFC? And if you found a
difference, would the bug be in the reference implimentation or in the
RFC?

Note also, that IMHO, the RFC is too concerned with implimentation,
rather than setting out the requirements of the standard. Certainly the
format of the packets being interchanged must be completely specified,
but what the machine then does with the contents of those packets should
be much more loosely specified. For example, chrony uses those contents
much differently than does ntpd. One could argue that chrony thus does
not comply with the RFC. But the experimental evidence is that in fact
chrony disciplines the system clock much more accurately than does ntpd. 
Certainly chrony must read and send the same packets, but is it a bad
thing that chrony then uses those packets very differently? I would say
not. 
>
> H



More information about the questions mailing list