[ntp:questions] chrony and ntp comparison-- ADSL hookup

Unruh unruh-spam at physics.ubc.ca
Fri Feb 22 21:54:44 UTC 2008


"David L. Mills" <mills at udel.edu> writes:

>Bill,

>You hint that it is now the responibility of the NTP crew to rebut your 
>claims. Please see the great detail in Chapter 6 of my book, which 

No. I state that IF someone claims that my results are wrong, it is their
duty to state why, and present the basis for that claim. Simply telling me
that I have not done enough measurements without any justifiation as to why
more measurements might change the results is unacceptable. 

>reports on the resulrs of hundreds of experiments with onboard 
>oscillators, PPS signals, LANs and various Internet paths all over the 
>globe. Included are wedge scattergrams, CDF funtions and time and 
>frequency plots, some covering thousands of hours.

As I said, ntp IS a well tested stable program. I do not doubt that at all. 
The only question is whether or not it is the best possible. I have
concentrated on testing the algorithm by contrasting it with a very
different algorithm for clock discipline. 

Chrony had definite failures. It has not refclock routines, which is a
serious lack. It runs on only Linux which is a serious limitation. HOwever,
from the algorithmic point of view it does seem to have definite
advantages-- speed of adaptation to changes, and accuracy of clock
discipline seem to be two of them. 



>I've said many times that you are not comparing chrony and NTP on a 
>level field. If you crank the NTP time constant comparable to yours, I 
>strongly expect the two will perform in very similar fashion. In your 
>particular LAN the Allan intercept, or when the phase noise about equals 
>the flicker noise, could be somewhat lower than assumed as default by 

That was why I tested two very different systems. In one the Allan
intercept is definitely smaller than the default. But it is also one where
the flicker noise does not follow the model assumed by you and Allan-- and
it is what a real system produces. In the other the Allan intercept is
almost certainly much larger than the default. Chrony makes no assumptions
about the Allan intercept or the behaviour of the flicker noise. ntp does.


>NTP, but the default depends on a number of considerations other than to 
>minimize the risetime.

Sure, no question, one of the key being stability of the feedback loop.
Chrony on the other hand has no feedback loop. (well at second order it
does, but then ntp's is also more complex than a simple feedback loop with
its clock filter, and huff and puff filters, etc).



>The only scientific question of interest here is whether a 
>linear-regression loop is better or worse than a phase-lock loop. Let's 
>not waste time on issues other than that. You can test that by changing 
>probably only a couple lines in your code. Be sure the risetime and 
>overshoot characteristics are similar.

There is not loop. There is no "risetime" or overshoot. To put is most
simply, chrony measures the slope and the offset by doing a linear
regression on the raw time (No change in offset or slope by the clock
discipline), and then corrects both the slope and the offset of the clock. 
It does not do this with  a feedback loop. Thus risetime and overshoot are
not relevant parameters. It is a very different clock discipline routine
than ntp's.

"Feedback" does enter but in a very different way. The number of points
used to do the linear regression is adaptive-- fewer are used if the
"linear plus noise" hypothesis is bad. That is a feedback of sorts, but
very hard to analyse. In the ADSL case, the linear regression is always the
max number of points (64) because the noise is (almost) all random
fluctuations of the delay. In the case of low transmission noise, the
regression tends to be shorter ( eg 20 points but fluctuating) because the
drift fluctutations dominate. Ie the system automatically adapts to the
actual "Allan intercept" rather than some assumed one. 


I agree that concentrating on the algorithms is what I want to do.
 I also agree that my measurements are preliminary and have been done on a
very finite number of systems. But also that at this point simply doing
"more of the same" is not very useful. If there is some reason to believe
that there are situations in which ntp will outperform chrony, and which I,
or someone else, can carry out comparisons for such systems, that would
certainly be worthwhile. 

I purchased a second GPS precisely to be able to make real comparisons
between the two. And I am surprized at how well chrony had done. (And I
have learned a lot both about how ntp and chrony work).

My biggest surprize with ntp is the clock-filter, in which 85% of the data
is discarded. While I understand why, it really really feels uncomfortable
discarding so much data. Even if the data were completely dominated by
gaussian transmission noise (ie the case in which extra data can be used to
beat down the variance) ntp chucks 85% of the data. Looking at the data,
and doing the Pie plots, the justification in my ADSL case for this is
extremely tenuous. Maybe there exist cases where it were justified( your
road to Mandalay examples) but surely it would be better to determine
whether you are in such a situation rather than assuming youare in the
worst of all possible worlds. (If 85% of the data really  is junk, I do not
believe the remaining 15 % either. In my case, maybe 5% is junk-- ie highly
biased delays. Chrony is set up to get rid of some of that-- if the deleay
is greater than 50% greater than the minimum for example). but adapts the
linear regression to take care of the rest. 







>Dave

>Unruh wrote:

>> "Richard B. Gilbert" <rgilbert88 at comcast.net> writes:
>> 
>> 
>>>Unruh wrote:
>>>
>>>>"David J Taylor" <david-taylor at blueyonder.neither-this-bit.nor-this-bit.co.uk> writes:
>>>>
>>>>
>>>>
>>>>>Unruh wrote:
>>>>>[]
>>>>>
>>>>>
>>>>>>>- the default for min and max poll are 6 and 10 (IIRC), and not 4
>>>>>>>and 7
>>>>>>
>>>>>>
>>>>>>And so? Exactly how is this supposed to make a difference?
>>>>>
>>>>>It means that your tests are not carried out on an "out of the box" 
>>>>>configuration, and therefore perhaps less useful than they might otherwise 
>>>>>be.  It's possible that some servers would see min/maxpoll of 4 and 7 as 
>>>>>abusive.
>>>>
>>>>
>>>>And no "out of box" test is possible. Neither ntp ( althoug it is much
>>>>closer) or chrony are set up to do so. 
>>>>Right now I do not care what servers find abusive. I am using my own
>>>>server, so I decide what is abusive. I am not advocating these parameters,
>>>>I am using them to test ntp and chrony. ALL of the tests indicate that
>>>>chrony disciplines the clock about a factor of 2-3 better than does ntp.
>> 
>> 
>>>Testing ntpd and chrony using a local server or servers is not a very 
>>>realistic test of "real world" performance!
>> 
>> 
>>>Try configuring both with the same set of four to ten internet servers. 
>>> Set up a stratum 1 server using GPS or a cesium clock as a reference 
>>>to serve as a standard to measure with.  Collect statistics for a month 
>>>or so.
>> 
>> 
>>>Then tell us what you learned.
>> 
>> 
>> You are starting to sound like the cigarette people when it was pointed out
>> that smoking might cause cancer.
>> 
>> I was trying to test the algorithms NOT the programs as a whole. 
>> 
>> I am not advocating that the world rush out and abandon NTP and alkl run
>> chonry. Too many problems. Chrony does not do refclocks. chrony does not
>> handle leap seconds. chrony runs only on Linux ( well a bit on a few others
>> but not many). However where chrony does run it "beats the pants off" ntp.
>> This suggests that the algorithm in chrony is a better clock discipline
>> algorithm than ntp's. Has this been tested extensively? No. Is it a
>> foolproof conclusion? No. But ALL the evidence at present is that chrony's
>> algorithm beats ntp's. If you believe that there are conditions under which
>> ntp's would beat chrony's it now becomes your duty to suggest why and
>> where. "You have not done enough tests" is the response of someone with a
>> completely closed mind.
>> 
>> I really do not care what you believe about ntp since you have zero data to
>> back up your position. I am not claiming that ntp fails. It is a robust,
>> well tested algorithm. It works and is certainly "good enough" for almost
>> everyone who uses it.  I am asking if it is the best we can do. My data
>> suggests it is not. 
>> 
>> Note the my latest test was NOT using a local server. It was a remoTE
>> stratum 1 server, over an ADSL hookup with a 16ms delay. 
>> It had 100 times the mean delay and 10 times or more the variance in that
>> delay of the "local" test. 
>> And exactly what would you expect to find if I ran it for a month rather
>> than a day? Would you really expect something about the chrony algorithm to
>> show up in  a month?  What? I suspect you have no idea whatsoever as to
>> what the chrony algorithm is so you could not if you wanted to.
>> 
>> 
>> And I used a GPS PPS as the reference source on the ADSL client to measure
>> the clock offset with.
>> 




More information about the questions mailing list