[ntp:questions] Asymmetric Delay and NTP

Joe Gwinn joegwinn at comcast.net
Wed Mar 19 10:55:21 UTC 2014

In article <5328AAA6.70200 at rubidium.dyndns.org>, Magnus Danielson
<magnus at rubidium.dyndns.org> wrote:

> On 18/03/14 01:36, Joe Gwinn wrote:
> > In article <5327757E.5040504 at rubidium.dyndns.org>, Magnus Danielson
> > <magnus at rubidium.dyndns.org> wrote:
> >
> >> Joe,
> >>
> >> On 16/03/14 23:16, Joe Gwinn wrote:
> >>> I recall seeing something from Dr. Mills saying that a formal proof had
> >>> been found showing that no packet-exchange protocol (like NTP) could
> >>> tell delay asymmetry from clock offset.  Can anyone provide a reference
> >>> to this proof?
> >>
> >> It's relative simple.
> >>
> >> You have two nodes (A and B) and a link in each direction (A->B and B->A).
> >>
> >> You have three unknowns, the time-difference between the nodes (T_B -
> >> T_A), the delay from node A to B (d_AB) and the delay from node B to A
> >> (d_BA).
> >>
> >> You make two observations of the pseudo-range from node A to node B
> >> (t_AB) and from node B to node A (t_BA). These are made by the source
> >> announcing it's time and the receiver time-stamping in it's own time
> >> when it occurs.
> >>
> >> t_AB = T_B - T_A + d_AB
> >> t_BA = T_A - T_B + d_BA
> >>
> >> We thus have three unknowns and two equations. You can't solve that.
> >> For each link you add, you add one observation and one unknown. For each
> >> node and two links you add, you add three unknowns and two observations.
> >> You can't win this game.
> >>
> >> There are things you can do. Let's take out observations and add them,
> >> then we get
> >>
> >> RTT = t_AB + t_BA = (T_B - T_A) + d_AB + (T_A - T_B) + d_BA
> >>       = d_AB + d_BA
> >>
> >> Now, that is useful. If we diff them we get
> >>
> >> /|T = t_AB - t_BA = (T_B - T_A) + d_AB - (T_A - T_B) - d_BA
> >>       = 2(T_B - T_A) + d_AB - d_BA
> >>
> >> TE = /|T / 2 = T_B - T_A + (d_AB - d_BA)/2
> >>
> >> So, diffing them gives the time-difference, plus half the asymmetric delay.
> >> If we assume that the delay is symmetric, then we can use these measures
> >> to compute the time-difference between the clocks, and if there is an
> >> asymmetry, it *will* show up as a bias in the slave clock.
> >>
> >> The way to come around this is to either avoid asymmetries like the
> >> plague, find a means estimate them (PTPv2) or calibrate them away.
> >>
> >> Is that formal enough for you?
> >
> > It may be.  This I did know, and would seem to suffice, but I recall a
> > triumphant comment from Dr. Mills in one of his documentation pieces.
> > Which I cannot recall well enough to find.  It may be the above
> > analysis that was being referred to, or something else.
> I can't recall. The above I came up with myself some 10 years ago or so.

When I awoke the day after writing the above, I saw two problems with
the above analysis.  

First is that with added message-exchange volleys, one does not get
added variables and equations, one instead gets repeats of the
equations one already has.  If there is no noise, the added volleys
convey no new information.  If there is noise, multiple volleys allows
one to average random noise out.

Second is that what is proven is that a specific message-exchange
protocol cannot work, not that there is no possible protocol that can

> Will see if I can find Dave's reference.

I hit pay dirt yesterday, while searching for data on outliers in 1588
systems.   Dave's reference may well be in the references of the
following article.

"Fundamental Limits on Synchronizing Clocks Over Networks", Nikolaos M.
Freris, Scott R. Graham, and P. R. Kumar, IEEE Trans on Automatic
Control, v.56, n.6, June 2011, pages 1352-1364.

> > I also took the next step, which is to treat d_AB and d_BA as random
> > variables with differing means and variances (due to interference from
> > asymmetrical background traffic), and trace this to the effect on clock
> > sync.  It isn't pretty on anything like a nanosecond scale.  The
> > required level of isolation between PTP traffic and background traffic
> > is quite stringent.
> It's even worse when you get into packet networks, as the delays contain 
> noise sources of variable mean and variable deviation, besides being 
> asymmetrical. NTP combats some of that, but doesn't get deep enough due 
> to too low packet rate. PTP may do it, but it's not in the standard so 
> it will be propritary algorithms. The PTP standard is a protocol 
> framework. ITU have spent time to fill in more of the empty spots.

Yes.  In closed networks, the biggest cause of asymmetry I've found is
interference between NTP traffic and heavy background traffic in the
operating system kernels of the hosts running application code. 
Another big hitter was background backups via NFS (Network File
System).  The network switches were not the problem.  What greatly
helps is to have a LAN for the heavy applications traffic, and a
different LAN for NTP and the like, forcing different paths in the OS
kernel to be taken.

Joe Gwinn

More information about the questions mailing list