[ntp:hackers] More on asymmetric links

David L. Mills mills at udel.edu
Thu Jul 12 20:41:39 UTC 2007


Greg,

I've studied PTP in detail and looked into the products and interfaces 
that use it. The problems to me are that Ethernet intefaces are 
expensive ($600) and gransmasters tht use it cost several thousand bucks 
and don't do vanilla NTP. The buzz in the PTP community is that NTP 
can't do better than one millisecond. Skeptics should look for instance 
at rackety.udel.edu or pogo.udel.edu. Most of the time both are better 
than 20 microseconds, even through a switch.

The best timestamps of course can be struck at interrupt time in the 
driver, but the soft queue level used by ntpd shouldn't be much worse, 
at least for received packets. The remaning issue is how to get a good 
transmit timestamp. This could be done by the media driver and the 
result left somewhere in the packet buffer. The ntpd transmit soft queue 
routine would then remember it and include as an extension field in the 
next transmitted packet. The protocol gets a litle tricky, since it has 
to verify that timestamp with respect to the timestamp in the original 
packet.

Dave

Greg Dowd wrote:

> I'm interested in looking at this. I've been doing a lot of testing on
> getting freq across asymm links and now we've been looking at phase as
> well, from core switched GigE down through ip/dslam. I've got some
> switch/router/dslam gear and hardware timestamping so it's probably
> easier for me to separate stack and network performance, although
> latency in ipdslam is so high it looks similar. Where can I get a copy
> of this code? Is your ntp-dev reflected in isc's tree? I know there's
> some lag.
>
> Have you tried using so_timestamp? That helps on the receive side where
> there is a lot more jitter. On the tx side, you don't have the
> interrupt latency on the network iface and the scheduler tick to content
> with so the lack of tstamp there is not such a big deal. Do you have a
> dataset available? It would be useful to see the timestamp noise and
> distribution. If it's highly modal, maybe it can be filtered.
>
> It's unfortunately that ntp is getting left behind in favor of ptp in
> many newer ip deployments because there is a perception that ntp can't
> achieve the same types of accuracy and freq stability as ptp which is
> nonsensical. socket options like timestamping help, hardware
> timestamping is showing up in more devices every day (hardcoded ptp but
> also packet inspection for qos from guys like Freescale). That
> functionality will likely make its way into servers soon.
>
> Now, if we can just get some decent poll rates and transparent clock
> functionality defined in ntp, the field is level :-)
>
>
>
>
>
> Greg Dowd
> gdowd at symmetricom dot com (antispam format)
> Symmetricom, Inc.
> www.symmetricom.com
> "The current implementation is non-obvious and may need to be improved."
>
>
>
>
> -----Original Message-----
> From: hackers-bounces+gdowd=symmetricom.com at lists.ntp.org
> [mailto:hackers-bounces+gdowd=symmetricom.com at lists.ntp.org] On Behalf
> Of David L. Mills
> Sent: Thursday, July 12, 2007 10:17 AM
> To: hackers at ntp.org
> Subject: [ntp:hackers] More on asymmetric links
>
> Guys,
>
> A suggestion was made some time ago that the asymmetric link problem
> could be solved by measuring the outbound and inbound data rates. This
> could be done by determining the four timestamps for messages of varying
> lengths. In particular, the various messages of the Autokey protocol.
> The reason I climbed on this train was prompted by NASA's requirement
> for deep space missions where the outbound and inbound link data rates
> are far different.
>
> I gave this a whack with varying degrees of success (see
> process_packet() in ntp_proto.c). First, knowing the outbound and
> inbound data rates gives a really precise measure of the actual offset
> corrections. However, the devil is in the details of how to measure
> these rates. I have a 2x64 kb/s ISDN link to campus and of course
> several machines on both the campus and home subnets. Running Autokey
> over the ISDN link got mixed results, as other traffic on the link
> seriously skewed the data, even if the link delays are basically
> symmetric.
>
> The results over a 100-Mb/s link was quite puzzling. The measured data
> rates were far below the actual wire speed and varied a lot. A 10,000
> bit packet takes 100 microseconds on the wire, but the measured
> roundtrip delay varies widely with different drivers and interfaces -
> from 300 microseconds with FreeBSD on a 2.4-GHz Intel to 1.4 ms with
> Solaris on a Sun Blade 60. However, roundtrip delays measured with ping
> are more consistent - 200-400 microseconds. I have to assume most of the
> NTP delay is in the OS, scatter/gather buffering and socket processing
> and could be quite difference for each direction of transmission. So,
> measuring ethernet link data rates with NTP is probably not a good idea.
>
> But, in those cases where the rates are known and, if not and NTP
> packets can be assigned high priority, the current scheme can measure
> the rates accurately. The code is in ntp-dev now, but disabled to
> actually make the corrections. To deploy the scheme some additions to
> the command syntax and semantics needs to be done. Three new options
> need to be added to the server and peer commands: autorate (a switch) to
> enable automatic rate determination and a pair of options outrate and
> inrate to specify the outbound and inbound data rates, respectively. The
> units of outrate and inrate don't matter as long as they are the same
> units.
>
> Adding these options on a per-association basis is a little messy and in
> truth exploits the new configuration design. In this design a syntax
> parse tree is constructed and interpreted when all commands have been
> read. This means the only thing the resolver has to remember is which
> node on the tree is involved. Great gobs of code in peer_config() and
> ntpdc become waste and those programs become essentially trivial. In
> addition, commands can be configured dynamically from afar with ntpq.
>
> I'm not volunteering to do this and anybody else volunteering is
> probably a fool, but it would be great fun and violently useful.
>
> Dave
>
>
> _______________________________________________
> hackers mailing list
> hackers at lists.ntp.org
> https://lists.ntp.org/mailman/listinfo/hackers




More information about the hackers mailing list