[ntp:questions] A proposal to use NIC launch time support to improve NTP

Brian Utterback brian.utterback at oracle.com
Wed Dec 19 13:48:41 UTC 2012

On 12/18/2012 7:05 PM, Ulf Samuelsson wrote:
> Brian Utterback <brian.utterback at oracle.com> wrote:
>> On 12/13/2012 5:00 AM, Jonatan Walck wrote:
>>>> This is going to be very hard to get it to be useful. Looking at
>>>>> the specs for the card, the timestamp you give is relative to a
>>>>> clock that is internal to the controller, and is only accurate to
>>>>> the nearest second. That is, it is like the PPS in that it is
>>>>> assumed that the clock is in sync to within .5 seconds to avoid
>>>>> aliasing the timestamp.
>>>>> Brian.
>>> The internal clock of the network controller is the PHC for IEEE1588,
>>> it has a 1 ns resolution, and can be steered with a 32 bit fractional
>>> of 1 ns. see SYSTIML and TIMINCA in the I210 datasheet.
>>> // jwalck
>> I know that. The problem is that there is going to be jitter introduced
>> when you set the clock from the kernel. That is generally the problem
>> with IEEE 1588, getting the time from the controller to the kernel and
>> vice versa. If you have to go across a PCI bus for instance that will
>> introduce jitter.
>> Brian
> No, you capture the time for the 1 PPS pulse in the network controller.
> Then you tweak the count rate of the timestamp counter up or down.
> Eventually you will have synchronized the timestamp counter with the 1 PPS
> pulse,
> If you run the network contoller from a clocksource derived from the Cesium
> clock
> You should get within a few ns from the pulse,
> It is an inconvenient approach since you can easily synch your timestamp
> with a 1 PPS pulse
> using a few counters if you understand the issue.
> Unfortunately, those counters are not available in the Intel chips, so we
> have to use it they way they
> want us to use it.
> Since launch time will be the arrival time of the client request (through
> H/W timestamping)
> + a constant delay, any delays in the PCIe bus will not affect the
> precision.
> You just have to make sure that the constant delay added, is longer than
> the processing time inside the server.

No, you are missing the point. You have two clocks in this scenario, the 
kernel clock and the network controller clock. If one gets a good time 
then you have to set the other from it. This means that this time will 
have to travel over the PCI bus which will introduce jitter.

Now, if you have a PPS signal available and can provide it to both the 
network controller and the kernel, then you don't have this problem 
since the PPS signal will sync the time to an accuracy better than the 
jitter that was introduced.

Even without the PPS signal to the kernel, your system might be usable, 
since the only timestamp used in the kernel will be for the 
originate/transmit timestamp, and this timestamp will be in sync with 
the the network controller timestamp by virtue of the use of launchtime. 
But you will have to be sure that the kernel clock is always a little 
ahead of the network controller clock, enough so that the actual delay 
in the stack doesn't cause the packet to reach the controller after the 
designated launchtime, but not so far ahead that the timestamp wraps 
(i.e. .5 second). Also, not so far ahead that you get too large a back 
log in the controller of packets waiting to be sent.

Of course, this also all depends on the controller providing receive 
timestamps as well. Otherwise you will replace jitter with systematic 
asymmetric delays, which are worse.


More information about the questions mailing list