[ntp:hackers] [Gpsd-dev] ntpd shm changes

Warner Losh imp at bsdimp.com
Sat Mar 19 15:05:13 UTC 2011

On 03/19/2011 05:20, Terje Mathisen wrote:
> Dave Hart wrote:
>> That's not the current protocol and I don't think it's been proposed,
>> and you show why it doesn't work.  The two reads of count are before
>> and after reading the timestamps, not both before.
>> To answer Harlan's query, yes, volatile restricts the compiler from
>> reordering accesses (reads and writes) amongst volatile variables,
>> which is why I suggest every member of the structs (or the struct
>> types) be volatile.  See for example [1].
> I have spent a few years in comp.arch where the issue of 'volatile' 
> comes up regularly:
> It _does not_ do what you believe it to do!
> In particular, you cannot _depend_ on volatile as a portable method to 
> handle cross-cpu synchronization. :-(

Yes.  Your best bet, and what most other programs/libraries that need to 
do these things are is to define a set of atomic operations.  Each OS 
has its own definitions, but there's a lot of overlap for one to define 
them correctly in terms of the native OS'.  The next tier down just 
reuses somebody's that has a good selection of architectures.  Nearly 
all architectures can have these atomic operations defined.  The 
exceptions are some older, low-end RISC architectures that need atomic 
operations from the OS to work.

Keep in mind, that this isn't just about cross-cpu synchronization.  
Your process can be preempted at any point, thus simulating most, if not 
all, the effects of cross CPU issues.  Memory barriers can only go so 
far, and are only part of the solution.

> It is still the best option available if you have to avoid proper, 
> os/cpu-specific, atomic operations.

Is there some reason to avoid them?  I've been involved in about a dozen 
efforts to avoid them over the years.  Every single one of them has 
thrown in the towel and either used the OS' native atomic API, or 
imported the atomic operations from BSD (when the code couldn't be 
distributed under the GPL or people wanted to use these) or Linux (when 
the code could and people wanted these).

>> To play it extra safe, we could retain valid (though I'd argue for
>> only the producer modifying it) and use a memory-barrier-producing
>> construct where available, such as __sync_syncrohronize [2].  Lacking
>> compiler support for memory barriers, the arguably redundant use of
>> valid and count-checking might keep us out of trouble with reordering
>> processors.
>> I've not heard of problems with the newer protocol (mode 1 in shared
>> memory struct) despite it lacking volatile qualifiers and memory
>> barriers.  Adding volatile (and where available a full memory barrier)
>> certainly shouldn't make things worse.
> The best you can do with a single producer and one/multiple readers 
> (which are allowed to read the same data), is probably to have 
> multiple slots for the data packets together with a single pointer to 
> the current/last updated slot.
> The pointer can even be a simple index, using a byte variable makes it 
> impossible for a reader to read a partially updated value.
> Terje

More information about the hackers mailing list