[ntp:hackers] [Gpsd-dev] Possible shm problem: Structure alignment?
tglassey at earthlink.net
Sat Mar 26 12:31:19 UTC 2011
On 3/25/2011 4:51 PM, Dave Hart wrote:
> On Fri, Mar 25, 2011 at 20:56 UTC, Terje Mathisen<terje at tmsw.no> wrote:
>> OK, there is already an attempt underway (started by Håkan from Sweden) to
>> define a new (POSIX?) shm interface with ns support, merging that work with
>> this effort to get a much more portable/stable interface would be good.
> Juergen Perlinger also has proposed changes to ntpd's refclock_shm.c
> in the form of a tarball and an html to-do list in
> Here's an attempt to itemize the various improvements suggested.
> 0. Leave existing reflcock_shm.c shared memory structure definition
> in place for backwards compat, allow user to control which variant to
> use via ntp.conf. I'm not sure what the default should be.
OK but this mandate also should weave its way through the other
suggestions too. Otherwise I actually dont think this is a good idea.
The backward compatibility for the internal layout of the reference port
is not something we need to stick to... but if we do then the rest of my
commentary is as follows regarding the compilation environ.
> 1. Ensure 32-bit and 64-bit clients use the same layout expressed in
> fixed-size types.
Again - this is not such a good idea since it doesnt allow for
per-architecture optimizations to be integrated as easily. The layout
of the client should be determined by the type of CPU its running on so
that it can be optimized to its best performance...
> 2. Use 64-bit wide storage where atomic access is required, declared
> as a union of u_int32, a double to force 64-bit alignment on 32-bit
> platforms lacking u_int64, and a u_int64, accessed through a member
> name aliased to the u_int64 on 64-bit, and the least significant
> u_int32 on 32-bit. This accomodates any 64-bit architectures where
> 32-bit access is not atomic. Even if there is no such architecture
> today, I believe it's worthwhile future-proofing.
Yes - good idea.
> 3. Similarly for time_t containers, use a union of time_t,
> u_int32, and a double to force 64-bit storage and alignment.
> 4. Switch to nanosecond resolution for timestamps.
> 5. Arrange the members to avoid compiler padding for alignment.
I disagree with this one. The ability to select or turn off padding is
going to become a critical in tuning internal loop functionality in
higher performance implementations. IMHO Padding must be allowed and in
some instances compiler runtime packaging will be critical to those
systems use of the reference port code.
> 6. Switch from sysv to POSIX named shared memory to broaden supported
> platforms. Assumption here is there are no interesting systems which
> support sysv shared memory but not POSIX.
No - I want to push back here on making this u-turn now. Allow a choice
at Compile Time so that POSIX Thread/SHM services which are not a part
of legacy systems still allows use of the system without forcing the
installation of the thread/shm kit as well to complete a runtime
> 7. Mark all members volatile to bar access re-ordering by the
> compiler. Guard the declaration with extern "C" when pulled into C++
> code to use C volatile semantics.
OK but this also needs to be selectable at compile time.
> 8. Come to agreement on the consistency protocol for one writer,
> multiple readers, including use of memory barriers where available,
> but do not require memory barrier support from the compiler, as (IIUC)
> x86/x64 do not need them for volatile atomic accesses to be coherent
> across processors, and the ovewhelming majority of current
> refclock_shm users are on x86 or x64.
OK but this also is something that needs to be thought out based on use
models. There are several I can think of which would be key in a
cloud-system using SHM type service interface for passing NTP data from
blade to blade in a tightly coupled system and those relative to the
more loosely coupled Beowulf type clusters.
> I have a feeling I've omitted at least one suggested improvement,
> please followup with any others that come to mind.
> If your message is rejected by either list, feel free to forward it to
> me with a request to repost to that list.
> Thanks for all the lively participation. I think we've come a long
> way towards consensus.
> Dave Hart
> hackers mailing list
> hackers at lists.ntp.org
More information about the hackers