[ntp:questions] Re: Jonathan Buzzard's radioclkd and FreeBSD

Barry Bouwsma ntp-200203 at remove-NOSPAM-to-reply.NOSPAM.dyndns.dk
Mon Dec 15 21:13:03 UTC 2003


[Valid address above works over IPv6 only, sometimes, maybe, or usually not]

[Sorry for the long delay since the original message was posted; I've
 hardly been online since then to post this response...]


> > I added a few sanity checks, plus auto-polarity detection for DCF,

> Again I would be interested to see these patches. If I can work

Hoo boy, they need a good cleaning.  I've started on that.  Actually,
what I am going to do is to extract the potability (*hic*, sorry, I
meant to write _portability_) patches for *BSD plus a few probably-
already-incorporated bugfixes against virginal 0.7 source, then as I
get more ambitious, I'll scrub and polish the extra features and post
them separately with explanation, more or less in the order I added
them.

So to start with the portability fixes:  These hacks appear to get the
0.7 radioclkd source to compile and run on both NetBSD and FreeBSD.
I've also looked at the OpenBSD CVS repository I have, to guess what I
need for that, but I don't have such a machine so I haven't been able
to verify this all works, and would be happy to hear if someone tries
it.  I've made no attempt to add DragonFly (should be comparable to
FreeBSD 4) and others so far.

The program works fine under FreeBSD for me (under RELENG_4, and also
well over a year ago on 5.x).  Also, I've gotten it working under
NetBSD, although there appears to be a problem in that if ntpd were
to disappear out from under radioclkd, such as when I have to shoot
ntpd in the head to get it to listen to and use recently-added IP
addresses, it seems that radioclkd goes into a tight loop and fills
my logs with hundreds of messages per second, so beware of that.
I have never seen this under FreeBSD.

My more-than-year-old NetBSD was missing sched_param and SCHED_FIFO
and such so I've wrapped them appropriately.  My CVS repository hints
they were relocated into optional header files that I don't have
installed (perhaps unproven-pthreads or something?) but they'll be
used if found -- feedback from someone with a more up-to-date and
complete system, or more knowledge than I, is appreciated.

Also, it appears that FreeBSD 5.x gained the mlockall() and munlockall()
calls in userland as of 11.Aug, but I haven't built a machine with
sources from around that time (this is after 5.0 and 5.1 hit the streets)
to verify it works.  There's a __FreeBSD_version comparison that I
just hacked in to theoretically allow one to use those calls from a
safe release, but the numbers need fillin' in.  Before this date, the
do-nothing stubs were there but didn't work.

I also added a kindergarten-level attempt to my kernel to imitate the
Linux TIOCMIWAIT ioctl which worked for some minutes to hours before
the inevitable race wedged my machine.  Without this, one must use
polling so I enable it automagically; also I revert to polling if a
TIOCMIWAIT fails (on my machine where the stub is there but returns
an error now).  My memory is fuzzy but avoiding polling manages to
drop CPU time by a factor of something like 10 or 20.

The BSDen (at least Free/Net?) support TIOCDCDTIMESTAMP, which gives an
accurate timestamp of the status line transition.  I think that's
only on the positive-going edge of the signal -- else one can specify
which edge to give the timestamp on, but I could be confusing another
ioctl() that's used for gathering PPS info.  Unfortunately, this is
the opposite polarity of the radioclkd default, so one needs my later
hacks to be able to use this.  Anyway, if one can use TIOCDCDTIMESTAMP,
I automatically make use of it and increase the polling interval.
One still must poll to detect the opposite transition, but as this
only is needed to determine the approximate pulse width (with 100/200
msec for DCF and several tens of msec margin of error), one can
easily do so less often, though on my old machine, disk activity is
often enough to throw the signal way off (it noticeably affects other
activities I do too, but nobody these days seriously considers using
such a machine for anything but an effective space heater).

The default 5msec poll interval gives 10msec inaccuracy of the
gettimeofday() timestamp, thus it is highly advantageous to use the
*TIMESTAMP ioctl if possible.  Assuming you're skilled enough with
a multimeter and soldering iron not to melt craters in the meter and
burn the skin off your palms, you should be able to add another
inverting transistor to the circuit, or start from scratch with an
off-the-shelf radio-controlled alarm clock, without too much difficulty.
Later patches allow one to specify an `-i / --invert' option so that
radioclkd will decode TIODCDTIMESTAMP-compatible polarity signals.
Still later, as noted, I automatically switch DCF77 polarity to match
the supplied signal.  But if you have a stock BSD, you want to deliver
a signal to the DCD pin that's mostly low, which can light an LED to
ground for short 100/200msec pulses, for best accuracy.


> > One other change I made to ntpd because of the radioclk program and/or
> > the GENERIC driver, was to be more tolerant of wildly-off time data,

> What I think we should do here is that the radioclkd program should
> check the time decoded against what it expects it should get, and

That's what I do, at least in part, deciding whether to update the
shared memory value every second.  I think I still pass way-off values
each minute.  The problem has been with the GENERIC RAWDCF driver with
PPS that has caused ntpd to bail -- it doesn't do this sanity checking,
and if it gets incorrect time info, it's still passing along the PPS
data, and I suspect this is what fools ntpd into believing it, though
I don't know for sure.  At least with minpoll cranked up as often as
possible.

I have notes for a TODO to save the last decoded time in radioclkd
whether or not it was decided to pass it to ntpd, and compare the
current decoded time not only to the system time, but also to the
last decoded time in case the system time might have gone bonkers.
I'm thinking of startup, if one doesn't have a trustworthy clock on the
machine and hasn't set the time (ntpdate or similar), when one does
not know whether the first radioclkd time is accurate, or if the
machine clock has stepped to an incorrectly-decoded value (unlikely,
but possible).  I'd have to hack through the overgrowth on my code to
see if I actually (try to) do that already.


> must be a load of rubbish, ignore. Currenty it only rejects times more
> than 1000 secs from the system time, and the above (which has just
> come to me) seems to be a worthwhile extra sanity check.

This 1000sec check must not be in 0.7 (my codebase), and I'm not
sure I agree with it.  I use radioclkd offline with ntpd as an ersatz
ntpdate at boot, when I should not trust the system time.  Also, if
something yanks the system time by more than this, like some fumble-
fingered operator abusing the `date' command, I suspect that then your
radioclkd will go silent.

Specifically, I have two modes of operation for radioclkd: startup
mode, and synced mode.  Okay, three modes -- unsynced mode.  In
startup mode, I assume the data from radioclkd is trustworthy, for
lack of anything better, and I'm wanting to get ntpd to sync to it
as quickly as possible.  I switch into synced mode as soon as the
time from radioclkd is within a user-defineable tolerance of the
system time.  During startup I can decide whether to auto-flip the
polarity and such; in synced mode in case of an out-of-tolerance
error I stop updating the timestamp every second, but still deliver
valid decoded data, accurate or not, each minute, I think.

The problem I see with your idea for a radioclkd sanity check is if
the system time gets yanked, then your idea of two minutes ago may
be wrong.  (I suspect something like nanotime/nanouptime might be of
help here; I need to learn more about how the system keeps track of
time before I can say anything, however).  Since other than some
programming error or consistent error in the radio signal is unlikely
to give repeated consistent time offsets, I prefer to let ntpd decide
if the radioclkd time is trustworthy, and let radioclkd do little
more than simple sanity checks on the decoded signal -- at boot, I
step the system time once with radioclkd/ntpdate, then give ntpd
another chance to set the system time in normal operation with the
radioclkd data, and after this, it should take more than a single
false top-of-minute timestamp in shared memory to cause ntpd to
step the time again, I should hope.

Also, as I've noted, I always connect the DCD line signal to the
data line as well, to allow ntpd's GENERIC PARSE driver to work on
the same DCF data for a second reference, in case one of them delivers
wonky data -- though now I have three other radio clocks so it's even
less likely one will deliver false data that ntpd decides to trust.


Anyway, enough blabbering, here's the portability patches.  I'll
post the feature patches RSN, though I'm not online enough for them
to appear with any timeliness, apologies.


--- radioclkd.c.orig	Fri Feb  8 16:26:26 2002
+++ radioclkd.c	Sun Nov 23 19:46:23 2003
@@ -46,6 +46,7 @@
  * 675 Mass Ave, Cambridge, MA 02139, USA.
  *
  */
+/* Ugly BSD portability and bugfix patches by Barry Bouwsma */
 
 static const char rcsid[]="$Id: radioclkd.c,v 1.4 2002/02/08 15:21:11 jab Exp jab $";
 
@@ -54,7 +55,14 @@
 #include<unistd.h>
 #include<time.h>
 #include<sys/time.h>
+#if (defined(__FreeBSD__) || defined(__NetBSD__) || defined(__OpenBSD__))
+#include <sys/ttycom.h>
+#else  /* BSD */
 #include<termio.h>
+#endif  /* BSD */
+#if (defined(__NetBSD__) || defined(__OpenBSD__))
+#include <sys/ioctl.h>
+#endif  /* NetBSD/OpenBSD */
 #include<sys/types.h>
 #include<sys/stat.h>
 #include<sys/mman.h>
@@ -66,6 +74,9 @@
 #include<syslog.h>
 #include<paths.h>
 #include<string.h>
+#if defined(__FreeBSD__)  /* to decide if mlockall() and its ilk should work */
+#include <osreldate.h>
+#endif  /* FreeBSD */
 
 
 #define PID_FILE _PATH_VARRUN "radioclkd.pid"
@@ -128,14 +139,14 @@
 int DecodeDCF77(char *code, int length)
 {
 	int bcd[] = { 4,3,1,4,2,1,4,2,3,4,1,4,4 };
-	int parity[] = { 8,7,21 };
+	int parity[] = { 8,7,23 };
 	int segment[13];
 	int i,j,k,sum;
 	struct tm decoded;
 
 
 	/* check the parity bits */
-	k = length-42;
+	k = length-38;
 	for(i=0;i<3;i++) {
 		sum = 0;
 		for(j=0;j<parity[i];j++,k++)
@@ -146,10 +157,10 @@
 
 	/* calculate all the individual BCD segments */
 	k = length-38;
-	for(i=0;i<12;i++) {
+	for(i=0;i<13;i++) {
 		sum = 0;
 		for(j=0;j<bcd[i];j++,k++)
-			sum += ((code[k]==1) ? 1 : 0) << (bcd[i]-j-1);
+			sum += ((code[k]==1) ? 1 : 0) << j;
 		segment[i] = sum;
 	}
 
@@ -164,16 +175,16 @@
 
 	/* some extra sanity checks */
 	if ((decoded.tm_min>59) || (decoded.tm_hour>23) ||
-			(decoded.tm_wday>6) || (decoded.tm_mday>31) ||
+			(decoded.tm_wday>7) || (decoded.tm_mday>31) ||
 			(decoded.tm_mon>11) ||(decoded.tm_year>199))
 		return -1;
 
 	/* daylight savings */
-	decoded.tm_isdst = (code[length-42]==1) ? 1 : 0;
+	decoded.tm_isdst = 0;
 
 	/* return adjust for CET */
 
-	return mktime(&decoded)-3600;
+	return mktime(&decoded)-(3600 * (code[length-42]+1));
 }
 
 
@@ -335,33 +346,58 @@
 {
 	int arg;
 
+	int error;
 	/* get the status of the DCD line */
+#ifdef TIOCDCDTIMESTAMP
+#define TIMESTAMP(x)	if (ioctl(fd, TIOCDCDTIMESTAMP, x)!=0) return -1
+#define TIOSLEEP 10000
+#else /* TIOCDCDTIMESTAMP */
+#define TIMESTAMP(x)	gettimeofday(x, NULL)
+#define TIOSLEEP 5000
+#endif /* TIOCDCDTIMESTAMP */
+
 	arg = 0;
 	if (ioctl(fd, TIOCMGET, &arg)!=0)
 		return -1;
 
 	/* return if the level is the specified one */
-	gettimeofday(tv, NULL);
-	if ((arg & TIOCM_CD) && (signal==1))
+	if ((arg & TIOCM_CD) && (signal==1)) {
+		TIMESTAMP(tv);
 		return 0;
-	else if ((!(arg & TIOCM_CD)) && (signal==0))
+	}
+	else if ((!(arg & TIOCM_CD)) && (signal==0)) {
+		gettimeofday(tv, NULL);
 		return 0;
+	}
 
 	/* otherwise wait for the DCD line to change status */	
 	while (poll==1) {
-		usleep(5000);
+		usleep(TIOSLEEP);
 		if (ioctl(fd, TIOCMGET, &arg)!=0)
 			return -1;
-		gettimeofday(tv, NULL);
-		if ((arg & TIOCM_CD) && (signal==1))
+		if ((arg & TIOCM_CD) && (signal==1)) {
+			TIMESTAMP(tv);
 			return 0;
-		else if ((!(arg & TIOCM_CD)) && (signal==0))
+		}
+		else if ((!(arg & TIOCM_CD)) && (signal==0)) {
+			gettimeofday(tv, NULL);
 			return 0;
+		}
 	}
 
-	if (ioctl(fd, TIOCMIWAIT, TIOCM_CD)!=0)
+#ifdef TIOCMIWAIT
+	error = ioctl(fd, TIOCMIWAIT, TIOCM_CD);
+	if (error!=0) {
+		poll=1;
+		fprintf(stderr,"Problem with TIOCMIWAIT (%d), reverting to polling\n", error);
 		return -1;
+	}
+	if ((arg & TIOCM_CD) && (signal==1)) {
+		TIMESTAMP(tv);
+		return 0;
+	}
 	gettimeofday(tv, NULL);
+#endif /* TIOCMIWAIT */
 	
 	return 0;
 }
@@ -495,7 +531,9 @@
 	if (test==0) {
 		syslog(LOG_INFO, "Exiting...");
 		unlink(PID_FILE);
+#if (defined(MCL_CURRENT) && (!defined(__FreeBSD__) /* XXX || (__FreeBSD_version >= 5xxxxx ???  ) */ ))  /* is the mlockall() call available? */
 		munlockall();
+#endif /* MCL_CURRENT ... */
 		shmdt(stamp);
 	} else {
 		fprintf(stderr, "radioclkd: Exiting...\n" );
@@ -560,13 +598,19 @@
 	int i,pid,error;
 	time_t decoded;
 	struct timeval local,radio;
+#if (defined(SCHED_FIFO))  /* missing from my NetBSD */
 	struct sched_param schedp;
+#endif
 	FILE *str;
 	char devname[16] = "/dev/";
 
 
 	/* process the command line arguments */
+#ifndef TIOCMIWAIT  /* must poll, sorry */
 	poll = 0;
+#else  /* TIOCMIWAIT */
+	poll = 1;
+#endif /* TIOCMIWAIT */
 	test = 0;
 	for (i=1;i<argc;i++) {
 		if ((!strcmp(argv[i], "-h")) || (!strcmp(argv[i], "--help"))) {
@@ -664,16 +708,24 @@
 			return 1;
 		}
 
+#if (defined(SCHED_FIFO))  /* can we do this, or not? */
 		/* set realtime scheduling priority */
 		memset(&schedp, 0, sizeof(schedp));
 		schedp.sched_priority = sched_get_priority_max(SCHED_FIFO);	
 		if (sched_setscheduler(0, SCHED_FIFO, &schedp)!=0)
 			syslog(LOG_INFO, "error unable to set real time "
 				"scheduling");
+#else  /* no, do without, but make note of it... */
+		syslog(LOG_INFO, "error unsupported real time scheduling call");
+#endif  /* SCHED_FIFO */
 
 		/* lock all memory pages */
+#if (defined(MCL_CURRENT) && (!defined(__FreeBSD__) /* XXX || (__FreeBSD_version >= 5xxxxx ???  ) */ ))  /* is the mlockall() call available? */
 		if (mlockall(MCL_CURRENT | MCL_FUTURE) !=0)
 			syslog(LOG_INFO, "error unable to lock memory pages");
+#else  /* no, do without, but make note of it... */
+		syslog(LOG_INFO, "error unsupported memory pages lock call");
+#endif  /* MCL_CURRENT ... */
 	
 	}
 


Some comments about these hacks -- for Linux, you probably want to
keep the original gettimeofday() calls where they were, rather than
delaying them to TIMESTAMP().  It won't matter for TIOCDCDTIMESTAMP
(BSD) because that ioctl returns the timestamp.  I figured I'd avoid
an unneeded call to gettimeofday(), at the expense of introducing a
slight further delay before invoking it.  In practice, you're not
going to tell the difference with a longwave radio clock signal and
the jitter provided by an off-the-shelf decoder that triggers on the
radio carrier strength.

I'm not a programmer, or a hacker, so the style of these hacks should
be atrocious.  I just wanted it to function.


Now, in order to use this well under FreeBSD, as noted, you want to
use DCD signals with a different polarity than that for which the
program was written.  The following patch adds a commandline option
to toggle the polarity of the signals radioclkd is looking for, and
should work on all signals.

Also, a couple personal preference hacks are here -- the device can
also be specified by full /dev/foo name (helps with shell completion)
and I personally don't need to wait 3 seconds for my clocks to become
stable -- pretty much every radio clock I have starts delivering
useful information almost immediately after battery is inserted.

This patchset depends on the previous portability/bugfix set in order
to apply cleanly, probably.  (Which is why I'm sneaking in a couple
non-important personal patches, so I don't have to keep excising
them from each set of diffs, sorry for the inconvenience.)

This `-i / --invert' option is handy in case `radioclkd -t' test
mode doesn't produce useful output every second.  As noted, using
inverted-detection mode is almost required to get accurate timing
with Free/NetBSD.  A later patch I'll post attempts to determine
this polarity automatically (at least for DCF77 signals, possibly
applicable to others when unambiguous), but that patch depends on
others that came before it in time.


--- radioclkd.c.OLD	Sun Nov 23 19:46:23 2003
+++ radioclkd.c	Sun Nov 23 19:45:55 2003
@@ -46,7 +46,11 @@
  * 675 Mass Ave, Cambridge, MA 02139, USA.
  *
  */
-/* Ugly BSD portability and bugfix patches by Barry Bouwsma */
+
+/*
+ *  Ugly BSD portability, signal polarity inversion, and
+ *  bugfix patches by Barry Bouwsma
+ */
 
 static const char rcsid[]="$Id: radioclkd.c,v 1.4 2002/02/08 15:21:11 jab Exp jab $";
 
@@ -108,6 +112,7 @@
 static int shmid;
 struct shmTime *stamp;
 
+static int polarity;
 
 enum { MSF=0x01, DCF77=0x02, WWVB=0x04 };
 
@@ -126,6 +131,7 @@
 Decode the time from a radio clock attached to a serial port\n\n\
   -t,--test     print pulse lengths and time to stdout\n\
   -p,--poll     poll the serial port instead of using interrupts\n\
+  -i,--invert   invert interpretation of serial port 0/1 signal polarity\n\
   -h,--help     display this help message\n\
   -v,--version  display version\n\
 Report bugs to jonathan at buzzard.org.uk\n"
@@ -419,7 +425,8 @@
 
 
 	/* wait till the signal goes high */
-	WaitOnSignal(fd, 1, &end);
+	/* This depends on the polarity of the serial signal... */
+	WaitOnSignal(fd, 1 - polarity, &end);
 
 	/* loop polling the serial port */
 	radio = 0;
@@ -428,7 +435,8 @@
 	for (i=1;i<128;) {
 	
 		/* wait till the start of a pulse */
-		WaitOnSignal(fd, 0, &start);
+		/* This depends on the polarity of the serial signal... */
+		WaitOnSignal(fd, polarity, &start);
 		timersub(&start, &end, &length);
 
 		/* check for the DCF77 minute marker */
@@ -442,7 +450,7 @@
 		if ((length.tv_usec>=60000) && (length.tv_usec<=150000)) {
 			code[i-1] = 3;
 			/* wait for signal to go high again */
-			WaitOnSignal(fd, 1, &start);
+			WaitOnSignal(fd, 1 - polarity, &start);
 			continue;
 		}
 
@@ -450,7 +458,7 @@
 		usleep(1000);
 
 		/* determine what sort of pulse we have just received */
-		WaitOnSignal(fd, 1, &end);
+		WaitOnSignal(fd, 1 - polarity, &end);
 		timersub(&end, &start, &length);
 		if ((length.tv_usec>=60000) && (length.tv_usec<150000)) {
 			code[i++] = 0;
@@ -612,6 +620,7 @@
 	poll = 1;
 #endif /* TIOCMIWAIT */
 	test = 0;
+	polarity = 0;
 	for (i=1;i<argc;i++) {
 		if ((!strcmp(argv[i], "-h")) || (!strcmp(argv[i], "--help"))) {
 			printf(USAGE_STRING);
@@ -623,8 +632,13 @@
 			poll = 1;
 		} else if ((!strcmp(argv[i], "-t")) || (!strcmp(argv[i], "--test"))) {
 			test = 1;
+		} else if ((!strcmp(argv[i], "-i")) || (!strcmp(argv[i], "--invert"))) {
+			polarity = 1;
 		} else {
-			strncat(devname, argv[i], 10);
+			if (!strncmp(argv[i], "/dev/",5))
+				strncpy(devname, argv[i], 15);
+			else
+				strncat(devname, argv[i], 10);
 		}
 	}
 			
@@ -658,7 +672,8 @@
 	}
 	
 	/* pause a few seconds */
-	sleep(3);
+/* XXXX 3 seconds is a bit too long for my taste... */
+	sleep(1);
 
 	/* do things specific to the daemon or test version */
 	if (test==1) {


The original poster (long ago -- sorry, I spend too much of my life
offline) probably will want to apply both of the above sets of hacks
to the 0.7 radioclkd, and/or try to shoehorn them into a more up-to-
date recent vintage as appropriate, in order to get a working
radioclkd for his FreeBSD...




Stay tuned; at this rate, sometime next year I'll get back online and
be able to post the next set of hacks...

Barry Bouwsma




More information about the questions mailing list