[ntp:hackers] Use MSG_CONFIRM flag to keep OS ARP table up to date?

Greg Dowd gdowd at symmetricom.com
Wed May 2 20:30:01 UTC 2012


I definitely agree it would be useful to have at the application layer.  I think one of the challenges there is that the default arp table depth is quite small although I can't remember it now.   Maybe gc_thresh3 is like 1024.  So you will still get thrashing as you boot old entries off the queue, yes?


Greg Dowd
gdowd at symmetricom dot com (antispam format)
Symmetricom, Inc.
www.symmetricom.com
"A clever person solves a problem.  A wise person avoids it." Albert Einstein


-----Original Message-----
From: Heiko Gerstung (Home Office) [mailto:heiko.gerstung at meinberg.de] 
Sent: Wednesday, May 02, 2012 12:16 PM
To: Greg Dowd
Cc: hackers at lists.ntp.org
Subject: Re: [ntp:hackers] Use MSG_CONFIRM flag to keep OS ARP table up to date?

I agree that this is a workaround that can be applied on a dedicated 
appliance (and we did this as well as a first measure to improve 
performance and reduce unnecessary ARP requests), but in general I would 
say that ntpd should follow the - IMHO - approach that is recommended by 
the OS. On the arp(7) manpage the MSG_CONFIRM flag is mentioned when it 
comes to UDP traffic.

Regards,
   Heiko

Am 02.05.12 18:53, schrieb Greg Dowd:
> We didn't modify the app but instead modified the size and stale timeout values for the arp table under linux to prevent the arp spam.
>
> Greg Dowd
> gdowd at symmetricom dot com (antispam format)
> Symmetricom, Inc.
> www.symmetricom.com
> "A clever person solves a problem.  A wise person avoids it." Albert Einstein
>
>
> -----Original Message-----
> From: hackers-bounces+gdowd=symmetricom.com at lists.ntp.org [mailto:hackers-bounces+gdowd=symmetricom.com at lists.ntp.org] On Behalf Of Heiko Gerstung
> Sent: Wednesday, May 02, 2012 3:10 AM
> To: hackers at lists.ntp.org
> Subject: [ntp:hackers] Use MSG_CONFIRM flag to keep OS ARP table up to date?
>
> Hi!
>
> We are currently running stress tests with more than 100k simulated NTP clients against one single NTP server and found out
> that Linux is sending ARP queries for every single NTP client every 30s, putting significant load onto the NTP server
> (running NTP 4.2.6p3) we use as the NTP reference for the simulated clients.
>
> This seems to be caused by Linux not automatically updating the ARP cache when NTP sends its NTP responses to the clients,
> resulting in all ARP table entries to become stale after a period of 30s (this can be changed but would require users to
> mess with their kernel / network configuration). This triggers a new ARP request for each stale table entry
>
> It is possible to use a flag called MSG_CONFIRM when calling send(), sendmsg() or sendto() that basically tells the kernel
> that the ARP table entry for the given MAC / IP Address pair is still valid, avoiding that the corresponding ARP table entry
> becomes stale.
>
> My question: Did anyone use the MSG_CONFIRM flag with a UDP based socket communication in other projects? Is there any
> obvious technical/security disadvantage using this flag within NTP? I patched my ntpd and applied this and it immediately
> improved overall performance of the NTP server (and obviously reduced ARP related traffic dramatically).
>
> I am not sure if Linux is the only affected OS as it is the only one I was able to test/look at.
>
> Another big problem with this large-scale simulation: NTP only uses one of the CPU cores, which also dramatically limits the
> performance on multi-processor machines. Do you know of anyone who looked at this and probably tried to add multi CPU
> support to ntpd?
>
> Thanks in advance for any help, comments or feedback!
>
> Regards,
>    Heiko
>
>



More information about the hackers mailing list