[ntp:hackers] Running out of Recvbufs

David L. Mills mills at udel.edu
Sun Dec 4 09:26:41 PST 2005


Terje,

Remember the fuzzball? Fastest gun on the planent 25 years ago. 
Configured as a gateway it handled 300 packets per second when its 
nearest competitor redlined at 10 packets per second. For a recent paper 
I calculated redline for a modern NTP server with modest processor at 
over 50,000 packets per second. Obviously, 40 packets in arrears of 
processing represents not a quirk in arrival rate but a failure 
somewhere along the pipeline.

Dave

Terje Mathisen wrote:

> Danny Mayer wrote:
>
>> David L. Mills wrote:
>>
>>> Danny,
>>>
>>> The grad student had serious problems with weeds due the stitched #ifdef
>>> code to handle different operating systems and versions. I tried to make
>>> a go of it some years back, but got lost myself in the weeds. The
>>> suggestion then was to aovid interleaved code in favor of separate
>>> routines for each system. I have the same problem now with the audio.c
>>> program and include files for the system clock routines.
>>>
>>> The recvbuf issue is troubling. Invariably it means something is broken
>>> and tossing packets might not help. As you suggest, it is probably
>>> better to purge the buffer list and start over, but with a note to the
>>> system log.
>>>
>>> Dave
>>>
>> I agree with your worry about having use up that many recvbuffs but I
>> don't know what else to do that would be reasonable. As Terje says the
>> timestamp is important, but without processing the packet you can't
>> attach a receive timestamp to it. Holding off on processing it means
>> that the timestamp will be much later and likely make your statistics
>> really bad in which case that's the equivalent of a DoS attack. We need
>> to avoid that. The code currently logs the "out-of-recvbuffs" condition.
>
>
> If we have multiple outstanding requests, then we can further streamline
> the processing by assigning the same timestamp T2 (and possibly T3)
> timestamp to every packet:
>
> After all, this is the best possible timestamp we can get, the only
> potential problem is that such a tight loop could conceivably pick up
> packets arriving _after_ the processing started, thereby assigning
> back-dated timestamps, but this should be a statistically rare occurrence.
>
> My mail problem here is the basic one: How is it even possible to run
> out of recvbufs?
>
> As soon as you start processing them, you should be able to handle them
> _much_ faster than they arrive, which means that the queue is totally
> flushed almost immediately.
>
> In particular, making a single call to the timestamping function removes
> almost all external overhead.
>
> My comparison is the network file read function in NetWare, which is
> almost certainly the most optimized networking code ever written:
>
> >From the time of the initial incoming packet interrupt, the handling
> code needed to parse the request packet, check access controls, locate
> the requested data in cache (which happened 93-99% of the time),
> construct the reply packet and hand it off to the (bus-master) network
> interface.
>
> Total code path for this was/is about 300 clock cycles, or ~100 ns on a
> modern 3 GHz CPU. It does require kernel mode code though!
>
> Doing the same for NTP requests is _much_ easier, and it would have
> allowed us to handle 10 Gig Ethernet at wire speed. It would also reduce
> the maximum timestamp error to the initial interrupt latency, all
> subsequent packets in the queue would have a smaller error.
>
> Terje
>
>
> ------------------------------------------------------------------------
>
>
>
> ***********************************************************************
> NOTICE: This e-mail transmission, and any documents, files or previous
> e-mail messages attached to it, may contain confidential or privileged
> information. If you are not the intended recipient, or a person
> responsible for delivering it to the intended recipient, you are
> hereby notified that any disclosure, copying, distribution or use of
> any of the information contained in or attached to this message is
> STRICTLY PROHIBITED. If you have received this transmission in error,
> please immediately notify the sender and delete the e-mail and attached
> documents. Thank you.
> ***********************************************************************
>



More information about the hackers mailing list