[ntp:hackers] Running out of Recvbufs
David L. Mills
mills at udel.edu
Sun Dec 4 09:26:41 PST 2005
Terje,
Remember the fuzzball? Fastest gun on the planent 25 years ago.
Configured as a gateway it handled 300 packets per second when its
nearest competitor redlined at 10 packets per second. For a recent paper
I calculated redline for a modern NTP server with modest processor at
over 50,000 packets per second. Obviously, 40 packets in arrears of
processing represents not a quirk in arrival rate but a failure
somewhere along the pipeline.
Dave
Terje Mathisen wrote:
> Danny Mayer wrote:
>
>> David L. Mills wrote:
>>
>>> Danny,
>>>
>>> The grad student had serious problems with weeds due the stitched #ifdef
>>> code to handle different operating systems and versions. I tried to make
>>> a go of it some years back, but got lost myself in the weeds. The
>>> suggestion then was to aovid interleaved code in favor of separate
>>> routines for each system. I have the same problem now with the audio.c
>>> program and include files for the system clock routines.
>>>
>>> The recvbuf issue is troubling. Invariably it means something is broken
>>> and tossing packets might not help. As you suggest, it is probably
>>> better to purge the buffer list and start over, but with a note to the
>>> system log.
>>>
>>> Dave
>>>
>> I agree with your worry about having use up that many recvbuffs but I
>> don't know what else to do that would be reasonable. As Terje says the
>> timestamp is important, but without processing the packet you can't
>> attach a receive timestamp to it. Holding off on processing it means
>> that the timestamp will be much later and likely make your statistics
>> really bad in which case that's the equivalent of a DoS attack. We need
>> to avoid that. The code currently logs the "out-of-recvbuffs" condition.
>
>
> If we have multiple outstanding requests, then we can further streamline
> the processing by assigning the same timestamp T2 (and possibly T3)
> timestamp to every packet:
>
> After all, this is the best possible timestamp we can get, the only
> potential problem is that such a tight loop could conceivably pick up
> packets arriving _after_ the processing started, thereby assigning
> back-dated timestamps, but this should be a statistically rare occurrence.
>
> My mail problem here is the basic one: How is it even possible to run
> out of recvbufs?
>
> As soon as you start processing them, you should be able to handle them
> _much_ faster than they arrive, which means that the queue is totally
> flushed almost immediately.
>
> In particular, making a single call to the timestamping function removes
> almost all external overhead.
>
> My comparison is the network file read function in NetWare, which is
> almost certainly the most optimized networking code ever written:
>
> >From the time of the initial incoming packet interrupt, the handling
> code needed to parse the request packet, check access controls, locate
> the requested data in cache (which happened 93-99% of the time),
> construct the reply packet and hand it off to the (bus-master) network
> interface.
>
> Total code path for this was/is about 300 clock cycles, or ~100 ns on a
> modern 3 GHz CPU. It does require kernel mode code though!
>
> Doing the same for NTP requests is _much_ easier, and it would have
> allowed us to handle 10 Gig Ethernet at wire speed. It would also reduce
> the maximum timestamp error to the initial interrupt latency, all
> subsequent packets in the queue would have a smaller error.
>
> Terje
>
>
> ------------------------------------------------------------------------
>
>
>
> ***********************************************************************
> NOTICE: This e-mail transmission, and any documents, files or previous
> e-mail messages attached to it, may contain confidential or privileged
> information. If you are not the intended recipient, or a person
> responsible for delivering it to the intended recipient, you are
> hereby notified that any disclosure, copying, distribution or use of
> any of the information contained in or attached to this message is
> STRICTLY PROHIBITED. If you have received this transmission in error,
> please immediately notify the sender and delete the e-mail and attached
> documents. Thank you.
> ***********************************************************************
>
More information about the hackers
mailing list