[ntp:questions] Re: "Listen on" semantics

Danny Mayer mayer at ntp.isc.org
Fri Sep 22 02:57:32 UTC 2006


Luc Pardon wrote:
> Danny Mayer wrote:
>> Luc Pardon wrote:
>>> Danny Mayer wrote:
>>>> By binding to all interfaces and addresses we
>>>> prevent other applications using them. The worst thing you can have is
>>>> two different applications modifying the clock.
>>>    This is, I think, where we have fundamentally different perspectives.
>>>
>>>    To you, the worst thing may well be a screwed-up clock (and
>>> incidentally that's a perfect mindset for a clock-daemon developer <g>).
>>> To me as an administrator, however, the Worst Thing is having the box
>>> hacked and rooted.
>>>
>>
>> At which point your log timestamps better be accurate so you can trace
>> things back. This is *NOT* from a developer's viewpoint.
>>
>     Yes, when I get rooted it is important to find out how, and having
> accurate time does help, although we could argue about the need for
> absolute versus relative time. But that would get off topic.

You're talking to someone who did his thesis in General Relativity.
There's no such thing as absolute time. All time is relative. Not that
this is relevant!

> 
>     My point was that I'd prefer a hacker-proof fortified castle (I
> know: "pull the plug!") with less accurate time to a open house where I
> can determine with millisecond precision when exactly I got rooted.
> 
>    Of course I want both the super-castle _and_ the super-clock <g>.
> 
>>>    Your reasoning makes sense, but you are reasoning from the assumption
>>> that there are no bugs in the code. Nobody can guarantee that, and
>>> certainly not an application with +73k lines of code (wc -l
>>> ntp-dev-4.2.3p39/ntpd/*.c). That's simply impossible.
>>>
>>
>> No, there *ARE* bugs in the code, that's not even speculation, but
>> that's not in any way related to the question at hand.
> 
>     Quite to the contrary, it is. If there are bugs, it's dangerous to
> take candy from strangers, as (x)ntpd does.
> 

It's dangerous to cross the street but we do it every day. If you're
driving you should be wearing a seatbelt and have airbags in your vehicle.

>>>    Your reasoning also assumes that every incoming packet is playing by
>>> the rules, or at least that the code can handle all kinds of malformed
>>> packets, that it will never cut its virtual fingers on the sharp edges
>>> of packets that were crafted on purpose by malicious minds. That is a
>>> dangerous assumption in today's world.
>>
>> No, the code actual spends a lot of time checking the packet that comes
>> in before it uses the contents. malformed packets will be immediately
>> rejected. The only other thing you can do to an NTP packet is put in bad
>> timestamps, which if it is incoming to a client will be dismissed as way
>> out of range. The only other parts in an NTP packet deals with security
>> but you won't get far with that.
>>
>> Are there other ways of doing something nasty to a packet? Undoubtedly
>> there are things we haven't thought of but there's not a lot in the
>> packet that you can change in the first place. None of this has anything
>> to do with binding to all interface addresses.
>>
> 
>    Again, your reasoning is valid only if there are no bugs. There have
> been boatloads of exploits, precisely because of bugs in UDP packet
> processing code, many of them precisely in the code that was intended to
> check that the packet is valid. Because all code can be assumed to
> contain bugs, all code must be assumed to be potentially vulnerable.
> 
>    Case in point #1: back in 2001, there was a bug in - yes - (x)ntpd
> that allowed remote root access. See, for example:
> 
> http://archive.cert.uni-stuttgart.de/archive/bugtraq/2001/04/msg00064.html
> 
>    If (x)ntpd had not listened and accepted and processed all junk that
> comes in, it would not have happened.
> 

Yes, and not only was it as is it still the only known ntp exploit, it's
how I got involved in the first place. The bug was fixed by someone I
happened to be working with at the time but I didn't know that until
several years later when I was looking at the code.

>    Case in point #2: only last week, my logs were being flooded because
> somebody sent icmp port unreachable packets to udp/123. Each packet is
> good for about 80 bytes of wasted disk space.

icmp doesn't use port numbers, it doesn't know what they are.

> A determined attacker,
> starting on Friday evening, could use a high-speed line to fill up a
> multi-gigabyte disk and have free game by Sunday afternoon. By that time
> none of his actions will be logged anymore because of disk full. 

That's a DDOS type of attack and is best prevented or at least managed
by a firewall router. So far this is not an ntp issue whether or not
it's listening on any particular address.

By
> Monday morning, the sysadmin will scratch his head over the "connection
> refused"'s and may not even know he's been hacked.

Connection refused by what?

> 
>    If (x)ntpd had not listened and accepted and processed all junk that
> comes in, this would not be possible.
> 

On the contrary, the attacker can use any service running on the system
for this. There's nothing special about ntp here.
> 
>>>    Yes, the code path may be much shorter if you drop the packets right
>>> away. But it takes only one typo to create an exploitable vulnerability.
>>> If not today, then maybe tomorrow, after the next maintenance cycle.
>>
>> Very true, but what has this to do with binding to all interface
>> addresses?
> 
>    If you accept packets, there is a potential vulnerability. (x)ntp
> unnecessarily accepts anything, junk and malicious packets included.
> 

It will drop malformed and invalid packets almost immediately.

>    If you'll allow me I'll give you an analogy (crippled like all of
> them). I am building a fortified castle and I'm letting you live in it.
> What you do is hanging a rope out of your window (of room 123 <g>) for
> the postman to climb in and bring you the packets that you order from a
> mail order company.
> 
>    When I object that my enemies can come in as easily as the mailman,
> you say: "no worry, I'll fend them off, and besides, I need to hang out
> that rope to protect you from other problems, and besides, there is no
> other way to get at my packets".
> 
>    Now:
> 
>    * we both know that you can't guarantee me that you'll fend them off
> (bugs, new tricks, ...).
>    * I ask you to let me decide if I want to be protected from other
> problems or not.
>    * Finally, there _is_ another way to get at your packets, see below.
> 

Well no, you have to have them come in the same way and examine them to
see if you want them.

>   Because I'm the landlord, you're going to pull your rope back in or
> convince me it's needed. If you fail to do either, I'll kick you out <g>.
> 
>>>    Besides, the -I switch won't help me with the most vulnerable
>>> interface on a two-NIC box acting as a stratum 3 time server for an
>>> internal network. I tried to disable the public interface (by specifying
>>> -I <internal interface>) but then it can't seem to reach the stratum 2
>>> servers outside. The net result is that I have no choice but to let any
>>> "packet from hell" make its way through the "restrict" processing.
>>>
>>
>> This does you no good, either way. 
> 
>     That sounds a little bit like we're talking different things ??
> 

I don't think so. The packet only gets to the restrict processing if it
is a valid well-formed packet, it gets dropped otherwise.

>> The *ONLY* solution here is to use
>> the restrict processing 
> 
>     No, restrict rules, while helpful, are not a solution. First, they
> may have bugs themselves, and second, they are based on IP addresses,
> which are easily spoofed with UDP. If I know where your IP is located,
> it's a fair guess what time servers you'll be using. I can send you a
> few dozen packets with various faked sender IP's until one sails right
> through your restrict rules and into the processing code, where
> potentially more bugs are waiting to be exploited.

But at this point not only must the packet be well-formed but if the
timestamps are off the server the it purports to come from will get
ignored. The clock routines that lie at the heart of ntp will drop any
server which is way out of synch with other servers.It's much harder for
you to know what servers to spoof and send packets allegedly coming from
them in a way that would cause ntp to accept them as valid and be close
enough to the correct value to be acceptable. If you are concerned
enough about the issue then you can add authentication to your
configuration file and it will drop packets that are not authenticated.

> 
>> since you *MUST* accept packets on both NIC's.
> 
>     Yes but no <g>. Obviously, I must accept responses from time servers
> that answer a request of mine, else no sync. But there is no reason to
> accept anything else.

Then you should install a stateful firewall and restrict packets as
appropriate.

> 
>> This is even if we had the ability to allow you to select which
>> interface addressess to bind to you'd still have the same problem.
>>
>> If you are suggesting that you want to run two copies of ntpd on your
>> server then you have a bigger problem since ntpd is not designed to work
>> that way and it would be hard to support such a situation.
> 
>    What I want is not so much two copies of ntpd as a separation between
> client and server functionality.
> 
>    The client should keep my clock on track. The server should tell all
> my other systems what time it is.

And you can do that via the restrict statements.

> 
>    The server should listen for incoming requests all right, but only on
> the interfaces that I specify. That interface would be the one connected
> to the internal network only. After all, that's the only one where valid
> queries may come in. Yes, that would leave me vulnerable to attacks from
> within, but I can deal with that - if need be with a wooden pole <g>.
> 
>    The client, on the other hand, should not be listening all the time,
> otherwise it's not a client but a server. 


You misunderstand how this works. The client only listens to reponses to
queries that it makes, it doesn't need to respond to any queries. Again
that's what the restrict statement is for.

When it's time to ask public
> server ntp1.example.com what time it is, the client should create a
> socket, fill in the sockaddr with the IP of ntp1.example.com, send the
> packet down the socket, receive the reply from the socket, then close
> the socket again. Yes, until the reply comes in, the client _will_ be
> listening, but with three important distinctions:  a) it will see only
> packets that (pretend to) come from ntp1.example.com,

Use authentication to validate this.

 b) it will not be
> listening on port 123 but on a random port assigned by the OS

There you want something like sntp.

 and c) the
> socket will be gone as soon as the reply is in.

Yes you could do that.

 An attacker would 1)
> have to guess the IP of the server I queried and fake it, 2) he would
> have to guess the random port number that I'm listening on, and 3) he
> would have to make the fake packet arrive after the request but ahead of
> the real response. Still vulnerable, but much harder to exploit.
> 

If you use authentication it doesn't matter. sntp doesn't support it, I
believe, you need the full ntpd daemon.

>    The problem is that ntpd currently does not seem to use client mode,
> or at least does it in asynchroneous mode. If it did things in the above
> way:
> 
>    * it would not have to do the "rebind" trick to detect changes of
> dynamic IP's
> 

If the server address is static it's not needed.

>    * it would have no problems dealing with multi-IP servers
> 

It doesn't now.

>    * it would not have to listen on - and handle every single packet
> from - the public interface
> 

True but so what? It's not as if it takes a lot of CPU time.

>>>    In any case, the simple fact is: as soon as you accept packets, you
>>> are posing a security risk.
>>>
>> Well the alternative to that is not to connect to anything.
>>
> 
>    There is an alternative in between, and that is to connect to the
> outside servers in the regular client/server way, as described above.
> 

People do it today with ntpdate.

>>>    Of course you are right that, if I run two different applications
>>> that modify the clock, I may have a problem, but at least it's a problem
>>> of my own making. If you really want to protect me from shooting myself
>>> in the foot by locking all unused interfaces, make it optional. Make it
>>> the default if you want, but at least give me the option to disable it.
>>
>> You have that option today by modifying the code to your liking. Not the
>> answer you want but it's even more fraught with potential security
>> issues that I wouldn't want to support such a configuration.
> 
>    You're probably going to kill me for this <g> but there is another
> option: I can switch to OpenNTPD. That does - or seems to do - precisely
> what I want. With that setup, things like the exploit that bit (x)ntpd
> in 2001, or the "connection refused" issue that cost me so much time to
> diagnose, are simply impossible.
> 

You are of course welcome to use if it you want sntp and you don't care
about clock accuracy, lack of support for things like ntpq, etc. for
check the state of your ntpd.

>    My only requirement is that all the clocks of my internal machines
> are in reasonable sync. I'm not running a Space Station, so as long as
> all the machines are off in the same way, I don't mind if their time is
> a little off the absolute time of the Universe.
> 

As I said before there is no absolute time.

>   What I'm testing out right now is OpenNTPD on the internal stratum 3
> server with the internal clients running (x)ntpd. It's been running for
> a week or so. Up to now it seems to do what I want.
> 
>    That means I have to trade off a millisecond-accurate clock against
> more security. I'm afraid that is pretty much a no-brainer.
> 

You mean less. You can easily fake out packets going to an OpenNTP
server and you would have no way of knowing that. At least with ntpd you
can authenticate the server packets and know that they are valid and use
them.


>    Well, that's why I gave it another try. But please keep in mind that
> I have no stake in this discussion, I have an alternative (see above).
> 
>    Unless I am mistaken, what is at stake for you is no less than a
> redesign of ntpd, or at least of the socket handling code.

I need a reason to do so. I did it once and now it's maintainable. It
wasn't back then. None of what we are discussing (like not listening at
all to certain IP addresses or interfaces is a big challenge any more. I
made sure of that.

> I don't
> expect that to happen and I'm not willing to "fight" to make it happen.
> That is why I proposed to leave it at that.
> 

It's easy enough to change the code if you want to do so, but I won't do
it without good well thought out reasons that will not make it less secure.

Danny
>     Luc
> 




More information about the questions mailing list