[ntp:security] Authorized (not nopeer) IPs can create server associations with certain query

Harlan Stenn stenn at nwtime.org
Fri Dec 8 23:37:41 UTC 2017



On 12/8/17 8:44 AM, Matt Nordhoff wrote:
> On Fri, Dec 8, 2017 at 11:03 AM, Harlan Stenn <stenn at nwtime.org> wrote:
>> Hi Matt,
>>
>> I think this is a reported problem, and if so, it's scheduled to be
>> fixed for 4.2.8p11.
> 
> Alright. Great. :-)
> 
>> It looks like I'm the one who will fix it, and the underlying issue is
>> intricate, and we need to be very careful and deliberate on exactly how
>> it is fixed.  If we don't go "far enough" there's still a problem.  If
>> we go "too far" we prohibit desirable behavior.
> 
> I'm sorry. Good luck. :-(
> 
>> I was hoping to release ntp-4.2.8p11 on Tuesday 8 Jan, but with the
>> holidays approaching, I don't know that we'll have it ready for our
>> institutional members to be able to test and integrate it by then.  So
>> it may be an additional week (the 15th), possibly two (the 22nd), before
>> it is released.
> 
> Thank you for explaining. January sounds pretty good to me.
> 
> (Happy holidays!)
> 
>> Does NTF have any paperwork with you?  If we do, we might be able to
>> arrange early access to the patch.  That patch must be kept confidential
>> until the pubic release (ie, not shared with anybody).
> 
> No, no paperwork. I don't want to sound too cavalier, but as far as i
> know, it seems to be a medium severity issue for NTP, not very
> amenable to non-targeted, massive exploitation. (Of course a client
> can try to map a server's sys peers, but still.) I'm not that
> concerned. Don't worry about going out of your way.

If you'd like early access to the patch, we'd just need to get you a
volunteer agreement, which has an NDA in it.

On the one hand, this is silly.  On the other hand, we're also dealing
with big companies who at least go thru the motions of "due diligence"
checks and often have internal compliance audits, and if we can't check
off their boxes they don't work with is.

>> H
> 
> For what it's worth, if i had a personal security disclosure policy --
> and i don't yet -- i would probably copy Project Zero, which would put
> the deadline in February. (And it would be unfair to jump you with
> something else later after writing that sentence.) So i won't complain
> about releasing in January.

Thanks, and it's extra fun for us as we're so badly under-resourced.

> For reference, NTPsec has a 45 day disclosure policy, or earlier or
> later with "[e]xtenuating circumstances", and i first emailed them 45
> days ago today. I can't speculate, but it seems they have the
> potential to do anything at any time. (Which would ultimately be my
> responsibility, at one remove.) :-/

I'm not going to get started on NTPsec.  Much.  But I will say that
their "Project Manager" has publicly stated that one difference between
us and them is that we treat NTP as a carrier-class product, doing
significant testing and engineering on it, while they treat it as an
ordinary open source project.  If they do something that breaks, they
just fix it in the next release.  They can also "get away with this" as
they used our code as the base for their code.

H
> Matt
> 
>> On 12/4/17 7:29 AM, Matt Nordhoff wrote:
>>> Hi again,
>>>
>>> Do you have any news on this?
>>>
>>> If so, will you share it? I'm terribly curious what the bug actually is. :-P
>>>
>>> On Sun, Nov 12, 2017 at 9:23 PM, Harlan Stenn <stenn at nwtime.org> wrote:
>>>> Hi Matt,
>>>>
>>>> NTPsec doesn’t communicate with us.
>>>>
>>>> Thanks for contacting us. We’ll dig.
>>>>
>>>> Sent from my iPhone
>>>>
>>>>> On Nov 12, 2017, at 2:56 AM, Matt Nordhoff <mnordhoff at mn0.us> wrote:
>>>>>
>>>>> Hi,
>>>>>
>>>>> To be honest, I first noticed this issue on a Pool server running
>>>>> NTPsec. (I'm sorry.) I have since reproduced it -- partly -- on NTP
>>>>> 4.2.8p10 by replaying a packet.
>>>>>
>>>>> I first emailed security at ntpsec.org 2017-10-24. I first sent them
>>>>> pcaps 2017-11-08. I don't know if they have contacted you. They
>>>>> haven't given me notable information at this time.
>>>>>
>>>>> I have a stratum 2 server running Ubuntu 16.04 and NTP 4.2.8p10.
>>>>>
>>>>> Certain weird packets from certain clients can cause ntpd to create
>>>>> some sort of preemptable server association.
>>>>>
>>>>> I'm not sure what's happening. Maybe something weird with manycast
>>>>> mode, i don't know. I'm not certain it's exploitable, but i think it
>>>>> is.
>>>>>
>>>>> (I wonder if restrict notrust would help?)
>>>>>
>>>>> In NTP, restrict nopeer apparently usually stops anything from
>>>>> happening: I've only successfully caused associations to be created
>>>>> with "client" IPs that are currently or previously configured servers
>>>>> and already whitelisted by restrict source.
>>>>>
>>>>> (In NTPsec, any IP can do it!)
>>>>>
>>>>> If the server has loose restricts, or an attacker can spoof their
>>>>> source IP and knows what servers you're using, it can presumably be
>>>>> exploited, but it would obviously be harder.
>>>>>
>>>>> Notably, a client can create multiple associations.
>>>>>
>>>>> As an example, i believe a single Pool server -- especially one a
>>>>> running a customized NTP server -- could create numerous associations
>>>>> and obtain undue influence overs its clients' clocks.
>>>>>
>>>>> In some cases the packets seem to trigger ntpd's 0 origin checks, but
>>>>> other times they seem to work. (Unconfirmed speculation: Maybe the
>>>>> packet creates 1 association, but also causes 1 packet from another
>>>>> existing association for the same IP to be rejected.)
>>>>>
>>>>> In the real world, my Pool server in the Brazil zone gets this traffic
>>>>> from a number of seemingly ordinary clients. (Reverse DNS looks like
>>>>> random consumer ISP addresses.) They use ephemeral ports that aren't
>>>>> running actual NTP servers, so the association is totally unusable and
>>>>> is eventually removed. (Again, only NTPsec or [presumably] NTP with
>>>>> "restrict default" without "nopeer" create associations at all.)
>>>>>
>>>>> pcap of a single packet from a Brazilian client:
>>>>>
>>>>> <https://mn0.us/71WFPp6JPy43QEdDEAqihb1/ntp7_2017-11-12_03:00:22_186.233.180.198_ntp.pcap>
>>>>> (130 bytes)
>>>>>
>>>>> $ tcpdump -nttttvxr ntp7_2017-11-12_03:00:22_186.233.180.198_ntp.pcap
>>>>> reading from file ntp7_2017-11-12_03:00:22_186.233.180.198_ntp.pcap,
>>>>> link-type EN10MB (Ethernet)
>>>>> 2017-11-12 03:00:22.676707 IP (tos 0x0, ttl 113, id 2423, offset 0,
>>>>> flags [none], proto UDP (17), length 76)
>>>>>    186.233.180.198.64094 > 45.33.103.94.123: NTPv1, length 48
>>>>>        Server, Leap indicator:  (0), Stratum 3 (secondary reference),
>>>>> poll 3 (8s), precision -23
>>>>>        Root Delay: 0.360198, Root dispersion: 0.099517, Reference-ID:
>>>>> 54.206.109.187
>>>>>          Reference Timestamp:  3719442935.600897507 (2017/11/12 02:35:35)
>>>>>          Originator Timestamp: 0.000000000
>>>>>          Receive Timestamp:    3719444422.526398753 (2017/11/12 03:00:22)
>>>>>          Transmit Timestamp:   3719444422.526416904 (2017/11/12 03:00:22)
>>>>>            Originator - Receive Timestamp:  3719444422.526398753
>>>>> (2017/11/12 03:00:22)
>>>>>            Originator - Transmit Timestamp: 3719444422.526416904
>>>>> (2017/11/12 03:00:22)
>>>>>        0x0000:  4500 004c 0977 0000 7111 3bfb bae9 b4c6
>>>>>        0x0010:  2d21 675e fa5e 007b 0038 355a 0c03 03e9
>>>>>        0x0020:  0000 5c36 0000 197a 36ce 6dbb ddb2 31f7
>>>>>        0x0030:  99d4 6b48 0000 0000 0000 0000 ddb2 37c6
>>>>>        0x0040:  86c2 1199 ddb2 37c6 86c3 421c
>>>>>
>>>>> ntp.conf (with comments modified):
>>>>>
>>>>> <https://mn0.us/DXGMaQRVotwGnZwzXZE6p5x/ntp.conf> (3 KiB)
>>>>>
>>>>> Stuff from my NTP server after using Scapy to send it a number of
>>>>> packets with that payload:
>>>>>
>>>>> mnordhoff at clover:~$ date && ntpq -c lpeers
>>>>> Sun Nov 12 10:32:13 UTC 2017
>>>>>     remote           refid      st t when poll reach   delay   offset  jitter
>>>>> ==============================================================================
>>>>> ntp-pool        .POOL.          16 p    -   64    0    0.000    0.000   0.000
>>>>> time.nist.gov   .POOL.          16 p    -   64    0    0.000    0.000   0.000
>>>>> ntp.ubuntu.com  .POOL.          16 p    -   64    0    0.000    0.000   0.000
>>>>> 2.us.pool.ntp.o .POOL.          16 p    -   64    0    0.000    0.000   0.000
>>>>> +six0.ntp3.mattn 128.59.0.245     2 u  849 1024  377   18.578   -0.515   0.477
>>>>> #2604:a880:400:d 130.183.99.210   3 u  437 1024  377   41.530   -0.126   0.329
>>>>> six1.ntp5.mattn .STEP.          16 u    - 1024    0    0.000    0.000   0.000
>>>>> +ec2-54-243-186- 45.79.187.10     3 u   76 1024  377   31.163   -0.455   0.443
>>>>> #six0.ntp7.mattn 35.73.197.144    2 u 113m 1024  300   18.627    0.142   0.068
>>>>> #2600:1f16:ec6:e 209.51.161.238   2 u 1025 1024  377   34.927   -0.010   0.224
>>>>> -tick.uh.edu     .GPS.            1 u 1037 1024  347   10.325   -1.544   0.432
>>>>> *clock.fmt.he.ne .CDMA.           1 u  552 1024  377   37.490   -0.300   0.279
>>>>> #bedast01.beaust 129.7.1.66       2 u  345 1024  377    1.343    1.487   1.195
>>>>> +awesome.bytesta 216.218.254.202  2 u  265 1024  377    1.317   -0.439   0.370
>>>>> #ntp.jtsage.com  127.67.113.92 2 u  862 1024  377    1.068    0.012   0.375
>>>>> #dev.smatwebdesi 192.168.204.60   3 u  175 1024  377    1.251    0.766   0.958
>>>>> #serenity.melanc 129.7.1.66       2 u  732 1024  377    0.332    0.360   0.440
>>>>> #ntp.quintex.com .CDMA.           1 u  436 1024  377   39.356    0.112   0.507
>>>>> +six0.ntp7.mattn 35.73.197.144    2 u  184 1024  377   18.603   -0.390   0.345
>>>>> +six0.ntp7.mattn 35.73.197.144    2 u  210 1024  377   21.952   -0.190   0.237
>>>>> #four0.jane.matt 35.73.197.144    2 u   39   64  177   21.861   -0.270   0.292
>>>>>
>>>>> (The last 3 associations were created by Scapy.)
>>>>>
>>>>> mnordhoff at clover:~$ date && ntpq -c "rv &9"
>>>>> Sun Nov 12 10:32:59 UTC 2017
>>>>> associd=59303 status=951a conf, reach, sel_backup, 1 event, sys_peer,
>>>>> srcadr=six0.ntp7.mattnordhoff.net, srcport=123,
>>>>> dstadr=2600:3c00:e000:15a::, dstport=123, leap=00, stratum=2,
>>>>> precision=-22, rootdelay=1.434, rootdisp=21.133, refid=35.73.197.144,
>>>>> reftime=ddb28603.2721672c  Sun, Nov 12 2017  8:34:11.152,
>>>>> rec=ddb2874b.b2db92da  Sun, Nov 12 2017  8:39:39.698, reach=300,
>>>>> unreach=0, hmode=3, pmode=4, hpoll=10, ppoll=10, headway=0, flash=00 ok,
>>>>> keyid=0, offset=0.142, delay=18.627, dispersion=19.100, jitter=0.068,
>>>>> xleave=0.170,
>>>>> filtdelay=    18.77   18.63   18.71   18.61   18.58   18.78   18.70   18.64,
>>>>> filtoffset=    0.07    0.14    0.07    0.07    0.06    0.23    0.15    0.09,
>>>>> filtdisp=      0.00   15.74   31.43   47.25   62.73   78.21   94.31  110.36
>>>>> mnordhoff at clover:~$ date && ntpq -c "rv &19"
>>>>> Sun Nov 12 10:33:08 UTC 2017
>>>>> associd=59325 status=1314 reach, sel_outlier, 1 event, reachable,
>>>>> srcadr=six0.ntp7.mattnordhoff.net, srcport=123,
>>>>> dstadr=2600:3c00:e000:15a::, dstport=123, leap=00, stratum=2,
>>>>> precision=-22, rootdelay=1.312, rootdisp=39.841, refid=35.73.197.144,
>>>>> reftime=ddb29aa8.271ad31b  Sun, Nov 12 2017 10:02:16.152,
>>>>> rec=ddb2a0f5.b2cd5cb2  Sun, Nov 12 2017 10:29:09.698, reach=377,
>>>>> unreach=1, hmode=3, pmode=4, hpoll=10, ppoll=10, headway=0, flash=00 ok,
>>>>> keyid=0, offset=-0.390, delay=18.603, dispersion=19.365, jitter=0.345,
>>>>> xleave=0.174,
>>>>> filtdelay=    18.65   18.60   18.70   18.70   18.59   18.59   18.78   18.72,
>>>>> filtoffset=   -0.29   -0.39   -0.40   -0.08   -0.01   -0.03    0.09    0.08,
>>>>> filtdisp=      0.00   16.22   31.73   47.55   63.08   78.90   94.83   95.84
>>>>> mnordhoff at clover:~$ date && ntpq -c "rv &20"
>>>>> Sun Nov 12 10:33:21 UTC 2017
>>>>> associd=59326 status=1314 reach, sel_outlier, 1 event, reachable,
>>>>> srcadr=six0.ntp7.mattnordhoff.net, srcport=123, dstadr=2600:3c00::2:b401,
>>>>> dstport=123, leap=00, stratum=2, precision=-22, rootdelay=1.312,
>>>>> rootdisp=39.444, refid=35.73.197.144,
>>>>> reftime=ddb29aa8.271ad31b  Sun, Nov 12 2017 10:02:16.152,
>>>>> rec=ddb2a0db.b3a3de58  Sun, Nov 12 2017 10:28:43.701, reach=377,
>>>>> unreach=1, hmode=3, pmode=4, hpoll=10, ppoll=10, headway=0, flash=00 ok,
>>>>> keyid=0, offset=-0.190, delay=21.952, dispersion=15.056, jitter=0.237,
>>>>> xleave=0.120,
>>>>> filtdelay=    21.95   22.05   21.95   21.92   21.94   21.88   22.04   21.97,
>>>>> filtoffset=   -0.19   -0.38   -0.33    0.04    0.10    0.01    0.12    0.06,
>>>>> filtdisp=      0.00   15.62   30.98   47.24   63.42   78.98   94.52   95.52
>>>>> mnordhoff at clover:~$ date && ntpq -c "rv &21"
>>>>> Sun Nov 12 10:33:22 UTC 2017
>>>>> associd=59327 status=1514 reach, sel_backup, 1 event, reachable,
>>>>> srcadr=four0.jane.mattnordhoff.net, srcport=123, dstadr=45.79.1.70,
>>>>> dstport=123, leap=00, stratum=2, precision=-22, rootdelay=1.312,
>>>>> rootdisp=34.927, refid=35.73.197.144,
>>>>> reftime=ddb29aa8.271ad31b  Sun, Nov 12 2017 10:02:16.152,
>>>>> rec=ddb2a186.87cee83c  Sun, Nov 12 2017 10:31:34.530, reach=177,
>>>>> unreach=7, hmode=3, pmode=4, hpoll=10, ppoll=6, headway=0,
>>>>> flash=01 pkt_dup, keyid=0, offset=-0.270, delay=21.861,
>>>>> dispersion=77.059, jitter=0.292, xleave=0.139,
>>>>> filtdelay=    21.86   21.92   21.84   21.82   21.88   21.94   21.82    0.00,
>>>>> filtoffset=   -0.27   -0.38   -0.23    0.08    0.05    0.11    0.09    0.00,
>>>>> filtdisp=      0.00   15.92   32.18   48.24   64.22   65.19   66.20 16000.0
>>>>>
>>>>> NTP's syslog messages:
>>>>>
>>>>> Nov 12 07:55:54 clover ntpd[4737]: 45.33.103.94 local addr 45.79.1.70 -> <null>
>>>>> Nov 12 08:00:14 clover ntpd[4737]: 45.33.103.94 local addr 45.79.1.70 -> <null>
>>>>> Nov 12 08:10:48 clover ntpd[4737]: 2600:3c02::13:5230 local addr
>>>>> 2600:3c00::2:b401 -> 2600:3c00:e000:15a::
>>>>> Nov 12 08:11:31 clover ntpd[4737]: 45.33.103.94 local addr 45.79.1.70 -> <null>
>>>>> Nov 12 08:19:45 clover ntpd[4737]: 2600:3c02::13:5230 local addr
>>>>> 2600:3c00:e000:15a:: -> <null>
>>>>> Nov 12 08:37:57 clover ntpd[4737]: 2600:3c02::13:5230 local addr
>>>>> 2600:3c00::2:b401 -> 2600:3c00:e000:15a::
>>>>> Nov 12 08:38:02 clover ntpd[4737]: 2600:3c02::13:5230 local addr
>>>>> 2600:3c00::2:b401 -> 2600:3c00:e000:15a::
>>>>> Nov 12 08:38:07 clover ntpd[4737]: 2600:3c02::13:5230 local addr
>>>>> 2600:3c00::2:b401 -> 2600:3c00:e000:15a::
>>>>> Nov 12 08:41:28 clover ntpd[4737]: 2600:3c02::13:5230 local addr
>>>>> 2600:3c00::2:b401 -> 2600:3c00:e000:15a::
>>>>> Nov 12 08:41:29 clover ntpd[4737]: receive: Drop 0 origin timestamp
>>>>> from server at 2600:3c02::13:5230 xmt 0xddb237c6.86c3421c
>>>>> Nov 12 08:41:30 clover ntpd[4737]: receive: Drop 0 origin timestamp
>>>>> from server at 2600:3c02::13:5230 xmt 0xddb237c6.86c3421c
>>>>> Nov 12 08:47:05 clover ntpd[4737]: 2600:3c02::13:5230 local addr
>>>>> 2600:3c00:e000:15a:: -> <null>
>>>>> Nov 12 08:47:47 clover ntpd[4737]: 2600:3c02::13:5230 local addr
>>>>> 2600:3c00:e000:15a:: -> <null>
>>>>> Nov 12 08:47:50 clover ntpd[4737]: 2600:3c02::13:5230 local addr
>>>>> 2600:3c00:e000:15a:: -> <null>
>>>>> Nov 12 08:49:11 clover ntpd[4737]: receive: Drop 0 origin timestamp
>>>>> from server at 2600:3c02::13:5230 xmt 0xddb237c6.86c3421c
>>>>> Nov 12 08:57:27 clover ntpd[4737]: receive: Unexpected origin
>>>>> timestamp 0xddb28b77.ae070b4f does not match aorg 0000000000.00000000
>>>>> from server at 2600:3c02::13:5230 xmt 0xddb28b77.b06f9d59
>>>>> Nov 12 09:10:10 clover ntpd[4737]: receive: Drop 0 origin timestamp
>>>>> from server at 2600:3c02::13:5230 xmt 0xddb237c6.86c3421c
>>>>> Nov 12 09:14:48 clover ntpd[4737]: receive: Unexpected origin
>>>>> timestamp 0xddb28f88.ae02ddad does not match aorg 0000000000.00000000
>>>>> from server at 2600:3c02::13:5230 xmt 0xddb28f88.b074ed27
>>>>> Nov 12 09:32:42 clover ntpd[4737]: receive: Unexpected origin
>>>>> timestamp 0xddb293ba.ae02ed68 does not match aorg 0000000000.00000000
>>>>> from server at 2600:3c02::13:5230 xmt 0xddb293ba.b079bf05
>>>>> Nov 12 09:50:16 clover ntpd[4737]: receive: Unexpected origin
>>>>> timestamp 0xddb297d8.ae02452d does not match aorg 0000000000.00000000
>>>>> from server at 2600:3c02::13:5230 xmt 0xddb297d8.b058211b
>>>>> Nov 12 10:07:53 clover ntpd[4737]: receive: Unexpected origin
>>>>> timestamp 0xddb29bf9.ae08e00f does not match aorg 0000000000.00000000
>>>>> from server at 2600:3c02::13:5230 xmt 0xddb29bf9.b05331aa
>>>>> Nov 12 10:25:32 clover ntpd[4737]: receive: Unexpected origin
>>>>> timestamp 0xddb2a01c.ae02b238 does not match aorg 0000000000.00000000
>>>>> from server at 2600:3c02::13:5230 xmt 0xddb2a01c.b05f2c19
>>>>> Nov 12 10:29:50 clover ntpd[4737]: 45.33.103.94 local addr 45.79.1.70 -> <null>
>>>>> Nov 12 10:31:34 clover ntpd[4737]: receive: Drop 0 origin timestamp
>>>>> from server at 45.33.103.94 xmt 0xddb237c6.86c3421c
>>>>> Nov 12 10:42:54 clover ntpd[4737]: receive: Unexpected origin
>>>>> timestamp 0xddb2a42e.ae0295d4 does not match aorg 0000000000.00000000
>>>>> from server at 2600:3c02::13:5230 xmt 0xddb2a42e.b05ac12c
>>>>>
>>>>> pcaps I've sent NTPsec:
>>>>>
>>>>> <https://mn0.us/g82RmQ8uVbWFpEGoywoJdk/ntp7_2017-11-08_09:35_143.137.65.13_ntp.pcap>
>>>>> (9 KiB)
>>>>>
>>>>> <https://mn0.us/L4Hpt2fbGtnGYr1GnKLY7cj/ntp7_2017-11-08_10_187.1.57.195_ntp.pcap>
>>>>> (66 KiB)
>>>>>
>>>>> <https://mn0.us/rD9DouTTX97LidW49gHvFdf/ntp7_2017-11-08_15_200.199.238.226_ntp.pcap>
>>>>> (1 KiB)
>>>>>
>>>>> Some of them include traffic from my NTPsec ntpd, and the first one
>>>>> may be incomplete because I ran out of disk space for a few minutes.
>>>>> (That was fun.)
>>>>>
>>>>> Cheers :-/

-- 
Harlan Stenn <stenn at nwtime.org>
http://networktimefoundation.org - be a member!


More information about the security mailing list