[ntp:security] Authorized (not nopeer) IPs can create server associations with certain query

Matt Nordhoff mnordhoff at mn0.us
Sat Dec 9 22:13:12 UTC 2017


On Fri, Dec 8, 2017 at 11:37 PM, Harlan Stenn <stenn at nwtime.org> wrote:
> On 12/8/17 8:44 AM, Matt Nordhoff wrote:
>> On Fri, Dec 8, 2017 at 11:03 AM, Harlan Stenn <stenn at nwtime.org> wrote:
>>> Hi Matt,
>>>
>>> I think this is a reported problem, and if so, it's scheduled to be
>>> fixed for 4.2.8p11.
>>
>> Alright. Great. :-)
>>
>>> It looks like I'm the one who will fix it, and the underlying issue is
>>> intricate, and we need to be very careful and deliberate on exactly how
>>> it is fixed.  If we don't go "far enough" there's still a problem.  If
>>> we go "too far" we prohibit desirable behavior.
>>
>> I'm sorry. Good luck. :-(
>>
>>> I was hoping to release ntp-4.2.8p11 on Tuesday 8 Jan, but with the
>>> holidays approaching, I don't know that we'll have it ready for our
>>> institutional members to be able to test and integrate it by then.  So
>>> it may be an additional week (the 15th), possibly two (the 22nd), before
>>> it is released.
>>
>> Thank you for explaining. January sounds pretty good to me.
>>
>> (Happy holidays!)
>>
>>> Does NTF have any paperwork with you?  If we do, we might be able to
>>> arrange early access to the patch.  That patch must be kept confidential
>>> until the pubic release (ie, not shared with anybody).
>>
>> No, no paperwork. I don't want to sound too cavalier, but as far as i
>> know, it seems to be a medium severity issue for NTP, not very
>> amenable to non-targeted, massive exploitation. (Of course a client
>> can try to map a server's sys peers, but still.) I'm not that
>> concerned. Don't worry about going out of your way.
>
> If you'd like early access to the patch, we'd just need to get you a
> volunteer agreement, which has an NDA in it.
>
> On the one hand, this is silly.  On the other hand, we're also dealing
> with big companies who at least go thru the motions of "due diligence"
> checks and often have internal compliance audits, and if we can't check
> off their boxes they don't work with is.

Yeah, I understand. I'm curious what the steps to do a volunteer
agreement are -- cheque? fax? gpg? passport? -- and maybe i'll think
about it but...

It's simpler to save both of us the inconvenience and say no now. :-)

Unless i wake up and change my mind later, no early patch access, and
no volunteer agreement, is fine.

>>> H
>>
>> For what it's worth, if i had a personal security disclosure policy --
>> and i don't yet -- i would probably copy Project Zero, which would put
>> the deadline in February. (And it would be unfair to jump you with
>> something else later after writing that sentence.) So i won't complain
>> about releasing in January.
>
> Thanks, and it's extra fun for us as we're so badly under-resourced.
>
>> For reference, NTPsec has a 45 day disclosure policy, or earlier or
>> later with "[e]xtenuating circumstances", and i first emailed them 45
>> days ago today. I can't speculate, but it seems they have the
>> potential to do anything at any time. (Which would ultimately be my
>> responsibility, at one remove.) :-/
>
> I'm not going to get started on NTPsec.  Much.  But I will say that
> their "Project Manager" has publicly stated that one difference between
> us and them is that we treat NTP as a carrier-class product, doing
> significant testing and engineering on it, while they treat it as an
> ordinary open source project.  If they do something that breaks, they
> just fix it in the next release.  They can also "get away with this" as
> they used our code as the base for their code.

:-X

My curiosity exceeds my good sense.

> H
>> Matt
>>
>>> On 12/4/17 7:29 AM, Matt Nordhoff wrote:
>>>> Hi again,
>>>>
>>>> Do you have any news on this?
>>>>
>>>> If so, will you share it? I'm terribly curious what the bug actually is. :-P
>>>>
>>>> On Sun, Nov 12, 2017 at 9:23 PM, Harlan Stenn <stenn at nwtime.org> wrote:
>>>>> Hi Matt,
>>>>>
>>>>> NTPsec doesn’t communicate with us.
>>>>>
>>>>> Thanks for contacting us. We’ll dig.
>>>>>
>>>>> Sent from my iPhone
>>>>>
>>>>>> On Nov 12, 2017, at 2:56 AM, Matt Nordhoff <mnordhoff at mn0.us> wrote:
>>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> To be honest, I first noticed this issue on a Pool server running
>>>>>> NTPsec. (I'm sorry.) I have since reproduced it -- partly -- on NTP
>>>>>> 4.2.8p10 by replaying a packet.
>>>>>>
>>>>>> I first emailed security at ntpsec.org 2017-10-24. I first sent them
>>>>>> pcaps 2017-11-08. I don't know if they have contacted you. They
>>>>>> haven't given me notable information at this time.
>>>>>>
>>>>>> I have a stratum 2 server running Ubuntu 16.04 and NTP 4.2.8p10.
>>>>>>
>>>>>> Certain weird packets from certain clients can cause ntpd to create
>>>>>> some sort of preemptable server association.
>>>>>>
>>>>>> I'm not sure what's happening. Maybe something weird with manycast
>>>>>> mode, i don't know. I'm not certain it's exploitable, but i think it
>>>>>> is.
>>>>>>
>>>>>> (I wonder if restrict notrust would help?)
>>>>>>
>>>>>> In NTP, restrict nopeer apparently usually stops anything from
>>>>>> happening: I've only successfully caused associations to be created
>>>>>> with "client" IPs that are currently or previously configured servers
>>>>>> and already whitelisted by restrict source.
>>>>>>
>>>>>> (In NTPsec, any IP can do it!)
>>>>>>
>>>>>> If the server has loose restricts, or an attacker can spoof their
>>>>>> source IP and knows what servers you're using, it can presumably be
>>>>>> exploited, but it would obviously be harder.
>>>>>>
>>>>>> Notably, a client can create multiple associations.
>>>>>>
>>>>>> As an example, i believe a single Pool server -- especially one a
>>>>>> running a customized NTP server -- could create numerous associations
>>>>>> and obtain undue influence overs its clients' clocks.
>>>>>>
>>>>>> In some cases the packets seem to trigger ntpd's 0 origin checks, but
>>>>>> other times they seem to work. (Unconfirmed speculation: Maybe the
>>>>>> packet creates 1 association, but also causes 1 packet from another
>>>>>> existing association for the same IP to be rejected.)
>>>>>>
>>>>>> In the real world, my Pool server in the Brazil zone gets this traffic
>>>>>> from a number of seemingly ordinary clients. (Reverse DNS looks like
>>>>>> random consumer ISP addresses.) They use ephemeral ports that aren't
>>>>>> running actual NTP servers, so the association is totally unusable and
>>>>>> is eventually removed. (Again, only NTPsec or [presumably] NTP with
>>>>>> "restrict default" without "nopeer" create associations at all.)
>>>>>>
>>>>>> pcap of a single packet from a Brazilian client:
>>>>>>
>>>>>> <https://mn0.us/71WFPp6JPy43QEdDEAqihb1/ntp7_2017-11-12_03:00:22_186.233.180.198_ntp.pcap>
>>>>>> (130 bytes)
>>>>>>
>>>>>> $ tcpdump -nttttvxr ntp7_2017-11-12_03:00:22_186.233.180.198_ntp.pcap
>>>>>> reading from file ntp7_2017-11-12_03:00:22_186.233.180.198_ntp.pcap,
>>>>>> link-type EN10MB (Ethernet)
>>>>>> 2017-11-12 03:00:22.676707 IP (tos 0x0, ttl 113, id 2423, offset 0,
>>>>>> flags [none], proto UDP (17), length 76)
>>>>>>    186.233.180.198.64094 > 45.33.103.94.123: NTPv1, length 48
>>>>>>        Server, Leap indicator:  (0), Stratum 3 (secondary reference),
>>>>>> poll 3 (8s), precision -23
>>>>>>        Root Delay: 0.360198, Root dispersion: 0.099517, Reference-ID:
>>>>>> 54.206.109.187
>>>>>>          Reference Timestamp:  3719442935.600897507 (2017/11/12 02:35:35)
>>>>>>          Originator Timestamp: 0.000000000
>>>>>>          Receive Timestamp:    3719444422.526398753 (2017/11/12 03:00:22)
>>>>>>          Transmit Timestamp:   3719444422.526416904 (2017/11/12 03:00:22)
>>>>>>            Originator - Receive Timestamp:  3719444422.526398753
>>>>>> (2017/11/12 03:00:22)
>>>>>>            Originator - Transmit Timestamp: 3719444422.526416904
>>>>>> (2017/11/12 03:00:22)
>>>>>>        0x0000:  4500 004c 0977 0000 7111 3bfb bae9 b4c6
>>>>>>        0x0010:  2d21 675e fa5e 007b 0038 355a 0c03 03e9
>>>>>>        0x0020:  0000 5c36 0000 197a 36ce 6dbb ddb2 31f7
>>>>>>        0x0030:  99d4 6b48 0000 0000 0000 0000 ddb2 37c6
>>>>>>        0x0040:  86c2 1199 ddb2 37c6 86c3 421c
>>>>>>
>>>>>> ntp.conf (with comments modified):
>>>>>>
>>>>>> <https://mn0.us/DXGMaQRVotwGnZwzXZE6p5x/ntp.conf> (3 KiB)
>>>>>>
>>>>>> Stuff from my NTP server after using Scapy to send it a number of
>>>>>> packets with that payload:
>>>>>>
>>>>>> mnordhoff at clover:~$ date && ntpq -c lpeers
>>>>>> Sun Nov 12 10:32:13 UTC 2017
>>>>>>     remote           refid      st t when poll reach   delay   offset  jitter
>>>>>> ==============================================================================
>>>>>> ntp-pool        .POOL.          16 p    -   64    0    0.000    0.000   0.000
>>>>>> time.nist.gov   .POOL.          16 p    -   64    0    0.000    0.000   0.000
>>>>>> ntp.ubuntu.com  .POOL.          16 p    -   64    0    0.000    0.000   0.000
>>>>>> 2.us.pool.ntp.o .POOL.          16 p    -   64    0    0.000    0.000   0.000
>>>>>> +six0.ntp3.mattn 128.59.0.245     2 u  849 1024  377   18.578   -0.515   0.477
>>>>>> #2604:a880:400:d 130.183.99.210   3 u  437 1024  377   41.530   -0.126   0.329
>>>>>> six1.ntp5.mattn .STEP.          16 u    - 1024    0    0.000    0.000   0.000
>>>>>> +ec2-54-243-186- 45.79.187.10     3 u   76 1024  377   31.163   -0.455   0.443
>>>>>> #six0.ntp7.mattn 35.73.197.144    2 u 113m 1024  300   18.627    0.142   0.068
>>>>>> #2600:1f16:ec6:e 209.51.161.238   2 u 1025 1024  377   34.927   -0.010   0.224
>>>>>> -tick.uh.edu     .GPS.            1 u 1037 1024  347   10.325   -1.544   0.432
>>>>>> *clock.fmt.he.ne .CDMA.           1 u  552 1024  377   37.490   -0.300   0.279
>>>>>> #bedast01.beaust 129.7.1.66       2 u  345 1024  377    1.343    1.487   1.195
>>>>>> +awesome.bytesta 216.218.254.202  2 u  265 1024  377    1.317   -0.439   0.370
>>>>>> #ntp.jtsage.com  127.67.113.92 2 u  862 1024  377    1.068    0.012   0.375
>>>>>> #dev.smatwebdesi 192.168.204.60   3 u  175 1024  377    1.251    0.766   0.958
>>>>>> #serenity.melanc 129.7.1.66       2 u  732 1024  377    0.332    0.360   0.440
>>>>>> #ntp.quintex.com .CDMA.           1 u  436 1024  377   39.356    0.112   0.507
>>>>>> +six0.ntp7.mattn 35.73.197.144    2 u  184 1024  377   18.603   -0.390   0.345
>>>>>> +six0.ntp7.mattn 35.73.197.144    2 u  210 1024  377   21.952   -0.190   0.237
>>>>>> #four0.jane.matt 35.73.197.144    2 u   39   64  177   21.861   -0.270   0.292
>>>>>>
>>>>>> (The last 3 associations were created by Scapy.)
>>>>>>
>>>>>> mnordhoff at clover:~$ date && ntpq -c "rv &9"
>>>>>> Sun Nov 12 10:32:59 UTC 2017
>>>>>> associd=59303 status=951a conf, reach, sel_backup, 1 event, sys_peer,
>>>>>> srcadr=six0.ntp7.mattnordhoff.net, srcport=123,
>>>>>> dstadr=2600:3c00:e000:15a::, dstport=123, leap=00, stratum=2,
>>>>>> precision=-22, rootdelay=1.434, rootdisp=21.133, refid=35.73.197.144,
>>>>>> reftime=ddb28603.2721672c  Sun, Nov 12 2017  8:34:11.152,
>>>>>> rec=ddb2874b.b2db92da  Sun, Nov 12 2017  8:39:39.698, reach=300,
>>>>>> unreach=0, hmode=3, pmode=4, hpoll=10, ppoll=10, headway=0, flash=00 ok,
>>>>>> keyid=0, offset=0.142, delay=18.627, dispersion=19.100, jitter=0.068,
>>>>>> xleave=0.170,
>>>>>> filtdelay=    18.77   18.63   18.71   18.61   18.58   18.78   18.70   18.64,
>>>>>> filtoffset=    0.07    0.14    0.07    0.07    0.06    0.23    0.15    0.09,
>>>>>> filtdisp=      0.00   15.74   31.43   47.25   62.73   78.21   94.31  110.36
>>>>>> mnordhoff at clover:~$ date && ntpq -c "rv &19"
>>>>>> Sun Nov 12 10:33:08 UTC 2017
>>>>>> associd=59325 status=1314 reach, sel_outlier, 1 event, reachable,
>>>>>> srcadr=six0.ntp7.mattnordhoff.net, srcport=123,
>>>>>> dstadr=2600:3c00:e000:15a::, dstport=123, leap=00, stratum=2,
>>>>>> precision=-22, rootdelay=1.312, rootdisp=39.841, refid=35.73.197.144,
>>>>>> reftime=ddb29aa8.271ad31b  Sun, Nov 12 2017 10:02:16.152,
>>>>>> rec=ddb2a0f5.b2cd5cb2  Sun, Nov 12 2017 10:29:09.698, reach=377,
>>>>>> unreach=1, hmode=3, pmode=4, hpoll=10, ppoll=10, headway=0, flash=00 ok,
>>>>>> keyid=0, offset=-0.390, delay=18.603, dispersion=19.365, jitter=0.345,
>>>>>> xleave=0.174,
>>>>>> filtdelay=    18.65   18.60   18.70   18.70   18.59   18.59   18.78   18.72,
>>>>>> filtoffset=   -0.29   -0.39   -0.40   -0.08   -0.01   -0.03    0.09    0.08,
>>>>>> filtdisp=      0.00   16.22   31.73   47.55   63.08   78.90   94.83   95.84
>>>>>> mnordhoff at clover:~$ date && ntpq -c "rv &20"
>>>>>> Sun Nov 12 10:33:21 UTC 2017
>>>>>> associd=59326 status=1314 reach, sel_outlier, 1 event, reachable,
>>>>>> srcadr=six0.ntp7.mattnordhoff.net, srcport=123, dstadr=2600:3c00::2:b401,
>>>>>> dstport=123, leap=00, stratum=2, precision=-22, rootdelay=1.312,
>>>>>> rootdisp=39.444, refid=35.73.197.144,
>>>>>> reftime=ddb29aa8.271ad31b  Sun, Nov 12 2017 10:02:16.152,
>>>>>> rec=ddb2a0db.b3a3de58  Sun, Nov 12 2017 10:28:43.701, reach=377,
>>>>>> unreach=1, hmode=3, pmode=4, hpoll=10, ppoll=10, headway=0, flash=00 ok,
>>>>>> keyid=0, offset=-0.190, delay=21.952, dispersion=15.056, jitter=0.237,
>>>>>> xleave=0.120,
>>>>>> filtdelay=    21.95   22.05   21.95   21.92   21.94   21.88   22.04   21.97,
>>>>>> filtoffset=   -0.19   -0.38   -0.33    0.04    0.10    0.01    0.12    0.06,
>>>>>> filtdisp=      0.00   15.62   30.98   47.24   63.42   78.98   94.52   95.52
>>>>>> mnordhoff at clover:~$ date && ntpq -c "rv &21"
>>>>>> Sun Nov 12 10:33:22 UTC 2017
>>>>>> associd=59327 status=1514 reach, sel_backup, 1 event, reachable,
>>>>>> srcadr=four0.jane.mattnordhoff.net, srcport=123, dstadr=45.79.1.70,
>>>>>> dstport=123, leap=00, stratum=2, precision=-22, rootdelay=1.312,
>>>>>> rootdisp=34.927, refid=35.73.197.144,
>>>>>> reftime=ddb29aa8.271ad31b  Sun, Nov 12 2017 10:02:16.152,
>>>>>> rec=ddb2a186.87cee83c  Sun, Nov 12 2017 10:31:34.530, reach=177,
>>>>>> unreach=7, hmode=3, pmode=4, hpoll=10, ppoll=6, headway=0,
>>>>>> flash=01 pkt_dup, keyid=0, offset=-0.270, delay=21.861,
>>>>>> dispersion=77.059, jitter=0.292, xleave=0.139,
>>>>>> filtdelay=    21.86   21.92   21.84   21.82   21.88   21.94   21.82    0.00,
>>>>>> filtoffset=   -0.27   -0.38   -0.23    0.08    0.05    0.11    0.09    0.00,
>>>>>> filtdisp=      0.00   15.92   32.18   48.24   64.22   65.19   66.20 16000.0
>>>>>>
>>>>>> NTP's syslog messages:
>>>>>>
>>>>>> Nov 12 07:55:54 clover ntpd[4737]: 45.33.103.94 local addr 45.79.1.70 -> <null>
>>>>>> Nov 12 08:00:14 clover ntpd[4737]: 45.33.103.94 local addr 45.79.1.70 -> <null>
>>>>>> Nov 12 08:10:48 clover ntpd[4737]: 2600:3c02::13:5230 local addr
>>>>>> 2600:3c00::2:b401 -> 2600:3c00:e000:15a::
>>>>>> Nov 12 08:11:31 clover ntpd[4737]: 45.33.103.94 local addr 45.79.1.70 -> <null>
>>>>>> Nov 12 08:19:45 clover ntpd[4737]: 2600:3c02::13:5230 local addr
>>>>>> 2600:3c00:e000:15a:: -> <null>
>>>>>> Nov 12 08:37:57 clover ntpd[4737]: 2600:3c02::13:5230 local addr
>>>>>> 2600:3c00::2:b401 -> 2600:3c00:e000:15a::
>>>>>> Nov 12 08:38:02 clover ntpd[4737]: 2600:3c02::13:5230 local addr
>>>>>> 2600:3c00::2:b401 -> 2600:3c00:e000:15a::
>>>>>> Nov 12 08:38:07 clover ntpd[4737]: 2600:3c02::13:5230 local addr
>>>>>> 2600:3c00::2:b401 -> 2600:3c00:e000:15a::
>>>>>> Nov 12 08:41:28 clover ntpd[4737]: 2600:3c02::13:5230 local addr
>>>>>> 2600:3c00::2:b401 -> 2600:3c00:e000:15a::
>>>>>> Nov 12 08:41:29 clover ntpd[4737]: receive: Drop 0 origin timestamp
>>>>>> from server at 2600:3c02::13:5230 xmt 0xddb237c6.86c3421c
>>>>>> Nov 12 08:41:30 clover ntpd[4737]: receive: Drop 0 origin timestamp
>>>>>> from server at 2600:3c02::13:5230 xmt 0xddb237c6.86c3421c
>>>>>> Nov 12 08:47:05 clover ntpd[4737]: 2600:3c02::13:5230 local addr
>>>>>> 2600:3c00:e000:15a:: -> <null>
>>>>>> Nov 12 08:47:47 clover ntpd[4737]: 2600:3c02::13:5230 local addr
>>>>>> 2600:3c00:e000:15a:: -> <null>
>>>>>> Nov 12 08:47:50 clover ntpd[4737]: 2600:3c02::13:5230 local addr
>>>>>> 2600:3c00:e000:15a:: -> <null>
>>>>>> Nov 12 08:49:11 clover ntpd[4737]: receive: Drop 0 origin timestamp
>>>>>> from server at 2600:3c02::13:5230 xmt 0xddb237c6.86c3421c
>>>>>> Nov 12 08:57:27 clover ntpd[4737]: receive: Unexpected origin
>>>>>> timestamp 0xddb28b77.ae070b4f does not match aorg 0000000000.00000000
>>>>>> from server at 2600:3c02::13:5230 xmt 0xddb28b77.b06f9d59
>>>>>> Nov 12 09:10:10 clover ntpd[4737]: receive: Drop 0 origin timestamp
>>>>>> from server at 2600:3c02::13:5230 xmt 0xddb237c6.86c3421c
>>>>>> Nov 12 09:14:48 clover ntpd[4737]: receive: Unexpected origin
>>>>>> timestamp 0xddb28f88.ae02ddad does not match aorg 0000000000.00000000
>>>>>> from server at 2600:3c02::13:5230 xmt 0xddb28f88.b074ed27
>>>>>> Nov 12 09:32:42 clover ntpd[4737]: receive: Unexpected origin
>>>>>> timestamp 0xddb293ba.ae02ed68 does not match aorg 0000000000.00000000
>>>>>> from server at 2600:3c02::13:5230 xmt 0xddb293ba.b079bf05
>>>>>> Nov 12 09:50:16 clover ntpd[4737]: receive: Unexpected origin
>>>>>> timestamp 0xddb297d8.ae02452d does not match aorg 0000000000.00000000
>>>>>> from server at 2600:3c02::13:5230 xmt 0xddb297d8.b058211b
>>>>>> Nov 12 10:07:53 clover ntpd[4737]: receive: Unexpected origin
>>>>>> timestamp 0xddb29bf9.ae08e00f does not match aorg 0000000000.00000000
>>>>>> from server at 2600:3c02::13:5230 xmt 0xddb29bf9.b05331aa
>>>>>> Nov 12 10:25:32 clover ntpd[4737]: receive: Unexpected origin
>>>>>> timestamp 0xddb2a01c.ae02b238 does not match aorg 0000000000.00000000
>>>>>> from server at 2600:3c02::13:5230 xmt 0xddb2a01c.b05f2c19
>>>>>> Nov 12 10:29:50 clover ntpd[4737]: 45.33.103.94 local addr 45.79.1.70 -> <null>
>>>>>> Nov 12 10:31:34 clover ntpd[4737]: receive: Drop 0 origin timestamp
>>>>>> from server at 45.33.103.94 xmt 0xddb237c6.86c3421c
>>>>>> Nov 12 10:42:54 clover ntpd[4737]: receive: Unexpected origin
>>>>>> timestamp 0xddb2a42e.ae0295d4 does not match aorg 0000000000.00000000
>>>>>> from server at 2600:3c02::13:5230 xmt 0xddb2a42e.b05ac12c
>>>>>>
>>>>>> pcaps I've sent NTPsec:
>>>>>>
>>>>>> <https://mn0.us/g82RmQ8uVbWFpEGoywoJdk/ntp7_2017-11-08_09:35_143.137.65.13_ntp.pcap>
>>>>>> (9 KiB)
>>>>>>
>>>>>> <https://mn0.us/L4Hpt2fbGtnGYr1GnKLY7cj/ntp7_2017-11-08_10_187.1.57.195_ntp.pcap>
>>>>>> (66 KiB)
>>>>>>
>>>>>> <https://mn0.us/rD9DouTTX97LidW49gHvFdf/ntp7_2017-11-08_15_200.199.238.226_ntp.pcap>
>>>>>> (1 KiB)
>>>>>>
>>>>>> Some of them include traffic from my NTPsec ntpd, and the first one
>>>>>> may be incomplete because I ran out of disk space for a few minutes.
>>>>>> (That was fun.)
>>>>>>
>>>>>> Cheers :-/
-- 
Matt Nordhoff


More information about the security mailing list