[ntp:security] Authorized (not nopeer) IPs can create server associations with certain query

Matt Nordhoff mnordhoff at mn0.us
Fri Dec 8 16:44:30 UTC 2017


On Fri, Dec 8, 2017 at 11:03 AM, Harlan Stenn <stenn at nwtime.org> wrote:
> Hi Matt,
>
> I think this is a reported problem, and if so, it's scheduled to be
> fixed for 4.2.8p11.

Alright. Great. :-)

> It looks like I'm the one who will fix it, and the underlying issue is
> intricate, and we need to be very careful and deliberate on exactly how
> it is fixed.  If we don't go "far enough" there's still a problem.  If
> we go "too far" we prohibit desirable behavior.

I'm sorry. Good luck. :-(

> I was hoping to release ntp-4.2.8p11 on Tuesday 8 Jan, but with the
> holidays approaching, I don't know that we'll have it ready for our
> institutional members to be able to test and integrate it by then.  So
> it may be an additional week (the 15th), possibly two (the 22nd), before
> it is released.

Thank you for explaining. January sounds pretty good to me.

(Happy holidays!)

> Does NTF have any paperwork with you?  If we do, we might be able to
> arrange early access to the patch.  That patch must be kept confidential
> until the pubic release (ie, not shared with anybody).

No, no paperwork. I don't want to sound too cavalier, but as far as i
know, it seems to be a medium severity issue for NTP, not very
amenable to non-targeted, massive exploitation. (Of course a client
can try to map a server's sys peers, but still.) I'm not that
concerned. Don't worry about going out of your way.

> H

For what it's worth, if i had a personal security disclosure policy --
and i don't yet -- i would probably copy Project Zero, which would put
the deadline in February. (And it would be unfair to jump you with
something else later after writing that sentence.) So i won't complain
about releasing in January.

For reference, NTPsec has a 45 day disclosure policy, or earlier or
later with "[e]xtenuating circumstances", and i first emailed them 45
days ago today. I can't speculate, but it seems they have the
potential to do anything at any time. (Which would ultimately be my
responsibility, at one remove.) :-/

Matt

> On 12/4/17 7:29 AM, Matt Nordhoff wrote:
>> Hi again,
>>
>> Do you have any news on this?
>>
>> If so, will you share it? I'm terribly curious what the bug actually is. :-P
>>
>> On Sun, Nov 12, 2017 at 9:23 PM, Harlan Stenn <stenn at nwtime.org> wrote:
>>> Hi Matt,
>>>
>>> NTPsec doesn’t communicate with us.
>>>
>>> Thanks for contacting us. We’ll dig.
>>>
>>> Sent from my iPhone
>>>
>>>> On Nov 12, 2017, at 2:56 AM, Matt Nordhoff <mnordhoff at mn0.us> wrote:
>>>>
>>>> Hi,
>>>>
>>>> To be honest, I first noticed this issue on a Pool server running
>>>> NTPsec. (I'm sorry.) I have since reproduced it -- partly -- on NTP
>>>> 4.2.8p10 by replaying a packet.
>>>>
>>>> I first emailed security at ntpsec.org 2017-10-24. I first sent them
>>>> pcaps 2017-11-08. I don't know if they have contacted you. They
>>>> haven't given me notable information at this time.
>>>>
>>>> I have a stratum 2 server running Ubuntu 16.04 and NTP 4.2.8p10.
>>>>
>>>> Certain weird packets from certain clients can cause ntpd to create
>>>> some sort of preemptable server association.
>>>>
>>>> I'm not sure what's happening. Maybe something weird with manycast
>>>> mode, i don't know. I'm not certain it's exploitable, but i think it
>>>> is.
>>>>
>>>> (I wonder if restrict notrust would help?)
>>>>
>>>> In NTP, restrict nopeer apparently usually stops anything from
>>>> happening: I've only successfully caused associations to be created
>>>> with "client" IPs that are currently or previously configured servers
>>>> and already whitelisted by restrict source.
>>>>
>>>> (In NTPsec, any IP can do it!)
>>>>
>>>> If the server has loose restricts, or an attacker can spoof their
>>>> source IP and knows what servers you're using, it can presumably be
>>>> exploited, but it would obviously be harder.
>>>>
>>>> Notably, a client can create multiple associations.
>>>>
>>>> As an example, i believe a single Pool server -- especially one a
>>>> running a customized NTP server -- could create numerous associations
>>>> and obtain undue influence overs its clients' clocks.
>>>>
>>>> In some cases the packets seem to trigger ntpd's 0 origin checks, but
>>>> other times they seem to work. (Unconfirmed speculation: Maybe the
>>>> packet creates 1 association, but also causes 1 packet from another
>>>> existing association for the same IP to be rejected.)
>>>>
>>>> In the real world, my Pool server in the Brazil zone gets this traffic
>>>> from a number of seemingly ordinary clients. (Reverse DNS looks like
>>>> random consumer ISP addresses.) They use ephemeral ports that aren't
>>>> running actual NTP servers, so the association is totally unusable and
>>>> is eventually removed. (Again, only NTPsec or [presumably] NTP with
>>>> "restrict default" without "nopeer" create associations at all.)
>>>>
>>>> pcap of a single packet from a Brazilian client:
>>>>
>>>> <https://mn0.us/71WFPp6JPy43QEdDEAqihb1/ntp7_2017-11-12_03:00:22_186.233.180.198_ntp.pcap>
>>>> (130 bytes)
>>>>
>>>> $ tcpdump -nttttvxr ntp7_2017-11-12_03:00:22_186.233.180.198_ntp.pcap
>>>> reading from file ntp7_2017-11-12_03:00:22_186.233.180.198_ntp.pcap,
>>>> link-type EN10MB (Ethernet)
>>>> 2017-11-12 03:00:22.676707 IP (tos 0x0, ttl 113, id 2423, offset 0,
>>>> flags [none], proto UDP (17), length 76)
>>>>    186.233.180.198.64094 > 45.33.103.94.123: NTPv1, length 48
>>>>        Server, Leap indicator:  (0), Stratum 3 (secondary reference),
>>>> poll 3 (8s), precision -23
>>>>        Root Delay: 0.360198, Root dispersion: 0.099517, Reference-ID:
>>>> 54.206.109.187
>>>>          Reference Timestamp:  3719442935.600897507 (2017/11/12 02:35:35)
>>>>          Originator Timestamp: 0.000000000
>>>>          Receive Timestamp:    3719444422.526398753 (2017/11/12 03:00:22)
>>>>          Transmit Timestamp:   3719444422.526416904 (2017/11/12 03:00:22)
>>>>            Originator - Receive Timestamp:  3719444422.526398753
>>>> (2017/11/12 03:00:22)
>>>>            Originator - Transmit Timestamp: 3719444422.526416904
>>>> (2017/11/12 03:00:22)
>>>>        0x0000:  4500 004c 0977 0000 7111 3bfb bae9 b4c6
>>>>        0x0010:  2d21 675e fa5e 007b 0038 355a 0c03 03e9
>>>>        0x0020:  0000 5c36 0000 197a 36ce 6dbb ddb2 31f7
>>>>        0x0030:  99d4 6b48 0000 0000 0000 0000 ddb2 37c6
>>>>        0x0040:  86c2 1199 ddb2 37c6 86c3 421c
>>>>
>>>> ntp.conf (with comments modified):
>>>>
>>>> <https://mn0.us/DXGMaQRVotwGnZwzXZE6p5x/ntp.conf> (3 KiB)
>>>>
>>>> Stuff from my NTP server after using Scapy to send it a number of
>>>> packets with that payload:
>>>>
>>>> mnordhoff at clover:~$ date && ntpq -c lpeers
>>>> Sun Nov 12 10:32:13 UTC 2017
>>>>     remote           refid      st t when poll reach   delay   offset  jitter
>>>> ==============================================================================
>>>> ntp-pool        .POOL.          16 p    -   64    0    0.000    0.000   0.000
>>>> time.nist.gov   .POOL.          16 p    -   64    0    0.000    0.000   0.000
>>>> ntp.ubuntu.com  .POOL.          16 p    -   64    0    0.000    0.000   0.000
>>>> 2.us.pool.ntp.o .POOL.          16 p    -   64    0    0.000    0.000   0.000
>>>> +six0.ntp3.mattn 128.59.0.245     2 u  849 1024  377   18.578   -0.515   0.477
>>>> #2604:a880:400:d 130.183.99.210   3 u  437 1024  377   41.530   -0.126   0.329
>>>> six1.ntp5.mattn .STEP.          16 u    - 1024    0    0.000    0.000   0.000
>>>> +ec2-54-243-186- 45.79.187.10     3 u   76 1024  377   31.163   -0.455   0.443
>>>> #six0.ntp7.mattn 35.73.197.144    2 u 113m 1024  300   18.627    0.142   0.068
>>>> #2600:1f16:ec6:e 209.51.161.238   2 u 1025 1024  377   34.927   -0.010   0.224
>>>> -tick.uh.edu     .GPS.            1 u 1037 1024  347   10.325   -1.544   0.432
>>>> *clock.fmt.he.ne .CDMA.           1 u  552 1024  377   37.490   -0.300   0.279
>>>> #bedast01.beaust 129.7.1.66       2 u  345 1024  377    1.343    1.487   1.195
>>>> +awesome.bytesta 216.218.254.202  2 u  265 1024  377    1.317   -0.439   0.370
>>>> #ntp.jtsage.com  127.67.113.92 2 u  862 1024  377    1.068    0.012   0.375
>>>> #dev.smatwebdesi 192.168.204.60   3 u  175 1024  377    1.251    0.766   0.958
>>>> #serenity.melanc 129.7.1.66       2 u  732 1024  377    0.332    0.360   0.440
>>>> #ntp.quintex.com .CDMA.           1 u  436 1024  377   39.356    0.112   0.507
>>>> +six0.ntp7.mattn 35.73.197.144    2 u  184 1024  377   18.603   -0.390   0.345
>>>> +six0.ntp7.mattn 35.73.197.144    2 u  210 1024  377   21.952   -0.190   0.237
>>>> #four0.jane.matt 35.73.197.144    2 u   39   64  177   21.861   -0.270   0.292
>>>>
>>>> (The last 3 associations were created by Scapy.)
>>>>
>>>> mnordhoff at clover:~$ date && ntpq -c "rv &9"
>>>> Sun Nov 12 10:32:59 UTC 2017
>>>> associd=59303 status=951a conf, reach, sel_backup, 1 event, sys_peer,
>>>> srcadr=six0.ntp7.mattnordhoff.net, srcport=123,
>>>> dstadr=2600:3c00:e000:15a::, dstport=123, leap=00, stratum=2,
>>>> precision=-22, rootdelay=1.434, rootdisp=21.133, refid=35.73.197.144,
>>>> reftime=ddb28603.2721672c  Sun, Nov 12 2017  8:34:11.152,
>>>> rec=ddb2874b.b2db92da  Sun, Nov 12 2017  8:39:39.698, reach=300,
>>>> unreach=0, hmode=3, pmode=4, hpoll=10, ppoll=10, headway=0, flash=00 ok,
>>>> keyid=0, offset=0.142, delay=18.627, dispersion=19.100, jitter=0.068,
>>>> xleave=0.170,
>>>> filtdelay=    18.77   18.63   18.71   18.61   18.58   18.78   18.70   18.64,
>>>> filtoffset=    0.07    0.14    0.07    0.07    0.06    0.23    0.15    0.09,
>>>> filtdisp=      0.00   15.74   31.43   47.25   62.73   78.21   94.31  110.36
>>>> mnordhoff at clover:~$ date && ntpq -c "rv &19"
>>>> Sun Nov 12 10:33:08 UTC 2017
>>>> associd=59325 status=1314 reach, sel_outlier, 1 event, reachable,
>>>> srcadr=six0.ntp7.mattnordhoff.net, srcport=123,
>>>> dstadr=2600:3c00:e000:15a::, dstport=123, leap=00, stratum=2,
>>>> precision=-22, rootdelay=1.312, rootdisp=39.841, refid=35.73.197.144,
>>>> reftime=ddb29aa8.271ad31b  Sun, Nov 12 2017 10:02:16.152,
>>>> rec=ddb2a0f5.b2cd5cb2  Sun, Nov 12 2017 10:29:09.698, reach=377,
>>>> unreach=1, hmode=3, pmode=4, hpoll=10, ppoll=10, headway=0, flash=00 ok,
>>>> keyid=0, offset=-0.390, delay=18.603, dispersion=19.365, jitter=0.345,
>>>> xleave=0.174,
>>>> filtdelay=    18.65   18.60   18.70   18.70   18.59   18.59   18.78   18.72,
>>>> filtoffset=   -0.29   -0.39   -0.40   -0.08   -0.01   -0.03    0.09    0.08,
>>>> filtdisp=      0.00   16.22   31.73   47.55   63.08   78.90   94.83   95.84
>>>> mnordhoff at clover:~$ date && ntpq -c "rv &20"
>>>> Sun Nov 12 10:33:21 UTC 2017
>>>> associd=59326 status=1314 reach, sel_outlier, 1 event, reachable,
>>>> srcadr=six0.ntp7.mattnordhoff.net, srcport=123, dstadr=2600:3c00::2:b401,
>>>> dstport=123, leap=00, stratum=2, precision=-22, rootdelay=1.312,
>>>> rootdisp=39.444, refid=35.73.197.144,
>>>> reftime=ddb29aa8.271ad31b  Sun, Nov 12 2017 10:02:16.152,
>>>> rec=ddb2a0db.b3a3de58  Sun, Nov 12 2017 10:28:43.701, reach=377,
>>>> unreach=1, hmode=3, pmode=4, hpoll=10, ppoll=10, headway=0, flash=00 ok,
>>>> keyid=0, offset=-0.190, delay=21.952, dispersion=15.056, jitter=0.237,
>>>> xleave=0.120,
>>>> filtdelay=    21.95   22.05   21.95   21.92   21.94   21.88   22.04   21.97,
>>>> filtoffset=   -0.19   -0.38   -0.33    0.04    0.10    0.01    0.12    0.06,
>>>> filtdisp=      0.00   15.62   30.98   47.24   63.42   78.98   94.52   95.52
>>>> mnordhoff at clover:~$ date && ntpq -c "rv &21"
>>>> Sun Nov 12 10:33:22 UTC 2017
>>>> associd=59327 status=1514 reach, sel_backup, 1 event, reachable,
>>>> srcadr=four0.jane.mattnordhoff.net, srcport=123, dstadr=45.79.1.70,
>>>> dstport=123, leap=00, stratum=2, precision=-22, rootdelay=1.312,
>>>> rootdisp=34.927, refid=35.73.197.144,
>>>> reftime=ddb29aa8.271ad31b  Sun, Nov 12 2017 10:02:16.152,
>>>> rec=ddb2a186.87cee83c  Sun, Nov 12 2017 10:31:34.530, reach=177,
>>>> unreach=7, hmode=3, pmode=4, hpoll=10, ppoll=6, headway=0,
>>>> flash=01 pkt_dup, keyid=0, offset=-0.270, delay=21.861,
>>>> dispersion=77.059, jitter=0.292, xleave=0.139,
>>>> filtdelay=    21.86   21.92   21.84   21.82   21.88   21.94   21.82    0.00,
>>>> filtoffset=   -0.27   -0.38   -0.23    0.08    0.05    0.11    0.09    0.00,
>>>> filtdisp=      0.00   15.92   32.18   48.24   64.22   65.19   66.20 16000.0
>>>>
>>>> NTP's syslog messages:
>>>>
>>>> Nov 12 07:55:54 clover ntpd[4737]: 45.33.103.94 local addr 45.79.1.70 -> <null>
>>>> Nov 12 08:00:14 clover ntpd[4737]: 45.33.103.94 local addr 45.79.1.70 -> <null>
>>>> Nov 12 08:10:48 clover ntpd[4737]: 2600:3c02::13:5230 local addr
>>>> 2600:3c00::2:b401 -> 2600:3c00:e000:15a::
>>>> Nov 12 08:11:31 clover ntpd[4737]: 45.33.103.94 local addr 45.79.1.70 -> <null>
>>>> Nov 12 08:19:45 clover ntpd[4737]: 2600:3c02::13:5230 local addr
>>>> 2600:3c00:e000:15a:: -> <null>
>>>> Nov 12 08:37:57 clover ntpd[4737]: 2600:3c02::13:5230 local addr
>>>> 2600:3c00::2:b401 -> 2600:3c00:e000:15a::
>>>> Nov 12 08:38:02 clover ntpd[4737]: 2600:3c02::13:5230 local addr
>>>> 2600:3c00::2:b401 -> 2600:3c00:e000:15a::
>>>> Nov 12 08:38:07 clover ntpd[4737]: 2600:3c02::13:5230 local addr
>>>> 2600:3c00::2:b401 -> 2600:3c00:e000:15a::
>>>> Nov 12 08:41:28 clover ntpd[4737]: 2600:3c02::13:5230 local addr
>>>> 2600:3c00::2:b401 -> 2600:3c00:e000:15a::
>>>> Nov 12 08:41:29 clover ntpd[4737]: receive: Drop 0 origin timestamp
>>>> from server at 2600:3c02::13:5230 xmt 0xddb237c6.86c3421c
>>>> Nov 12 08:41:30 clover ntpd[4737]: receive: Drop 0 origin timestamp
>>>> from server at 2600:3c02::13:5230 xmt 0xddb237c6.86c3421c
>>>> Nov 12 08:47:05 clover ntpd[4737]: 2600:3c02::13:5230 local addr
>>>> 2600:3c00:e000:15a:: -> <null>
>>>> Nov 12 08:47:47 clover ntpd[4737]: 2600:3c02::13:5230 local addr
>>>> 2600:3c00:e000:15a:: -> <null>
>>>> Nov 12 08:47:50 clover ntpd[4737]: 2600:3c02::13:5230 local addr
>>>> 2600:3c00:e000:15a:: -> <null>
>>>> Nov 12 08:49:11 clover ntpd[4737]: receive: Drop 0 origin timestamp
>>>> from server at 2600:3c02::13:5230 xmt 0xddb237c6.86c3421c
>>>> Nov 12 08:57:27 clover ntpd[4737]: receive: Unexpected origin
>>>> timestamp 0xddb28b77.ae070b4f does not match aorg 0000000000.00000000
>>>> from server at 2600:3c02::13:5230 xmt 0xddb28b77.b06f9d59
>>>> Nov 12 09:10:10 clover ntpd[4737]: receive: Drop 0 origin timestamp
>>>> from server at 2600:3c02::13:5230 xmt 0xddb237c6.86c3421c
>>>> Nov 12 09:14:48 clover ntpd[4737]: receive: Unexpected origin
>>>> timestamp 0xddb28f88.ae02ddad does not match aorg 0000000000.00000000
>>>> from server at 2600:3c02::13:5230 xmt 0xddb28f88.b074ed27
>>>> Nov 12 09:32:42 clover ntpd[4737]: receive: Unexpected origin
>>>> timestamp 0xddb293ba.ae02ed68 does not match aorg 0000000000.00000000
>>>> from server at 2600:3c02::13:5230 xmt 0xddb293ba.b079bf05
>>>> Nov 12 09:50:16 clover ntpd[4737]: receive: Unexpected origin
>>>> timestamp 0xddb297d8.ae02452d does not match aorg 0000000000.00000000
>>>> from server at 2600:3c02::13:5230 xmt 0xddb297d8.b058211b
>>>> Nov 12 10:07:53 clover ntpd[4737]: receive: Unexpected origin
>>>> timestamp 0xddb29bf9.ae08e00f does not match aorg 0000000000.00000000
>>>> from server at 2600:3c02::13:5230 xmt 0xddb29bf9.b05331aa
>>>> Nov 12 10:25:32 clover ntpd[4737]: receive: Unexpected origin
>>>> timestamp 0xddb2a01c.ae02b238 does not match aorg 0000000000.00000000
>>>> from server at 2600:3c02::13:5230 xmt 0xddb2a01c.b05f2c19
>>>> Nov 12 10:29:50 clover ntpd[4737]: 45.33.103.94 local addr 45.79.1.70 -> <null>
>>>> Nov 12 10:31:34 clover ntpd[4737]: receive: Drop 0 origin timestamp
>>>> from server at 45.33.103.94 xmt 0xddb237c6.86c3421c
>>>> Nov 12 10:42:54 clover ntpd[4737]: receive: Unexpected origin
>>>> timestamp 0xddb2a42e.ae0295d4 does not match aorg 0000000000.00000000
>>>> from server at 2600:3c02::13:5230 xmt 0xddb2a42e.b05ac12c
>>>>
>>>> pcaps I've sent NTPsec:
>>>>
>>>> <https://mn0.us/g82RmQ8uVbWFpEGoywoJdk/ntp7_2017-11-08_09:35_143.137.65.13_ntp.pcap>
>>>> (9 KiB)
>>>>
>>>> <https://mn0.us/L4Hpt2fbGtnGYr1GnKLY7cj/ntp7_2017-11-08_10_187.1.57.195_ntp.pcap>
>>>> (66 KiB)
>>>>
>>>> <https://mn0.us/rD9DouTTX97LidW49gHvFdf/ntp7_2017-11-08_15_200.199.238.226_ntp.pcap>
>>>> (1 KiB)
>>>>
>>>> Some of them include traffic from my NTPsec ntpd, and the first one
>>>> may be incomplete because I ran out of disk space for a few minutes.
>>>> (That was fun.)
>>>>
>>>> Cheers :-/
-- 
Matt Nordhoff


More information about the security mailing list