[ntp:questions] Ntpq.exe memory issue with windows 2019
Sadique Urf Arbaz Sayyed
sadique.sayyed at logicmonitor.com
Wed Oct 21 14:44:13 UTC 2020
On Wednesday, October 21, 2020 at 8:02:33 PM UTC+5:30, Jakob Bohm wrote:
> On 2020-10-21 15:16, Sadique Urf Arbaz Sayyed wrote:
> > On Wednesday, October 21, 2020 at 6:19:21 PM UTC+5:30, Jakob Bohm wrote:
> >> On 2020-10-21 12:45, Sadique Urf Arbaz Sayyed wrote:
> >>> On Wednesday, October 21, 2020 at 4:14:05 PM UTC+5:30, Sadique Urf Arbaz Sayyed wrote:
> >>>> We started with a brand new windows server 2019 datacenter edition and installed an infrastructure monitoring agent on it and strictly no other program. The machine had 8 GB of memory. As part of monitoring NTP offset from sync'd host we scheduled a ntpqexe. The problem started after 4-5 days, the memory utilisation had increased to significant level >80%. On analysis we found it was a gradual increase and using RAMMAP we saw every time the ntpq.exe will run it will leave behind 24k of memory in PAGE Table with 0 B in Private. Moreover this issue is specific to windows server 2019 we tried following same steps on windows server 2012 machine and it worked perfectly fine with no memory creeping issues.
> >>>> Any help or pointer are appreciated
> >>> Full problem is mentioned here https://social.technet.microsoft.com/Forums/en-US/d45ba91a-cc43-4010-9a2c-d65b1a4cc33a/windows-2019-server-ntp-increases-memory-utilisation-by-acquiring-page-table?forum=ws2019
> >> From what you have posted above (I have not checked the MS forum) it
> >> sounds like the leak is of "PAGE"-d memory consumption, not page tables
> >> (also known in the NT world as hyperspace).
> >> Question is what is actually registered as owning that page-able memory.
> >> Maybe you have found a memory leak in the NTP server itself, maybe
> >> something in NT 10.??.2019?? (server 2019) leaks memory every time ntpq
> >> makes an UDP socket connection to the NTP service to query it.
> >> There are some advanced tools for checking what owns outstanding global
> >> (kernel) memory allocations on NT systems. However first you should
> >> check the "commit charge" of the NTP service process, as that is a
> >> direct measure of how much pageable virtual memory is allocated in the
> >> user mode part of that process.
> >> Enjoy
> >> Jakob
> >> --
> >> Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com
> >> Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10
> >> This public discussion message is non-binding and may contain errors.
> >> WiseMo - Remote Service Management for PCs, Phones and Embedded
> > Ok so this NT server is used by our customers but in house I do not have any server. I'm simply running ntpq.exe -nc peers 127.0.0.1, here I am not expecting the output but why is it hogging onto memory. As you pointed out ntpq uses the UDP port 123 which is by default used by Windows Time service. And this issue does not occur if I have the windows time service disabled. So my question here is why does the process hogs onto memory if I have the Windows time service running (In windows 2019 it's running by default).
> ntpq.exe is designed to talk to ntpd.exe from the same source
> code bundle. Microsoft's NTP service (win32time) probably
> completely lacks the ability to correctly handle that query.
> Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com
> Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10
> This public discussion message is non-binding and may contain errors.
> WiseMo - Remote Service Management for PCs, Phones and Embedded
Yes Correct, But John here suppose my ntpd is running on a server and from the client side (Suppose windows time service is running) I run ntpq.exe with above command I do see the issue. The process will hold the 24Kb but actually the process is not visible in Task manager or anywhere but in RamMap we see that the executed process still occupied the Page table memory. So each query fired will consume 24KB and server will eventually see the 100% memory utilisation. I even had a call with Microsoft support they confirmed that this would be the application specific issue where caching is not cleared properly. I had the similar memory issue with IPMI utils as well and they provided me with the Hotfix.
More information about the questions