Hi, we've had a few brave end users upgrade their Windows 7 Pro systems to Windows 10 Pro.
I've run several patch scans on a couple of these and they aren't showing correct info for Windows 10 - it looks like it's still displaying info for the previous Windows installation (200+ installed patches, 20 or so missing Windows 7 patches, etc).
Has anyone else seen this happening?
I have seen similar behavior from 8.1 > 10 endpoints as well.
I believe this is all part of a framework change where Kaseya needs to set the right parameters for Windows 10 endpoints.
Thanks for the confirmation Oscar. We're on VSA R9.1 which claims 'Beta' support for Windows 10.
I accept that means not all VSA/agent functionality will work, but surely at least if the patch scan process can't return meaningful results it should indicate this. Right now if we distribute executive reports our customers will think these machines aren't being patched...
I'll log a ticket with support. Let's see what they say.
Hello Combo oromero
I have check on a new Windows 10 install and the Patch Status is not getting populated. So it looks like it is historical data from the OS before the Upgrade that is been displayed still.
I will check with support to see if they have any reported this to Engineering as yet.
I'll update everyone as soon as I have more information.
We will also verify what 'Beta' support for Windows 10 in R9.1 actually means.
I used Agent / Install Agents / Delete account now without uninstalling the agent and that got rid of old patch status when the agent checked-in again.
Yes, I can also confirm that my experience with machines upgraded to 10 from an older OS, patch management shows the "installed patches" of the old OS. I'm using R9.1 with patch 220.127.116.11.
The "missing approved", "missing denied" etc. data appears accurate.
Re-running baseline audit doesn't appear to help. I think I could clear the stale data with some simple SQL, however I have no idea how to re-populate the field with accurate data from the new OS.
It's probably a hidden procedure that runs as part of a new agent checking in for the first time. Perhaps Kaseya can reveal to us how to run the appropriate script to refresh the machine status.
In the meantime, the delete agent without uninstall works; however of course you loose all your config of the agent if you do that (credentials, lancache, monitor sets, AV, etc) so if you don't use policy management you'd need to recreate all that afterwards.
Craig Hart, Combo,
I'm hearing two different experiences and would like to clarify. It sounds as though Combo is reporting the Missing patches appear incorrect but Craig Hart is seeing the Missing data reported correctly but is seeing the Installed patches still reporting the older/legacy patch information. Am I understanding this correctly?
I would expect the behavior Craig Hart (chart71) is reporting. When Kaseya installs a patch, we keep a record of that patch as installed. That record will 'follow' the endpoint for its lifetime. An upgrade of the OS would not negate the fact that a patch for Win7 (for example) was previously installed on the machine at some point in history. As long as the agentGUID remains the same, the patches installed under a previous OS would remain listed as Installed. Audit will not change this as Audit does not collect patch information. Patch info is gathered/processed based on a Patch Scan. The patch scan is looking at patches currently applicable to the machine and the status of those patches (installed, not installed). A patch scan, however, should not 'remove' a patch from patch history that has been previously reported as installed. That is the expected behavior. If you want to 'refresh' the patch history list, you'd need to uninstall/reinstall the agent - that has other ramifications, and I don't necessarily recommend doing so as you'll lose all other historical info such as logs and config settings.
What Combo is reporting (or what I'm understanding from the post) I would consider unexpected. Since Kaseya leverages Microsoft's Windows Update Agent (WUA) for scans and installs, we provide the info into the VSA that's gathered by WUA through its .api. If what you're seeing in the VSA doesn't appear to be correct, I would recommend running a scan on the machine locally using the windows update client (Control panel > windows update) and check whether what's reported in the local client matches what you're seeing in the VSA. If the local client is returning results for missing Win7 patches, even though the machine is running Win10, then MS has determined the machine needs that patch. If you're not seeing any Win7 patches on the local scan but you are finding them listed as missing within the VSA, then that may mean Kaseya is not properly communicating with WUA.api or that we're not properly consuming the resulting data. Either of those would necessitate a ticket to support to look into further.
I have not yet upgraded (personally) to Win10, but do plan to do so soon. Once I've upgraded, I'll do some testing to see what behavior I'm getting.
Let me know if you have questions and/or if I've misunderstood the behavior you're seeing.
Brande, you understand me correctly. I'll also mention that since zero patches have as yet been made available for Win10, it's a bit hard for me to confirm if the missing/needed counts are accurate as yet!
What I can say is the missing/needed stats show as "-", the Win10 box's WU also shows no patches pending -- so as far as I know that's correct. However, the win8.1 box also had no pending patches before it was upgraded to win10, so i don't now if old data has been carried over (i.e. the state hasn't changed) or if the data is actually the result of a successful win10 patch scan.
I've done some further investigation and suspect that the patch scan scripts (on our VSA at least) aren't operating as expected on Windows 10 agents. I have tested with a fresh installation of Windows 10 Enterprise and the 'patch scan' process completes nearly instantly. No installed or missing patches are shown in the VSA but there is definitely a missing cumulative update on the endpoint (KB3081424). The only evidence I can see of anything being attempted on the agent is these entries in agentmon.log:
2015/08/10 17:43:04.475 [12b0]: execFile(): Path="C:\Windows\system32\cmd.exe", arg=""C:\Windows\system32\cmd.exe"...", flag=0x3, timeout=7200
2015/08/10 17:43:04.991 [12b0]: C:\Windows\system32\cmd.exe "C:\Windows\system32\cmd.exe" /C NET START wuauserv exited with code: 2
I can see no evidence of any scan results in the kworking or agent installation folders, e.g. no ptchscn2.xml, no wsusscn2.cab, etc.
Interestingly when we were running VSA 6.5 and had a few Windows 10 Preview agents these were returning scan results. Either the upgrade to VSA R9.1 or changes to Windows 10 between the preview and release versions seem to have introduced this new behaviour where patch scans 'complete' without results.
I'll dig into the script tables in SQL when I get a bit of time and post back with the results.
What about c:\windows\windowsupdate.log ?
If you have a chance, run a scan locally (Control Panel > Windows Update, assuming the location hasn't changed in Win10) and see if there are any missing patches. Immediately after that, run a scan via Kaseya. If the local machine is reporting patches but Kaseya is not, gather <workingDirectory>\ptchscn2.xml and %Windows%\WindowsUpdate.log and submit those in a ticket at helpdesk.kaseya.com. The ticket will be necessary to get the proper investigation and dev resources. The back-to-back scan makes comparing the results of the two scans more relevant, since the data from both scans *should* get written to windowsupdate.log.
From what you're describing, it sounds like the root cause of the issue may be different than Craig Hart. The Net Start returning code 2 indicates "Access Denied". This is the return result from Microsoft, not Kaseya (we're just reporting what MS has told the agent), which begs the question whether there's an agent communication issue or something deeper than that. Be sure the service called Windows Update (service name wuauserv) is set to Enabled and, preferably, is Started. You can try to start the service through the UI or through cmd line ("net start wuauserv"). Also confirm the agent is up-to-date and even try to update the agent (enable the "Force Update" option), then reboot the machine and try a patch scan again?
FWIW, wsusscn2.cab will only exist on the endpoint if it's needed, and it's only needed if/when an endpoint cannot reach the internet for the patch scan. The absence of this file is not indicative of a problem in itself. The other files missing make sense since the patch scan isn't getting past the windows service start step. The scan itself is never having a chance to start since the process is failing at the service start step, which comes well before the instructions to invoke the scan. In this case, I would expect %windows%\windowsupdate.log has no record of a Kaseya-invoked scan since it doesn't appear we're making it that far. This may just be an issue of appropriate access to the service - it's possible once that's resolved, the rest will work properly. At a minimum, the service issue has to be resolved before we can determine whether any other roadblocks exist.
Not sure if this is useful information, but I did a clean install (reformatted and repartitioned) my workstation to win10 and installed the kaseya agent without issue and it can't seem to see any available patches, even though windows update has found quite a few patches.
Brande, Dirkgent - Windows 10 doesn't have a conventional c:\windows\windowsupdate.log - there is now a powershell cmdlet required to compile a human-readable log of Windows Update activity. See support.microsoft.com/.../3036646
Hi Brande, I'm not convinced the errorlevel code 2 is just 'access denied' - it appears to be a generic return code when the net start (or net stop) doesn't return 100% success. In this case the wuauserv service is already running so it returns code 2 because it couldn't start the service. This is clearly not a valid reason to stop the patch scan process.
As an experiment I tried stopping the wuauserv process beforehand (so that the patch scan process can start it without the error code 2) and now see this in agentmon.log:
execFile(): Path="C:\Windows\system32\cmd.exe", arg=""C:\Windows\system32\cmd.exe"...", flag=0x3, timeout=7200
Here's the results of my testing from the command line:
First as an unelevated user:
Microsoft Windows [Version 10.0.10240](c) 2015 Microsoft Corporation. All rights reserved.
C:\Users\Combo>net start wuauservThe requested service has already been started.
More help is available by typing NET HELPMSG 2182.
Then as an elevated administrator:
C:\Windows\system32>net stop wuauservThe Windows Update service is stopping.The Windows Update service was stopped successfully.
C:\Windows\system32>net start wuauservThe Windows Update service is starting.The Windows Update service was started successfully.
C:\Windows\system32>net start wuauservThe requested service has already been started.
That results code is thrown by MS, and MS reports the code as meaning Access Denied. Doing some additional googling, third parties report that error code may be thrown for several reasons. There are a number of suggestions online on how to determine which 'subgroup' of Access Denied is occurring for a particular case. I can't speak to MS's results or why specifically they use one error code to mean multiple things. They may categorize errors into larger groups - in this case 2 meaning Access Denied (or MS's interpretation of "access denied") but with the specific cause of whatever Help Message 2182 means. What's happening with your tests is not what I'm seeing (more on that below). I would recommend opening a ticket with support to figure out what's going on. Digging into the MS error code might lead to resolution, but if there's a new environmental issue Kaseya needs to account for in some situations, a ticket will be the right way to get the necessary investigation done and the appropriate dev resources allocated.
I'm curious, though, if you set the service to Manual (triggered) and/or stop the service whether the scan will complete successfully. I'm able to run a scan without procedure errors whether the service is manual or automatic, but that's just a single machine and shouldn't be indicative that it will "always" work for others. It might be worth giving both a shot - try setting the service to manual, then scan; try stopping the service, then scan.
Combo and Craig Hart,
I've updated my system to Windows 10. Here's what I see so far:
Kaseya is able to invoke a scan through WUA.api. This is evident based on entries in windowsupdate.log (which you now need to build using powershell to bring together several ETL files. Easy enough to do, but it's a new extra step. This KB article outlines what needs to be done: blogs.technet.com/.../windows-10-windowsupdate-log-and-how-to-view-it-with-powershell-or-tracefmt-exe.aspx.).
In my windowsupdate.log file, I see this entry:
* START * Finding updates CallerId = KPtchMgt2 Id = 2
This is Kaseya invoking the scan via WUA.api. There's an entry several lines later "ending" the scan. The work is all WUA, but the reason WUA is doing the work is based on a request from Kaseya. The file, KPtchMgt2, is actually our .dll file that allows communication between WUA and Kaseya.
I also see ptchscn2.xml created in my working directory. I cleared out the existing file to ensure a new file would create.
The info in ptchscn2.xml appears accurate.
Ptchscn2.xml within the agentguid folder on the KServer matches ptchscn2.xml on my local machine. The file is being successfully uploaded to the KServer.
The info I see in the VSA is in line with what i see in ptchscn2.xml on the local machine/in the agentguid folder. This indicates the file is being successfully consumed by the KServer, and the data properly populated.
My machine is currently up-to-date on updates. When I run a scan locally, the only patches I see reported as Missing are Microsoft Defender definitions. These are unsupported patch classifications for Kaseya (see kaseya.zendesk.com/.../36010308-Unsupported-Patch-Classifications-Definition-Updates-i-e-MS-Defender-MS-Forefront-and-Device-Drivers for more info). Even though they're listed as missing on the local machine, these patches would not appear in Kaseya since they're unsupported. This has been true for all versions of Kaseya, all OSs, and is not new to either R9.1 or Windows10.
From what I see, the scan *appears* to be working successfully.
To do further testing with my personal machines, I'll need to wait until MS releases some updates applicable to my machine. Tomorrow is Patch Tuesday, so perhaps there will be some updates available. I'll do some additional testing once there are some available missing patches.
On my windows 10 box, the Windows Update GUI shows 1 patch pending when a check is run. The machine gets it's updates from a WSUS Server on the LAN. BITS Service is set to automatic (Delayed Start) and running. Windows Update is set to manual (Trigger start) and running.
The above machine worked perfectly with Kaseya under windows 8.1.
I run the patch scan procedure. I note that the ptchscn2.xml is not re-created after a patch scan. No errors in the agent procedures log - the patch scan script appears to run normally.
generating the windowsupdate.log and checking it, the string KPtchMgt2 does not appear anywhere in the file.
I had not "set credential" against this machine. I did so and re-tested. No change.
So, it seems that in my case, Kaseya never successfully calls windows update via the WUA.api.
My agent version is 18.104.22.168 and kPtchMgt2.dll in the agent working folder is file version 22.214.171.124 product version 126.96.36.199 dated 29/07/2015
Not sure where to go from here.