So, it's "Happy Patch Tuesday" everyone!
Logged in this evening to go through my commensurate list of evaluating 70-odd patches for approval, and found that a very large portion of them (in the two most critical sections, about 90%) are listed as "Internet Based Install Only".
So...
How exactly do I patch this month, given that I can't approve around 40 rows because they'll all fail?
Oddly, in Kaseya 2008 (we're in the middle of migrating systems from the old Kaseya VSA to 6.x), they're all listed with their normal commandlines. It's just K2/6.x that lists all these same patches as "Internet based install only."
@DNeuwir Watch out for this. We've had this problem a few times over the last year, We also have some 5.1 servers and we check the patches across all servers before approving. I denied a list of patches that were marked as internet based install only to find the next day they were no longer tagged this way, We now leave them a day or two, depending on the critical level of the patch, in case our Kaseya server is having a moment.
I did log a call with Kaseya showing that two patches on different servers were the same but I don't remember how the issue was resolved.
true.
Specially those .net patches are causing lot of trouble from last few days..
There is a manual process on the Kaseya side that provides every KServer around the globe with the download link for each new patch. The 30,000-foot-level overview is:
Kaseya receives reports about newly discovered patches
Kaseya gathers the download link (for those patches that MS has provided a distributable file) and posts details to a file
VSAs run a system-scheduled process multiple times per day to retrieve new file with download link information
Patch information is updated in the db to include the dl info
If you're running patch scans daily, you'll often see a larger number of these internet-based install only patches on patch Tuesdays, particularly earlier in the day. Because the individuals who do the manual portions of this process are based on the US West Coast, later in the day (Pacific time) or by Wednesday, you should see the numbers of the internet-only installs is greatly reduced. The days following patch Tuesday will often have high numbers of new patches discovered each day, but that generally dwindles to just a few new patches (speaking from a world-wide perspective, not your individual VSA) by the Thursday or Friday following Patch Tuesday (adjusting as appropriate for those servers installed on the other side of the International Date Line from US West Coast).
Some patches will remain as Internet-based install only if Microsoft has not provided a suitable distributable version (.exe/.msi/.msu) version of a particular patch.
Okay, I get the process detailed above. But there's some inconsistency. First, it's now Wednesday night at roughly 9:30PDT. I've seen only two additional entries change to "installable" by our 6.x VSA. This leaves roughly 50 patch lines that are still listed as "Internet Only". But what's more odd is that our K2008 (5.1) VSA has, since last night, had virtually every single patch line "live" and installable now. This marks a difference between K2008 and K2.
The unfortunate issue is that basically patching is useless this month for our K2 nodes. We're at T+36 hours since these patches have been released, and have not rolled much at all. My concern is that we're starting to get e-mails from users wondering why their Windows Update says they need updates and why aren't we doing them---"Isnt' that what I'm paying you for" is NOT something any IT company ever wants to hear from their clients.
I'm almost at the point where I'm ready to just set "Use Windows Update" and have all the nodes just update themselves, which kind of leads me to wonder if this is going to be the new trend. The reality is right now, Windows Update works, and Kaseya update doesn't.
@Alistair, Thanks for the heads' up. Actually, we're just leaving them in an "un-approved" (Pending) state. But it's odd that our 5.1 VSA has gone "live" with all these patches, and the 6.0 VSA still shows "Internet Based Install Only" on the same patches.
Also seeing a lot more of these over the last two months.
Okay, now at T+3 days, and we're still seeing (almost) all patches from this week at "Internet only..." Trying to figure out why I'm paying Kaseya so much money for features that don't work.
All,
Any patch that is tagged as Internet-based install only can be installed by Kaseya (as long as the patch is approved within your VSA/for the patch policies you're using). When a patch is tagged this way, Kaseya will leverage the Windows Update Agent to install the patches. This is the same method used if your File Source configuration is set to "Download from Internet" and Kaseya's Automatic Update function is used to install the patches. If you are using this configuration, then all patches will install using the Windows Update Agent (WUA) based on specific requests from Kaseya through the WUA.api.
If you are using a LAN share or the System Server as your patch source, then any patch that is NOT tagged as internet-based install only will be downloaded to the defined file share. That patch will then be distributed to the endpoint for installation. Patches that are targeted to this same endpoint that ARE tagged as internet-based install only will be installed using leveraging WUA through the .api.
In short, whether you're using a LAN, the System Server, or Download from Internet, those patches tagged as Internet-based install only can install, provided the endpoint has sufficient access to the internet. If the endpoint does NOT have sufficient access to the internet, then these patches would not likely be discovered as missing since Patch Scan (which detects missing patches) requires access to the Microsoft websites to determine which patches are missing from and applicable to the endpoint.
Still, there should only be a limited number of patches tagged as internet-based install only. A few months back, Kaseya introduced a content delivery network (CDN) to better improve the availability of files and services for customer around the globe. The patch files that include the location updates have been updated on the primary site, but have not been replicated by the CDN the past few days. We are currently working to update the patch location files on the CDN, and we are also looking into what is preventing the CDN from refreshing these files for availability to servers around the globe. A background process runs on VSAs every four hours, so each VSA will automatically receive the updated files once the CDN is updated and that background process runs.
Patch Tuesday will inevitably have a large number of internet-based install only patches, but by Wednesday, these should significantly diminish. Some will always remain because Microsoft does not release a distributable version of every patch (WUA installation is required), but, in general, that should be no more than 10%(ish) of all patches known to your VSA. If you see behavior other than this, it can indicate an issue with your VSA or, as in this case, with the CDN, and I encourage anyone seeing this sort of issue in the future to open a ticket with Support so we can investigate the underlying problem. I try to remain fairly active on the forums, but there are times when general workload keeps me away for a couple of days.
I'll post here once I have confirmation that the files have been replicated out to the CDN.
Thanks,
Brande
The files that contain location information have been updated on the CDN. The number of patches tagged as internet-based install only should reduce to standard levels within the next four hours on all servers. Depending on where your VSA is in the four-hour cycle, you may have received the updated files already.
Additionally, the CDN issue has been determined and resolved. This should not occur again in the future.