I've run into this before, but never quite to the extent I did over this past weekend, so now it has me questioning if there is a better way to accomplish alerting for patch failures.
We have the patch failure alerts coming via email alerts to our ticketing system (not using Kaseya for ticketing). Overall what I'd really *like* to see for the alerts is some type of "Summary" alert once patching has run. Something like "10 failed patches on machine xxx.customerorg". Instead what the current alerting does is send an email on each and every patch failure. So in the above example I end up with 10 separate emails "Patch xyzkb1213 failed on machine xxx.customerorg". Most of the time not a huge deal, other than the shear noise it generates. However as I say I had an extreme case of this over the weekend. We had one customer site where for one reason or another they were a ways behind on patches, and then there happened to be some internet connectivity issues at the time it was trying to download the patches to apply, so the end result I had 20+ machines sending me 80+ emails each about patches that failed to apply. Ended up with over 2000 emails all about patch failures for just over 20 machines.
So I looked into this a little more, but I don't see any way to do the patch alerts other than one email for each patch that fails. Does anyone else have some suggestions on how to better handle situations like this? Ideally I don't need 1 ticket for every patch that fails, as if multiple patches fail on one machine it's likely that the cause is something at the machine level rather than something with any specific patch.
I would route this through Kaseya's Service Desk... and use that to de-dup... and then have Kaseya send a summary email from the ticket to your ticketing system. If you want specifics let me know.
I thought about that... Have to spend a little more time figuring out Service Desk, since we don't really utilize that... The overall issue with de-dup, is that the "summary" line of each one is a bit different since they basically start with [machinename] <Patch that failed> failed to install.
So with the de-dup stuff in service desk can you "combine" tickets that aren't "truly" duplicates?
Unfortunately with the Patch Alerts at the moment it will always send the message for each Patch install fail on each machine - maybe change this to Create Alarm rather than email and then send one report to your ticket system for review / investigation.
I have seen other customers setup a report to run after the Scheduled Automatic Patch - maybe the day after so if your Patches run over the weekend maybe schedule a report for Monday morning. This could be emailed to your ticketing system, the report can be grouped by machine - it would list the failed patches under each machine name.
Here's one suggestion:
Disable patch alerts (optional)
Create a View to show failed patches:
Create an Agent Procedure to send an email. If you don't care about the actual number of patches failed on a machine, just the fact that the machine reports failed patches, a simple sendEmail procedure would be sufficient. If you want to know the number of failed patches for a given machine, you would need to use the sqlRead step to query the database to check the actual failed count. If you choose to use this option, I recommend you review the sqlRead information in the help file: http://help.kaseya.com/WebHelp/EN/VSA/9010000/index.asp#11625.htm. You will need to query the PatchStatusTotals table (Failed field).
Create a Policy to use the above view and assign the agent procedure to the policy. Schedule the procedure within the policy to run once.
Assign the Policy with the defined Agent Procedure to your Global organization (or to a specific machine group/org if you want to limit which machines you are notified about).
When a machine qualifies for the "FailedPatches" view, the policy will be assigned. The policy will assign the AP to run at the next scheduled interval. The AP will run once. You would need to be sure to correct all failures before your next install cycle so the machine "falls out" of the FailedPatches view. That way, if any patches fail in the next cycle, the machine will re-qualify for the view and be reassigned the AP to run one time. If the machine stays in the "FailedPatches" view, there won't be a 'trigger' to reschedule the AP, since the AP would have already run once.
Kaseya Community - That seems like it could accomplish what I'm looking for, I'll have to look at it a little closer. ghanssen I'd also be interested in your idea of using Service Desk if you could give me a few pointers there.
Wow! That is cool. Thanks for that tip.
It's definitely possible - our platform uses Service Desk to consolidate these individual events. Any patch failures coming from the same host within a specific time period (currently 1 hour) are summarized into the first ticket, with all others discarded after obtaining the patch ID that failed. Thus, 1 ticket per machine. The ticket that remains has a list of all of the failed patches in the body. We do these as tickets for servers, but use simple reports for the workstations.