Installed Patch Level: 220.127.116.11 - When I click on "System - Server Management - Outbound Email" the circle spins and then I get a "Timed out" error message. Also, periodically, our KServer will stop sending outbound emails. If I log on to the KServer and restart the Kaseya Event Management Service email will begin flowing again for a day or two until it stops again. I've tried reapplying the schema without success. Anything else I can try short of starting attempts to enlist Kaseya support?
I tried another restart of my KServer which allowed me to click on Outbound Email again without the error. No indication anywhere (System Event Logs - KServer logs, etc) that I can find of what might be causing these issues.
I would definitely get a high priority support ticket in.
Unless there is an absurd amount of emails in pending, I do not see the outbound email queue to be the culprit behind this.
More likely, it is something else causing the event management service to time out - you can check the logs in C:\Kaseya\logs\services, MSMQ to see if you have a large build up of messages within a specific conduit.
Thanks, Nicolas. Unfortunately, the problem is intermittent and I'm pretty sure how this would go with support. We only have a couple hundred emails in the pending queue when it hangs. The Kaseya Event service shows as "running" when this happens but it obviously must be hung as restarting it causes email to begin flowing again.
I see lots of these errors in the Kaseya Event log:
2016-09-12 15:49:20,310 ERROR - Error while aging events
System.Data.SqlClient.SqlException (0x80131904): Timeout expired. The timeout period elapsed prior to completion of the operation or the server is not responding. ---> System.ComponentModel.Win32Exception (0x80004005): The wait operation timed out
at System.Data.SqlClient.SqlConnection.OnError(SqlException exception, Boolean breakConnection, Action`1 wrapCloseInAction)
at System.Data.SqlClient.TdsParser.ThrowExceptionAndWarning(TdsParserStateObject stateObj, Boolean callerHasConnectionLock, Boolean asyncClose)
at System.Data.SqlClient.TdsParser.TryRun(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj, Boolean& dataReady)
at System.Data.SqlClient.SqlCommand.FinishExecuteReader(SqlDataReader ds, RunBehavior runBehavior, String resetOptionsString)
at System.Data.SqlClient.SqlCommand.RunExecuteReaderTds(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, Boolean async, Int32 timeout, Task& task, Boolean asyncWrite, SqlDataReader ds, Boolean describeParameterEncryptionRequest)
at System.Data.SqlClient.SqlCommand.RunExecuteReader(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, String method, TaskCompletionSource`1 completion, Int32 timeout, Task& task, Boolean asyncWrite)
at System.Data.SqlClient.SqlCommand.InternalExecuteNonQuery(TaskCompletionSource`1 completion, String methodName, Boolean sendToPipe, Int32 timeout, Boolean asyncWrite)
at Kaseya.AppFoundation.Servers.EventServer.AgeEvents(IDbConnection connection, IDbCommand command)
Unfortunately, there is no "MSMQ" in the path you specified.
We opened Kaseya ticket #152871for the email hanging issue as it occurred again today.
I requested support management to get eyes on the ticket and coordinate a meeting with you to review further.
PM me if you hit any roadblocks on this.
Thanks again for your help!
Support was very helpful today. She wasn't exactly sure what was causing the issue with the email hanging but she did quite a bit of work cleaning things up and was very helpful. Her assistance is much appreciated. We will monitor for a while to see if the issue reoccurs.
On behalf of Support, thanks for the great feedback. I did see your ticket is on hold to allow some time to monitor for recurrence. I hope the issue is resolved, but definitely do let us know if the issue resurfaces.
Here's an update - They've worked on the issue a few times but have been, so far, unable to resolve the problem. We still have outbound email hanging, time out errors in a log file and are unable to view the "Pending" outbound mail queue.
I'm curious - having "several hundred" emails in the outbound mail queue seems surprising to me. Roughly how many endpoints are sending alerts to your K-server? How many events are processed in a typical day?
Hi gbarnas - the amount of email sent by the system varies. The problem is not the amount of email - which hasn't changed very much in all the years we've been running Kaseya, it's that the system now randomly stops processing and sending these emails so they then start piling up in the outbound email, "Pending" queue. Then, when we notice that we haven't been getting emails from the system, and go restart the Kaseya Event Management service it begins processing the emails again and then sends all the email that has backed up in the queue.
And, to make it worse, when we try to view the "Pending" queue in the VSA it fails with a "timed out" error so we can't even look to see how many emails may be sitting in the queue waiting to be processed...
Latest update - It's looking like support was able to resolve this issue. So far, knock on wood, the outbound mail queue has not hung since support's last foray onto our system. I got my fingers crossed but kudos for the work they've done and the effort they've made to resolve this issue.
If you want to monitor the status of this I created a KB article for something similar:
You can use the following query to track this (picture is from legacy knm version):
Very nice, Nicolas. Thank you again for all of your help and support.