Kaseya Community

Server or Internet down

  • We had an issue over the weekend where our exchange server went down for 10 hours before anybody realized it. This has made us review some of our monitoring. Here is the dilemma:

    We want to alert when a server goes offline, however we do not want to be alerted at 4:00 am because the internet dropped for a few minutes. We want to monitor when a server goes offline and try to ping outside (Google). However, we do not want to ping from the server, and can only select 1 machine per group to run the script on when the monitor is triggered.

    Any ideas on the most effective way to monitor agent offline and then test to see if the internet is up at the location or if the server is down based on the results of the ping? We will be emailing and also sending an alert to an on-call tech via pager.

    Thanks

    Legacy Forum Name: Server or Internet down,
    Legacy Posted By Username: JonJohnston
  • try setting agent status, it'll give you online/offline status of server via internet without needing to ping. But if you want to ping, try pinging from another PC or ping a time server, or your own server

    as for paging you, (email is what we use) you may want to look at this

    http://www.slipstick.com/outlook/smscell.htm

    you can send txt messages from email, so in the agent status you can set a phone sms email and/or a distrobution email. I use both on mine.

    Or if you like, you can sign up for the empower service by Kaseya, they'll call/page/email you till you're awake and throwing the pager/phone against a wall.... very useful, we use it on Top Tier customers

    Legacy Forum Name: Monitor Sets,
    Legacy Posted By Username: darcement
  • The reason for the ping is that we want to be able to determine if the internet is down or if it is the actual server. A server down is a higher priority then internet down.
    Thanks for the reply darcement, but that is almost what we currently have, just trying to make it better. We wanted to ping from another machine, but can only select one machine per group for the monitor, and we feel there is too great a risk of that one machine being offline at the time. Would really like to select 2 or 3 machines. Thanks.

    Legacy Forum Name: Monitor Sets,
    Legacy Posted By Username: JonJohnston
  • hummm, yea, we're in the same boat then. The problem is if the agent goes offline you're not really going to be able to know if your server is down or internet via email alert. May be able to set an SNMP trap on the router and monitor the gateway to email you before connection is lost. I've seen mixed results on this. Does your customer have a 2nd connection? You can set SNMP to push alerts via a dail up line or a spare dsl line. It'll be interesting to see what smarter people then me suggest.

    Maybe in v6 they'll allow more than one agent for ping.

    Legacy Forum Name: Monitor Sets,
    Legacy Posted By Username: darcement
  • We were wanting this for most of our clients, and only one or two have a second internet connection. There are many good ideas on how to do this, however Kaseya really limits what we can set up.
    I love most of Kaseya, but several areas need work. Allowing more then one machine to run a monitor script on would be a great addition.

    Legacy Forum Name: Monitor Sets,
    Legacy Posted By Username: JonJohnston
  • You can monitor for the server being up. When the alert is raised, you can run a script that will ping whatever you want it to (from where ever).

    Now how we handle this is this way:
    -> we monitor a server for online.
    -> if the server is offline for 3 mins, send an email alert
    -> When the server comes online, send an email alert.
    -> We do a constant ping check to the sites firewall as well (using ping check under system check in K). If there's a problem, it'll drop an email as well.

    This way we know when its a server issue, or an internet issue (or possibly a power issue).

    This way you can tell if the

    Legacy Forum Name: Monitor Sets,
    Legacy Posted By Username: boudj
  • boudj

    Now how we handle this is this way:
    -> we monitor a server for online.
    -> if the server is offline for 3 mins, send an email alert
    -> When the server comes online, send an email alert.
    -> We do a constant ping check to the sites firewall as well (using ping check under system check in K). If there's a problem, it'll drop an email as well.


    Sounds like what we were wanting, but how do you handle finding the IP to ping? Do you manually enter the info in or pull a variable to get the gateway? I tried using default gateway variable without much success.

    Any idea on how to differentiate between the server down or internet down for the alert? We are going to be sending to an on-call tech and do not want the pager going off at 3am for an internet drop. If just have the server offline alert the pager then it will also alert if the internet is down. My idea was a script ran from another machine that would try to ping Google or other outside Ip and report, but Kaseya limits a monitor script to being run on only one machine.

    Legacy Forum Name: Monitor Sets,
    Legacy Posted By Username: JonJohnston
  • JonJohnston
    Sounds like what we were wanting, but how do you handle finding the IP to ping? Do you manually enter the info in or pull a variable to get the gateway? I tried using default gateway variable without much success.
    .


    Well... you open ping from your network (public ip of your firewall) to their firewall (this way only you can ping them) and select one of your devices to do a contact ping.


    JonJohnston

    Any idea on how to differentiate between the server down or internet down for the alert? We are going to be sending to an on-call tech and do not want the pager going off at 3am for an internet drop. If just have the server offline alert the pager then it will also alert if the internet is down. My idea was a script ran from another machine that would try to ping Google or other outside Ip and report, but Kaseya limits a monitor script to being run on only one machine.


    If you get the alert that ping failed, and you then get an alert that the server is offline, you'll know why (remember, the alert from server offline has a 3 min delay). However, your tech needs to stay alert to receive the server is online alert (which he'd expect to receive once the Internet comes back online). Now, you do know one thing for sure, the clients internet is down. How about calling the ISP and report a trouble ticket?

    Legacy Forum Name: Monitor Sets,
    Legacy Posted By Username: boudj
  • A way to trigger an IF statement for the monitor would be great.

    Agent status offline, run script or monitor that pings client IP, alert pager if the ping is successfull. If ping fails send email as it is the internet down.

    Oh well, have to make do with what is available to us.

    Legacy Forum Name: Monitor Sets,
    Legacy Posted By Username: JonJohnston
  • This may or may not work for you depending upon if the server acts as an internet gateway or not. I usually put an alert on the server and one regular workstation that is to be left on at all times. This way, if the internet goes down, kaseya sends me an email that states "two or more computers in the ABC group went offline." If it just the server that went down, ill obviously get an alert that just that server went offline.

    Legacy Forum Name: Monitor Sets,
    Legacy Posted By Username: Resistance2Fly
  • I appreciate all of the replies, however they are basically what we currently have.
    It sounds like there is not going to be a way to send the alert only if the internet is up and the server offline?
    If the internet is down then the server is offline, so an alert will be sent to our on-call pager either way. I guess what we can do is only use one workstation (run script from monitor) to ping an outside source and then alert with the details so this would help by whether can ping outside or not.

    Legacy Forum Name: Monitor Sets,
    Legacy Posted By Username: JonJohnston
  • OK, it's a bit of extra work, but how about this:

    Setup a section in the registry on your kserver for each company (could be done with a script I think). You have a key for each company, lets say "InternetUp". 0 is false, 1 is true.

    You have a ping check set up to monitor your clients firewall. If the ping-check fails, you start a setInternetDown script for that company (you may need a seperate script for each company unfortunately). That script sets the registry value on your kserver for that company to 0.

    When a server goes offline, a script is run against your kserver that checks to see what the InternetUp registry value is for that company. If the InternetUp variable is 1, an email is sent to you telling you the Internet is up, but the Server is down. If the InternetUp variable is 0, then no email is sent (or an Internet down alert is sent/logged).

    Not that I have anything like this set up, but maybe someone else can expand on this (or clean it up).

    You'll also need a script to run on a regularly scheduled basis that resets the InternetUp registry key value to 1 for each company once the firewall starts responding to pings again.

    [EDIT] Instead of using a registry key, you could do this with a temp file.

    Legacy Forum Name: Monitor Sets,
    Legacy Posted By Username: ssugar
  • Ok, I might have found a solution that will work for what we are wanting, but what i have means writing 198 different scripts, 2 for each server. Now the scripts are simple but still...

    The script runs from our Kserver when the monitor is triggered
    Script Name: Gateway
    Script Description: Pings machine gateway to check for connectivity

    IF True
    THEN
    Execute Shell Command
    Parameter 1 : ping %client_Firewall%>>C:\utility\ping.txt
    Parameter 2 : 0
    OS Type : 0
    Execute Script
    Parameter 1 : Ping Email (NOTE: Script reference is NOT imported. Correct manually in script editor.
    Parameter 2 :
    Parameter 3 : 0
    OS Type : 0
    ELSE


    Script that is called upon
    Script Name: Ping Email
    Script Description:

    IF Test File
    Parameter 1 : C:\utility\ping.txt
    Contains :bytes=
    THEN
    Send Email
    Parameter 1 : support@domain.com, pagernumber@domain.net
    Parameter 2 : Server offline
    Parameter 3 : The client gateway is pinging but a server is offline. This needs checked immediately.
    OS Type : 0
    ELSE
    Send Email
    Parameter 1 : support@domain.com
    Parameter 2 : Server offline and no ping
    Parameter 3 : The server is offline, and the gateway is unreachable. This is probably an internet down situation.
    OS Type : 0



    What I need is a variable that will pull the machineid.groupid of the machine that triggers the Monitor and passes that variable to the script so that we know which machine is offline. I can use the MachineID variable, but it pulls the Kserver id. If I can get the machineid of the machine that is offline it means I only need 99 scripts.
    Ideas?
    Thanks for all of your help so far.

    Legacy Forum Name: Monitor Sets,
    Legacy Posted By Username: JonJohnston
  • If it was me and I really wanted to do what you are doing ( We are similar to Resistance2fly) I would go the safest route and setup separate SNMP monitoring on the firewall itself with a 3rd party SNMP monitoring tool. This way you know exactly what is going on with the Internet and Server. We are pretty exclusive to SonicWalls which have mibs and GMS that we can setup to show traffic in and out on the different interfaces. GMS is the way to go for us. It will send out emails, pages, smoke signals and all other types of alerts if the SonicWalls go down.

    Good Luck, I am anxious to see what you come up with.

    Legacy Forum Name: Monitor Sets,
    Legacy Posted By Username: TITaN
  • I am very surprised Kaseya doesn't have a way to monitor the gateway IP's. All of the gateway IP's are listed under the home tab. I am going to look into this. It is very important to know the minute a gateway stops responding, as that means an entire network is down (potentially worst scenario depending if the internal lan for the client is still working, but if everything goes through a single router it means the lan is down too).

    Basing everything off a server ping just doesn't provide enough information. I'll post once i find more information.

    Legacy Forum Name: Monitor Sets,
    Legacy Posted By Username: boostmr2