I have a powershell script that should be run within minutes after the server running Exchange 2010 or 2013 has rebooted. The procedure runs an exchange console powershell set of command to test service health of exchange services and if they are not started, to start them.
Now my question is where to best to kick off this procedure after a reboot? I want to run this after the server is back up and all other services are running.
Any thoughts or suggestions?
Thanks to the community in advance and Ben LeValle for his assistance.
Policy Management would be an adaptable way to run this procedure. Off the top of my head, I would consider creating a view for Exchange servers that have rebooted in the last X minutes (whatever threshold you find is just enough - maybe 5 or 10 minutes). Create a policy that uses this view to assign an agent procedure which populate a custom field. Your custom field might be a simple 0/1 string, where 0 means a reboot has not occurred and a 1 means a reboot has occurred.
Create a separate view to filter to machines where the custom field is equal to 1. Assign this view to a policy which runs an agent procedure to execute your powershell script. At the end of the procedure, reset the custom field to 0. Schedule this procedure in the policy to run only one time. Once the custom field is reset to 0, the machine will no longer qualify for the view, so the procedure will no longer run.
Assign both policies to the appropriate organization, machine group, or individual machines. These policies should work such that the first will update the custom field after a reboot, which will cause the machine to qualify for the second view. That second view will allow the policy containing the powershell script to be assigned to the machine, the powershell would run, the custom field would be reset to 0 and the server would fall out of qualification for the policy.
You might find you could accomplish this in a single policy. I would recommend testing to make sure the background timing of policy assignment works for your specific situation - it can take a few minutes for policies to be assigned to machines based on view membership changes.
If the reboots are occurring due to specific actions, you could also assign your powershell procedure to run after that task. For example, if the reboot is due to patch, assign your agent procedure containing the powershell script to run as a post-reboot procedure (configure on the Patch > Reboot Action function). If the reboots are due to an agent procedure that you run, call the powershell AP as a step within the parent procedure that calls the reboot.
If you do Kaseya Community 's Policy method, you might be tempted to bring your Policy Processing interval down in order to catch the machine falling into the View (i.e. View shows machines that rebooted in the last 15 minutes, and Policy Processing takes place every 10 minutes). My advice there would be to not do that--keep your Policy Processing interval high, especially if you have a lot of Policies that need processed! We ran into an issue (mind you, we have dozens and dozens of Policies) where the Policy Processing kept running overtop of the last Policy Processing job--thus a perpetual loop of Policy Processing sadness.
There is probably an Event in the System Event Log that you could key off of--apply a Policy to anything in your Exchange Server view that adds an Event Log Set that watches for a bootup-type Event Log in System... then, have that Event Log fire off an Agent Procedure. Make sure the Event only happens once during boot, or set the Ignore Additional Events for ____ Minutes such that multiple boot events won't fire multiple script instances.
Hope this helps... multiple ways to skin this... wait, I own a cat. That's not very nice :-)
You could also attach procedure to a Exchange service monitor set that can restart a service if it has stopped and execute your procedure with a delay to check that all the services are running and fix it if not.
You just need to make sure you use Suspend monitoring when you do maintenance to avoid triggering the procedure.
Any other options? After a reboot problem is the services appear as if they are running if you look at the services console.
If you run a powershell script for test-servicehealth. you will find some automatic services are not running.
Usually Transport or Edge Transport service. So while the service in the console is set to restart if a failure, it's not registering as a failed non-running service. Running the powershell script a few minutes after a reboot, will force all the services that are automatic to run. They stay running until the next reboot.
We spoke with Microsoft and this is a known bug in Exchange 2013. Similarly there is an issue in Exchange 2010, same particulars.
Therefore we are looking for a way to fire our powershell script about 5 minutes after a reboot on these Exchange servers.
All suggestions are very much welcomed and appreciated. I will respond with an update if after testing we get a positive result.
Another alternative to look at is to run you powershell script from the Task Scheduler using "When the computer starts" trigger, just add a delay to give the server enough time to start up fully.
You can still use a Kaseya script to deploy the powershell script and create the Task Scheduler entry.
One thing that I've done that has helped us with "Cascading procedures" is to configure a task on the server that creates an Event ID that I monitor for. When a server reboots, this event ID is created 5 minutes later which K2 watches for. This kicks off a script on another machine.
I'm sure you could set this up in your environment. The reason I did it this was was that the reboot Event ID kicks off before the agent has booted so we were missing reboots.
For your task, I'd have the server create Event ID 5-10 minutes after successful boot.
Next K2 records said ID and then runs an agent procedure that verifies those services are running.
You can then have another procedure kick off that sends an alert/ticket if services are not started.
I think I've seen several of the posters here mention having something that can look for running services and only create a ticket if it's not started.
I'm not to that point, yet. I'm currently getting emails on services that are being restarted, so I'm getting a lot of noise.