Kaseya Community

PERC RAID card issues with DELL PowerEdge 2900 servers with AVG installed

  • Another Win2k8 box with PERC6 having issues after KES update script runs, i'm doing more investigation, I'm not sure if the script is causing the reboot or the server is locking up and rebooting.



    http://community.kaseya.com/api/forums.ashx/forums/94/threads/9598/posts/47516

    Legacy Forum Name: KES,
    Legacy Posted By Username: cnwicsurrett
  • I am up to 3 Servers with all the same issues - Dell 2950 Server 2003 was the first -- froze on reboot when AVG9 was updated. On hard reboot - it did get in to give Firmware Errors with PERC5i --- On reboot -- froze again - work with Dell support for 4 hrs and they sent replacement PERC5i card t - no fix. We restored the C partition from backup (that had AVG 8.5) and were fine ... until we updated again to AVG9 ----remove AVG all fine. Second machine Server 2008 DC -- it rebooted in middle of day - to freeze on logon with applying computer settings. Since we know the issue -- we reboot to safe mode and remove AVG. Third Server - Dell R710 with PERC6i card -- reboot by itself in middle of day to freeze at logon - same procedure and all fine. We had updated all SErver to AVG 9 and all of our client have Dell SErver 2950, 2900, R710 --- and NOT ALL were affected with this issue but we now have removed AVG from all until we get a good answer from Kaseya and AVG on why this is happening. I have left a detail ticket with Kaseya on this.. .its been one week with NO ANSWERS - how can i get the 8.5 back on the machine for now??? As Kaseya says thats impossible ( i have a hosted solution with them) Any help on how to make that happen would be appreciated!

    Legacy Forum Name: KES,
    Legacy Posted By Username: Light Speed
  • Light Speed
    I am up to 3 Servers with all the same issues - Dell 2950 Server 2003 was the first -- froze on reboot when AVG9 was updated. On hard reboot - it did get in to give Firmware Errors with PERC5i --- On reboot -- froze again - work with Dell support for 4 hrs and they sent replacement PERC5i card t - no fix. We restored the C partition from backup (that had AVG 8.5) and were fine ... until we updated again to AVG9 ----remove AVG all fine. Second machine Server 2008 DC -- it rebooted in middle of day - to freeze on logon with applying computer settings. Since we know the issue -- we reboot to safe mode and remove AVG. Third Server - Dell R710 with PERC6i card -- reboot by itself in middle of day to freeze at logon - same procedure and all fine. We had updated all SErver to AVG 9 and all of our client have Dell SErver 2950, 2900, R710 --- and NOT ALL were affected with this issue but we now have removed AVG from all until we get a good answer from Kaseya and AVG on why this is happening. I have left a detail ticket with Kaseya on this.. .its been one week with NO ANSWERS - how can i get the 8.5 back on the machine for now??? As Kaseya says thats impossible ( i have a hosted solution with them) Any help on how to make that happen would be appreciated!


    Maybe a call into Jeff could get it to you?

    Legacy Forum Name: KES,
    Legacy Posted By Username: Coldfirex
  • I worked with K Support, and they finally got AVG SUpport to agree to work with me directly. Unfortunately, I have been swamped the past few days, and have not be able to find the time to work with them (I usually complain about K Support being slow respinding, they were originally slow in this case, but now I am the holdup.

    I hope to have time on Monday, Tuesday at the latest.

    Legacy Forum Name: KES,
    Legacy Posted By Username: lwolf
  • An update on 1 of the 3 problem servers....

    About a week ago, we reinstalled AVG on 1 of the 3 problem servers. The server had been crash free for more than a week, since uninstalling AVG. Within 2 days of reinstalling, the server crashed again. AVG Support told Kaseya Support to tell us to remove Link Scanner. I didn't see how the Link Scanner would cause the server crashes, but I complied. (We had NOT installed another of the other addon components - like Web Shield, Office Addin, Email Scanner, ID Protection or Firewall. But Link Scanner is part of our standard deployment settings, so it usualluy gets installed on servers). We uninstalled AVG, rebooted, then reinstalled without Link Scanner. The server has been running for 6 days now, with no crashes. So, it seems likely that Link Scanner was related to the crashing.

    They said that Link Scanner interacts somehow with the TDI drivers. I heard of TDI but was not very familiar with what is does. According to wikipdeia "The Transport Driver Interface or TDI is the protocol understood by the upper edge of the Transport layer of the Microsoft Windows kernel network stack."

    Somehow AVG Link Scanner must really get tied-up in TCPIP, and perhaps that is causing the crashes?

    Anyway, I am going to wait another week. If there are no more crashes, we will reinstall AVG on the other 2 probelm servers, and see what happens.

    Lloyd

    Legacy Forum Name: KES,
    Legacy Posted By Username: lwolf
  • Hi guys,

    Just chipping in with our recent experience after a fairly torrid week...

    We've had 4 servers experience lock ups this week, including our own. Symptoms include the machine still responding to PING, and responding to HTTPS as far as presenting the certificate (non-trusted CAs, browser warning shows it has received the cert details but once you click continue you get no further). On our own machine where we have eyes-on, there was no response to any keyboard (including ctrl alt del) or mouse input.

    The lockups have all happened within moments of a KES Update hence the conclusion that this cannot just be a coincidence and my visit here....

    The machines include a PowerEdge 2900 (Perc 6/i), HP Proliant ML110 G5 (Adaptec RAID), Dell PowerEdge R300 and Dell PowerEdge R200

    These are 4 of only 9 servers that we've upgraded to KES AVG9; it may or not be relevant but these are all SBS / Exchange servers. The other 5, which have experienced no lock ups, are not. All our other servers (not upgraded to v9 and including a number of Exchange/SBS boxes) have been just fine.

    Given the existence of this thread and the lack of any much response I'll be uninstalling KES too

    cheers
    Lee

    [edit] Oh, and we don't use Link Scaner on any of these boxes (only base real time scanner + exchange option)

    Legacy Forum Name: KES,
    Legacy Posted By Username: leeevans
  • Lee,

    Thanks for sharing the information. Sorry to hear you are havign problems, but it makes me feel a little better that I am not alone.

    For us, I was thinking that problem seemed to be related to Link Scanner. But from your notes, I see that you don't have Link Scanner on these machines, so I suppose Link Scanner is not the cause.

    One note... remember the bug in AVG uninstall that immediately forces a reboot of the machine. So be prepared for an immeidate server reboot whenever you uninstall KES.

    One question.... Is the OS on these machines 2003? or 2008? And are they 32-bit? or 64bit?

    Lloyd

    Legacy Forum Name: KES,
    Legacy Posted By Username: lwolf
  • leeevans
    The lockups have all happened within moments of a KES Update hence the conclusion that this cannot just be a coincidence and my visit here....
    Lee


    Lee,

    In talking with my ServiceDesk staff, they too reported that many of the server locks seem to have happened at the same time as the KES Update script running. That might be the issue.

    Perhaps Jeff could comment on what actually happens behind the scenes when this script runs?

    Lloyd

    Legacy Forum Name: KES,
    Legacy Posted By Username: lwolf
  • Hey Lloyd,

    Between the 4 servers, 2 are SBS 2008 (so 64bit / 2008 / exchange 2007) and 2 are SBS 2003 (so completely the opposite, 32bit 2003 w/ exchange 2003)

    Same issue repeated today with both the 2008 boxes. Again the lockups experienced only moments after KES update

    I've removed KES (didn't experience an auto-reboot though?) and if they all stay up for the rest of the week I think it's pretty conclusive.

    Cheers
    Lee

    Legacy Forum Name: KES,
    Legacy Posted By Username: leeevans
  • We might have run into this as well over the weekend. A SBS2008 box went offline right after a KES update script was ran. The server was pingable, but down in Kaseya, and would not accept any input at the console.

    Legacy Forum Name: KES,
    Legacy Posted By Username: Coldfirex
  • Coldfirex
    We might have run into this as well over the weekend. A SBS2008 box went offline right after a KES update script was ran. The server was pingable, but down in Kaseya, and would not accept any input at the console.


    I re-opened my support ticket with K Support (it had been closed). I am trying to find out more details about what happens whenever the KES Update script runs.

    Legacy Forum Name: KES,
    Legacy Posted By Username: lwolf
  • Coldfirex
    We might have run into this as well over the weekend. A SBS2008 box went offline right after a KES update script was ran. The server was pingable, but down in Kaseya, and would not accept any input at the console.


    I've had 10-15 servers lock up on reboot this week alone after doing a "KES Update" These servers seem to reboot just fine but the local console shows the computer being stuck at "Applying Computer settings"

    Legacy Forum Name: KES,
    Legacy Posted By Username: cnwicsurrett
  • cnwicsurrett
    I've had 10-15 servers lock up on reboot this week alone after doing a "KES Update" These servers seem to reboot just fine but the local console shows the computer being stuck at "Applying Computer settings"


    1) cnwicsurrett-> Can you tell me if Link Scanner was installed on any of these servers? And are they running Windows Server 2003? or 2008? And are they 32-bit or 64bit?

    2) To all-> FYI, K Support was able to get me setup with a remote control and text chat session with AVG Support (no telephone, ugh). AVG Support remotely connected to two server that have locked up over the past week. One server is running 2008 64-bit without link scanner, the other is running 2003 32-bit with Link Scanner. Both locked up after an AVG virus def udpate. AVG Support grabbed a bunch of log files, and is suppose toget back with me in 1-2 days. I will post back whenever I hear from them.

    Legacy Forum Name: KES,
    Legacy Posted By Username: lwolf
  • cnwicsurrett
    I've had 10-15 servers lock up on reboot this week alone after doing a "KES Update" These servers seem to reboot just fine but the local console shows the computer being stuck at "Applying Computer settings"


    The three that I am currently looking at are (our internal servers are all 64bit 2008 w/ Link scanner that also had this issue but we have since removed KES):

    SBS 2003 SP2 (32bit) - No Link Scanner
    Server 2003 R2 SP2 (32bit) - No Link Scanner
    Server STD 2003 SP2 (32bit) - No Link Scanner

    We have KES installed on 350+ servers, our NOC is trying to reboot a number of them every night to comply with the last update that took them all to "reboot required" status. if we reboot 25-30 servers a night, around 10-20% of them will never come back up with the last thing ran is KES Update and those server hung on "applying computer settings"

    Legacy Forum Name: KES,
    Legacy Posted By Username: cnwicsurrett
  • lwolf
    1) cnwicsurrett-> Can you tell me if Link Scanner was installed on any of these servers?


    I've removed link scanner from all servers under KES management.

    Legacy Forum Name: KES,
    Legacy Posted By Username: cnwicsurrett