Kaseya Community

Event log and performance log data retention - this may interest those who like SQL or consider themselves to be analysts

  • Hi guys and girls, 

    I have been deep diving into the Kaseya Database recently and making some cool pivot tables in Excel. 

    Really putting my analyst hat on, i am looking at data that i'd want to see if i was a customer, it turns out, as we probably already knew, the Kaseya Database is a gold mine for data. 

    As an I.T Manager, if i had this information at hand, i could plan well into the rest of the decade, however there are problems with the data as i see it.

    Is the data that gets stored in the database "historical enough", that is, does the data go far back enough to make any real business decisions. 

    Consider the following screenshot, data only gets kept for between 7 - 15 days. It is good data, but not long enough. 

    I have a few questions, keep in mind what i want to do, what i want to size up, is a KServer that can hold 180 days of data, if not more, a whole year would be good. 

    Does anyone here keep data for such periods of time?
    Do you experience slowness in your VSA like we did when we had more data being collected?
    Does anyone have a "second database server" that might hold and archive all the data?

    I was thinking about shipping the logs off every 7 days to a second server to keep building the data without affecting our Kservers day to day operations - Do any of you do this? 

    Take this as an example of the sort of data i want to trend better on, it is all the processes that are sending and receiving data, and at what rate, on what day.

                

    This data i consider to be valuable, when a customer says "I want to know why my server is slow" It is a very finite place to look 

    This data doesn't show up graphically in Kaseya very well if at all, i use Excel and query the database live, but that isn't my criticism, i want more data, the second picture shows each service and a day by day break down (with pivot tables i can show it any which way i want). Imagine having that data over a 12 month period, if i thought my server could handle it, i'd increase the data retention. 

    One month out of the end of the financial year i can say to my customer "Hey, your SQL box has grown 150 percent in 12 months, but it isn't just data, watch the individual process load on the server, look at the EXACT bits and bytes coming in and out, that is what capacity planning is. As a company, i am not interested in simple server box dropping, i want to project what is right for the company off real facts, and i want those facts from a massive amount of historical data, not just 7 days. 

    Please someone tell me it is just me, and that there is a justifiable reason for not having the data there

    As far as i am concerned, an MSP shouldn't be concerned with just break fix, the only thing 7 - 30 days worth of data is for is break fixing - defeats the purpose of being an MSP full stop.

    Anyway, enough for now, what does the community think? 

    Long story short - lets talk about data retention and capacity planning, i am VERY interested to know your thoughts ye' olde' community

    Another one of my famous rants over, Mark.

  • We had an 'issue' during an update some time ago and although we have a 30 day policy set eventually found ourselves able to report on data from over a year ago so it's definitely possible.  This box had close to 10k agents on it so the database was large but everything worked.  We did/do get slowdowns but it's usually when audits are taking place.

    I'd like to be able to pull the Exec Summary data out into a table which I could then report on historically.  You could then add things like weekly/daily disk capacity to project how long a server will last etc.

    Spoke to one of our SQL devs about this a while back and he reckoned we could create a data warehouse from the data then run our reports on that to our hearts content and there would be no real reason to have to purge the old data, disk space allowing.

  • I think it would be very helpful as well to be able to keep historical data for much longer periods than 30 days.  That is one of the main reasons I never really found Kaseya's monitoring worthwhile back in the v3 or 4 days or maybe even earlier.  I pretty much abandoned it for long term because I was told back then that it would not keep data past 30 days.

    I may not be an MSP but it is still vital to know how my servers have been performing over the last 3 months, 6 months and so on.  It would also be very handy to be able to report on things like drive space use over the last year to know when am I going to run out of space on certain systems or how much has that file server grown over that last year.

    Unfortunately I do not have an answer to this but am very much interested in it as well.  I have just started to use the new Kaseya Network Monitor but am not sure yet what all it will do and how long it will keep things for yet.

    It would be nice if Kaseya could tell us the best way to go about keeping their information for longer periods of time and what the best practice would be for that so that we do not cripple our production servers.

  • @Alistair, i have logged a job with my Systems guys to get this happening.

    The problem with this is identifying the tables we need to roll over into an "archival database" If any of that data is presented to the VSA through a stored procedure then it's not just some maintenance plan to roll the data over. I will prepare a project plan to get it done internally and see

    @ Roy

    Your input is much valued, i am going to seek an official answer from Kaseya and see if i can get it to their devs for comment. There is no reason we can't do this. Our KServer is in a datacenter with an SSL VPN up 24/7, i'd quite happily have the traffic flow over the VPN every Sunday night adding the previous weeks details into an offsite DB

    I'll keep everyone posted on whether it is possible / we end up doing it.