Documentation Deduplication Project Overview

Duplicate information eats away at the trust your staff have for the information systems you have in place.

Duplicates kill trust in a system faster than anything else. The first thing most of us think when we see a contact list with 8 entries of the same contact that are all slightly different is to discount it any further and ask someone for the details or look for an old excel staff list. Both of which circumvent the information system you have in place that is meant to make things easier.

Duplicates happen for many reasons, the systems may not have been integrated using best practices, ongoing maintenance may be non-existent or incorrect information could have been entered in the wrong places to name a few. 

Below are a range of 100% human created articles for your viewing pleasure.

MSP Backup Related MSP Consulting Related MSP Cyber Insurance And Risk
Backups Provided As A Service MSP Agreements Cheaper Than Internal IT? MSP Liability Does Not Care About Effort
Office 365 MSP Backup Options IT Documentation Horror Stories Backup Verification Explained To Clients
Openfiler Superior To TrueNAS? Profitability Facts For MSPs Expectations Of Clients And Backups
Managing Non Contracted Clients Starting An MSP From Nothing? Client Refusing Advice
Tape Backups Pointless? Kaseya And Their Poor Behaviour Policies Of Backup Retention
Fake Enterprise Drives From Amazon Hardware And MSP Buying Stats Terminated Employee And Data Retention
NAS Superior To Backup Appliances? HALO PSA As Good As They Say? Ransomware Incidents And MSP Requirements

Types Of Information Duplication

Below are some of the examples of information duplication that commonly exists within managed service providers when we first undertake a deduplication audit.

Contact Duplication

This occurs when you have multiple MSP systems that have not been integrated correctly. I have personally seen clients operating with three or more identical contacts in IT Glue and Hudu.

Like human beings everywhere, they get used to the status quo. The longer you leave a contact duplicate situation, the worse it gets because your help desk people are likely to randomly assign tickets to one of the 3 options meaning that if it is left for too long, it can be quite an ordeal to rectify.

Generally contacts will sync with your PSA such as Connectwise Manage and if not set up correctly or by people not experienced, they will often create contacts in ITGlue or Hudu instead of allowing contacts to sync both ways. So that can end up giving you two contacts that are identical.

You can also end up with three or even 4 duplicates of the same contact if you do not set up the Office 365 integration correctly as well as integrating your RMM monitoring tool correctly as well. 

I get some people do not use contacts in their documentation integration application, all I can say in this situation is that if your IT documentation program is not contact centric then you at most are getting perhaps 40% effective use out of the application.

Invest your time in getting your contact duplicates eliminated from all applications. Your documentation program should not be your application of truth. Connectwise Manage application or other PSA should be the program where contacts flow from.

You can add contacts from IT-Glue and they will sync to most PSAs; however, you should always treat the PSA as the more accurate copy of your contacts. Delete any unwanted client contacts from your PSA and then check for orphans in ITG and delete those.

I strongly recommend that you ensure that the Office 365 integration aligns with your PSA contacts and they flow through to ITG. Until this is completed successfully, it is best not to rely on contacts in your documentation platform.

Location Deduplication

This is another big area that organizations believe is not that big of a deal. Rarely do we see MSPs that have implemented an accurate and consistent naming convention for client sites. It is almost always left as “Main” 

I find this odd considering that after a specific contact the next question we have is “where are they located?” Good luck when a new hire has to guess from memorable names like “warehouse, second site, foothills office or 3rd floor office

Locations need a consistent naming convention across all clients and they must be created in the PSA rather than IT Glue or Hudu. Yes you can create a location in ITG however it has always been buggy and often will create a duplicate and then sync twice.

Other applications that are location centric such as Meraki need to have some thought put into them so that they align with the naming convention of ITGlue otherwise you will end up with multiple locations that are the same place.

Ensure that your locations in Connectwise Automate or other RMM are identical to the locations in both CW Manage and IT Glue. Make sure that the PSA is the source for all other integrated applications.

Sometimes people group devices under device types and use the location for that in Meraki. I recommend not doing that because the location should be used for the location, it is better to break those devices up into separate groups based on their physical location rather than having 2000 wifi access points in a group called WiFi APs.

If this all seems like too much then I encourage you to contact us as we are experienced in untangling duplicate and mismatched information.

Deduplication Of Configurations

The most common problem I see with configurations is when the integration is set up incorrectly. This can be quite a serious and time consuming problem to rectify because people will sometimes enter extra information into configurations manually as well as tag configurations to flexible assets and passwords.

I have seen some clients that will have north of 30,000 duplicate configurations all with manual information added and having been tagged to various assets. Significant amounts of time are often invested in linking up duplicate assets that require even more time to untangle.

So picture looking at up to 4 server configurations that are all identical then going into each one and seeing varying amounts of related items attached to each one. Which one do you keep?

There is no way to merge configurations, you have to delete the ones that you do not want. While ITG allows you to backup a flat excel spreadsheet, it does not export the related items information or tagged information, you lose it all.  Now picture doing that 5000 times?

See how it becomes almost an impossible task to rectify this without burning everything to the ground and starting from scratch. Thankfully the organizations that tend to have large issues with duplicate configurations often have not used ITGlue to its full potential and so there is not a huge number of related items we have to worry about.

Configuration Duplicate Solutions

Our recommendation in this situation is to either call us for assistance and failing that, export the csv from ITG and look for patterns as to where the duplicates are coming from. The biggest culprit is usually your RMM tool such as Connectwise Automate.

The biggest configuration mistake I see is when the configuration integration is set up both ways between the PSA and both ways between the RMM. This is a recipe for duplicate city and deleting them will not matter, they will come right back during the next sync.

My recommendation is to only sync 2 ways between ITGlue and Connectwise PSA coupled with syncing from Connectwise Automate to CW Manage. This ensures that chances of duplicates are heavily reduced. Nothing should be syncing configurations back to the RMM.

Configurations Outside An RMM Do Not Exist

This is a problem I often see and it means you put your MSP organization into a corner and it means any work undertaken with configurations is labor intensive because you cannot just hit the delete button and expect all of your configurations to come back. 

If a few percent of your configurations are manually created then it means having to review every single duplicate configuration record to check if it is automatic or not.

Documentation systems are meant to make life easier. If you have spent money on an RMM system then use it. If the configuration does not exist in the RMM so that it can be synced via the PSA and onto ITGlue then DO NOT manually create the configuration.

Fix the problem on the RMM side. If the RMM does not detect the device then fix the issue with SNMP or install the probe in the right location. Do anything except add a manual configuration record because it is the least automated thing you could do and is completely impractical and there is no ability to keep track of these configurations which are hugely labor intensive to create.

If you take anything away from this article, I hope it is a dislike of manual configuration entries. You should be able to delete every single configuration from IT Glue and feel comfortable that every single record will sync back at the next sync cycle.

Office 365 Tenants Must Match Connectwise Organizations

I have seen enough examples of mismatches between O365 tenants and other integrated systems within an MSP to know you guys cut corners on the documentation side whenever you are asked to make a change.

So Company-A buys Company-B or Company-B breaks off into Company-C and D and you make those changes in Connectwise manage and you give the client a good price on the change if you even charge them at all and give yourself a little pat on the back.

Only you did the tasks that are easy but left out the important task brushing them off as not important and I suppose to your client they probably are not that important because it is you that is going to be the one exposed to the inefficiency caused by these serious oversites.

I have spoken about this before and strongly recommend that you charge your clients appropriately for documentation work and that includes any work where there is a change to their company structure.

I realize it's a hassle, is labor intensive and  you do not see the upside but for the love of god, if your clients split into two companies or merge with another then modify their O365 tenant to reflect the change.  

The wasted time having to explain to every new hire that the contacts in ITG under company-A actually are for Company-B in CWM is infuriating especially after they may have spent 20 minutes trying to figure the problem out for themselves, not to mention the potential for complications when it comes to your cyber security insurance policy if your client entities are a jumbled mess.

There is no excuse, either use your documentation platform and invest the time so it will save you time or run the support desk out of notepad and sharepoint. No half measures!

You have to ask yourself, if nobody is using the documentation platform your organization is paying a substantial amount for, what is the point in it beyond a glorified password manager?

It has the capability to be so much more but for that to happen, deduplication of records and elimination of orphans is an absolute must.

How We Handle A Duplicate Elimination Project

We recommend engaging us as part of the documentation remediation step, to confirm all systems are correctly integrated. Once confirmed and after a full backup is undertaken, we go through and clean up all duplicates and orphan records in the system. There can be some complexity involved with some systems. 

We have created a set of deduplication templates we have developed over the last 10 years that helps us step through the various scenarios you may see yourself in.

While a deduplication project can be undertaken as a standalone documentation project, there are often areas that may need to be rectified as part of this project and we will need to determine those items as part of a documentation remediation audit prior to any remedial work taking place.

Below are a range of 100% human created articles for your viewing pleasure.

App Articles Backup Articles Consulting Articles App Articles Hardware Articles ChatGPT Articles Security Articles
MSP Application Articles MSP Backups Articles MSP Consulting Articles MSP Diagram Articles MSP Hardware Articles MSP ChatGPT Articles MSP Security Articles
Cloud Apps BaaS MSP Vendors Diagrams Guides Thin Clients ChatGPT And MSPs MSSPs
Application Stack Cloud Backups Legal Articles General Diagrams PC Cabinets ChatGPT Apps Security Apps
All MSP Diagram Articles Local Backups Consulting Finance Diagrams App V App Firewalls ChatGPT Overview Security Incidents
All MSP Hardware Articles Backup Strategies MSP Employees Diagram App Reviews Network Switches AI And Powershell Security Resources
Monitoring Apps Backup Vendors MSP Documentation Specialist Diagrams Hardware Servers ChatGPT And Log Files Security Services
Operating Systems Backup Applications MSP Clients Visio V LucidChart Rackmount Computers AI And Documentation Security Stack

Want More Information?

Our team specializes in strategies for IT Support Providers and we assist in improving profit margins through standardization and consistent record keeping strategies, so you can be confident that our content is tailored to your needs.

Please feel free to explore our other articles and click on any that interest you. If you have any questions or would like to learn more about how we can help you with your documentation needs, please click the "Get In Touch" button to the left and we will be happy to assist you. Thank you for choosing us as your trusted source for technology documentation.

Contact Us

We guarantee that you will be speaking with consultant with extensive experience in the area of MSP Documentation. Using the chat will be the quickest way to obtain answers. Alternatively, you can email us using the button below.

Email Contact