More than 50% of data migration projects will exceed budget and/or harm the business due to flawed strategy and execution, according to Gartner. Nevertheless – data migrations are a fact of IT life.
They can be complicated and time-consuming – and regardless if you are going cross-platform, cross-domain, or adding new systems, at the end of the day you’ve got to get it right.
What is Data Migration?
Data migration is the process of transitioning any kind of data from one system to another. You could be getting a new storage appliance to replace an old system. You could be migrating to a cloud-based storage solution. You could be upgrading an application database and need new hardware. Any of those situations require moving data from one system to a different system.
There are several factors to consider while planning and executing a data migration:
- Data Integrity
- Business Impact
- User Experience and Impact
- Potential Downtime
- Data Assessment
- Data Quality
Types of Data Migration
- Storage migration: Storage migrations focus on moving data from one storage device to a new or different device – on premise or on the cloud. These are the most straightforward types of data migrations on the surface, but that doesn’t mean you can just copy and paste a 5TB folder to a new drive. You need to plan and execute the migration to ensure success. Keep in mind when migrating sensitive and critical data, it’s especially important to understand what data is moving where, and who can (or should) have access to it.
- Database migration: Database migrations are required when you need to upgrade the database engine or move the database installation or the database files to a new device. There are more steps to a database migration than a storage migration, and you need to plan a database outage to perform the migration. You need to backup the databases, detach the databases from the engine, migrate the files and/or update the database engine, and then restore the files to the new database from the new location.
- Application migration: Application migrations can be some combination of the two options above. Applications can have databases, and they can have installation folders and data folders that need to be migrated. Application migrations may require additional steps per the application vendor.
Tips to Make Your Data Migration Seamless
It’s important to protect your data during a data migration: moving sensitive and critical data can be a delicate task – it’s important to make sure your data migration is seamless.
- Create and follow a data migration plan: Determine what data needs to be moved, how data should be moved, where it’s going, and who should access it. Set up a data migration plan that outlines each step, considering who will be affected, what the downtime will be, potential technical or compatibility issues, how to maintain data integrity, and how to protect that data during the migration.
- Fully understand the data you’re migrating: Take a good hard look at what you are migrating. Is there stale data that you can send to the great bitbucket in the sky? Is there regulated data that requires security controls and specific access management? What data should go where? Who should be able to access what?
- Extract, transform and deduplicate data before moving: It’s a good idea to do a full data clean-up before migration. Once the data is migrated, it’s probably going to be in that state until the next migration. Make sure you’re migrating the right data while preserving data integrity.
- Implement data migration policies: Establish policies to make sure data is going to the right place and ensure that data is protected once migrated. You can automate these policies to make the destination data even more secure than the source – and even set up rules to re-permission the data during the migration.
- Test and validate migrated data: Make sure everything’s where it should be, create automatic retention policy, clean up stale data, and double check permissions. Back up your old system, so that you’ll be able to find any missing files offline, if necessary.
- Audit and document the entire process: Your compliance team will appreciate it.
How to Avoid Data Migration Errors
- Don’t migrate bad habits: Clean up broken inheritance before you migrate the data. Migrating files and folders with broken inheritance will create a worse mess than you already have.
- Automate and simplify: So many bad things can happen when you try to use a workstation session to move large amounts of data. The session could disconnect, the computer could go offline, or the computer could blue screen. With the Data Transport Engine, you can automatically move data from one storage server to another, saving money and reducing overhead, all while maintaining or updating file access permissions. .
- Cover all your bases: Do you have a backup plan to the contingency plan? You probably should. If you were moving $1 million in cash how careful would you be? Now think of your data in dollars. How much is that data worth? It could be much more than $1 million.
Data Migration vs. Data Integration
Data migrations and data integrations are completely different tasks. Make sure you’re using the correct tool for the correct jobs.
Data integrations are taking two data repositories and merging them to make one big repository. You see this task in Big Data projects where you want to have large data stores available for many types of analytics tasks. The data might not be all the same kind of data, but it all lives in the same repository.
Data migrations are simply moving from one data store to the other.
How to Automate and Simplify Data Migrations
Data migrations can be huge (and difficult) projects – and when migrating sensitive and critical data, it’s more important than ever to make sure you’ve got a plan in place to migrate your data as securely as possible. You can take the guesswork out of it, while lowering risk, by leveraging the Data Transport Engine to move large amounts of data from one storage system to another, cross-platform, or to SharePoint. You can even map permissions from one system to the other, even if you are moving from NTFS to NFS.
Want to see how it works? Get in touch to see how Varonis helps automate and simplify data migrations.