How Data Migration Works in Your Project
If your project involves moving existing data into a new system — from a legacy platform, spreadsheets, a previous supplier, or another database — data migration is a critical and often complex piece of work. This article explains how we approach it.
Phases of Data Migration
- Data Discovery: We work with you to understand what data exists, where it lives, its format, and its quality
- Data Mapping: We document how data from the old system maps to fields in the new system
- Data Cleansing: We identify and handle data quality issues — duplicates, missing values, invalid formats
- Migration Script Development: We write automated scripts to transform and load the data
- Test Migration: We run the migration against a copy of your data in the staging environment and verify results
- Client Validation: You review a sample of migrated data and confirm it is correct
- Production Migration: The final migration runs against the live dataset
Your Responsibilities
- Providing access to the source data (database, CSV exports, API access)
- Identifying a subject matter expert who knows the existing data structure
- Validating migrated data samples during test migration
- Confirming final sign-off on migrated data quality before go-live
Data Quality Risks
We can only migrate what exists. If your existing data has quality issues (duplicates, missing fields, inconsistent formats), these must either be cleansed before migration or accepted as-is in the new system. Data cleansing is often scoped as a separate item and can be time-consuming — flag potential quality issues early.
Cutover Planning
For large migrations, we plan a "cutover" — the point at which the old system is decommissioned and the new one goes live. We will plan this carefully with you to minimise disruption, often migrating during off-peak hours or a weekend.