Data migration from Excel, Tally or an old desktop program

Data migration is the job of moving your records from whatever you use today into the new system, so that on Monday morning every balance, every patient and every pending order is where it should be. It has four parts: mapping (which old field goes to which new field), cleaning (fixing what is wrong or missing), reconciliation (proving the totals match) and the go-live weekend (the cutover itself). Done well it is boring. Done badly it is the reason staff go back to the old system.

Where does the data usually live?

Every Healthixio and Pathixio customer came from somewhere, and it is usually one of three places. Excel or Google Sheets, with one file per year and a different column order in each. Tally, which holds ledgers, parties and stock but not the operational records around them. Or a desktop program from a vendor who has since disappeared, with data in an Access, FoxPro or SQL Server file that nobody has opened directly. Sometimes it is paper, in which case migration means deciding what gets typed in and what gets scanned.

How does mapping work?

Mapping is a spreadsheet with three columns: old field, new field, rule. Most rows are simple. 'Pt_Name' becomes 'patient full name'. The interesting rows are the ones where the old system held two things in one field, or one thing in two fields, or a code that meant something to the person who set it up in 2011. The mapping is written before any code, reviewed with the person who knows the old system best, and signed off. It is the same discipline we use for the data model on custom projects: agree the shape first.

• One old field to one new field: copy, with a type conversion if needed.
• One old field to many: split, such as 'Name (Age/Sex)' into three fields.
• Many old fields to one: join, such as three address lines into one structured address.
• Codes to lookups: 'DEPT=3' becomes 'Orthopaedics', with a table you approve.
• Fields with no home: listed, and either dropped with your agreement or kept in a notes field.

What does cleaning involve?

Old data is dirty in predictable ways. Phone numbers with spaces, plus signs and a missing digit. Dates written as 03/04/2019 with no way to know which part is the month. Duplicate patients registered on different visits. GSTINs that fail the check digit. The same doctor spelt four ways. Cleaning is the process of running rules over the export, producing an exceptions list, and having someone from your side decide each exception. The rules are cheap to write. The decisions are yours, and they take longer than anyone expects, so start them in the first week, not the last.

Two rules we apply on every migration. Nothing is deleted from the source; exceptions are flagged, not dropped. And personal data is handled under the Digital Personal Data Protection Act from the moment we receive the export, which means a signed agreement about who holds copies and when they are destroyed.

How is reconciliation done?

Reconciliation is the proof. After a trial migration, the new system and the old one are compared on numbers a business owner cares about, not on row counts alone. The list is agreed in advance and signed off after each trial run.

1. Counts: patients, parties, products, open orders, pending samples, active members. Old versus new, with every difference explained.
2. Balances: receivables and payables per party, cash and bank, stock value. These must match Tally to the rupee or the difference must be named.
3. Spot checks: twenty records chosen by you, opened side by side.
4. Reports: the daily collection report and the outstanding report for the last month, run in both systems.
5. A sign-off sheet with the numbers, the differences, and a signature.

We run at least two trial migrations. The first finds the mapping errors. The second proves the fixes. The go-live run is then a repeat of something that has already worked twice.

What happens on the go-live weekend?

The cutover is planned around your quietest time, usually a weekend or a holiday. The sequence is fixed and rehearsed.

1. Friday evening: the old system is closed for new entries. A final export is taken and its counts are recorded.
2. Friday night: the migration script runs against the final export. Automated checks compare counts and balances to the recorded numbers.
3. Saturday: the reconciliation list is run and signed. Exceptions found now are fixed in the new system by hand, with a log.
4. Saturday afternoon: staff log in and do a real day's transactions as a drill. Printouts are checked.
5. Sunday: the old system is set to read-only, not switched off. It stays available for lookups for months.
6. Monday: the new system is live, and someone from our side is on the floor or on the phone all day.

Staff training happens before this weekend, not during it. The drill on Saturday is a rehearsal of Monday, not the first time anyone has seen the screens.

What to ask a vendor about migration

• Is migration in the price, and does it include Tally, Excel and the old program together?
• Will I get the mapping sheet and the exceptions list to review?
• How many trial runs, and what is on the reconciliation sheet?
• What happens to the old system after go-live, and for how long?
• Who holds copies of my export, and when are they destroyed?

Frequently asked questions

How long does a migration take?

For a clinic or lab on Excel and a desktop program, the technical work is often a couple of weeks, but the calendar time depends on how fast your side clears the exceptions list. Budget for two trial runs and a weekend cutover. Multi-branch businesses with years of Tally history take longer because balances must match per branch.

Should we migrate everything or start fresh?

Migrate what the business needs to operate and report on: master data, open balances, active records and at least a year of history. Older history can stay in the read-only old system or be imported later. Starting completely fresh sounds clean but means retyping outstanding balances by hand, which is where errors come from.

Can we keep using Tally after migration?

Yes, and many businesses do. Tally stays the accounting ledger and the new system handles operations, with vouchers exported to Tally on a schedule or through a connector. The mapping then has to be kept in both directions. Decide this before migration, because it changes which fields are the source of truth.