When implementing or upgrading to a new LIMS, a common question is “What should we do with our existing data?” There are many possible solutions, ranging from not moving any data and choosing to archive everything to doing a full-scale data migration to the new LIMS. Regardless of the selected path, careful consideration should be given to the data requirements of the business and the complexity of the solution. Further, it’s important to recognize there are many risks and difficulties when considering a data migration in a LIMS. 

Learn more about our LIMS consulting practice

Below, we outline four considerations for LIMS data migrations that businesses should keep in mind:

LIMS Data Migrations Considerations

1. Static data version history: LIMS static data (master data) has a version history. If you migrate the current version of a static data object to a new database, that migrated version number may not reflect the reality of the version (e.g., an object with Version 4 is migrated to a new database where it should be Version 1, but the version number was migrated as 4). Static data version number is also tied to dynamic data to ensure relational integrity; if you’re moving both dynamic data and static data, how will you ensure the versions stay aligned? 
2. Static data design strategy: If there’s a strategic change in how your static data is designed during the migration, how will you ensure that the migrated data matches the new strategy?  For example, perhaps in the previous environment, the stability specifications were stored separately from release specifications, but in the new environment, they will be stored together.  How will you transform the data to match while ensuring relational integrity with the dynamic data?  
3. Application version and data model: Are you changing applications? How should fields in the new application align with fields in the old application? Database fields may be added or changed with an upgrade or a new LIMS application. As such, you need to consider how the mapping between systems will be done. 
4. Dynamic data volume: Depending on the age of the system, there may be a very large number of records to migrate. As a thought experiment, imagine you have a laboratory that processes 10,000 samples a year. Each sample has four tests, and each test has four results. If this system was in place for 10 years, just the testing dynamic data alone would encompass 1.6 million records that must be extracted, transformed, loaded, and verified. Can you make an argument for the effort that is needed to perform this work? Consider that system performance may be affected due to the amount and types of records in the database.  

These considerations can lead to several different outcomes depending on schedule, resource availability, budget, and business need. With that in mind, there are a few options for LIMS data migrations that can help businesses realize benefits without performing a full migration: 

1. Archive data in a data warehouse: If a data warehouse is available and data retention is required by regulatory authorities or for other business needs, it can be relatively straightforward to identify the necessary data and store it in a warehouse where it can be easily reported on. This avoids difficulties with transformation and keeps the data in its original structure. 
2. Create custom database structures: A custom database table can be created to store specific data elements instead of moving large quantities of data. For example, extracting dynamic data and a subset of data identifiers and loading them into a table specifically created to store that data can give the end users access to the data without the need to transform data or identify destination fields. 
3. Recreate data in the target environment: For ongoing studies, such as stability, it would take far less effort, less risk, and less time to recreate the studies in the target environment and re-enter data. This approach typically works well with stability studies, but depending on the volume of ongoing studies, this may rely on a large amount of effort. If completed studies need to be retained for regulatory or other business purposes, a solution to store that data will also be required. 

Moving Forward with LIMS Data Migrations 

LIMS data migrations are complex and intensive efforts with many variables to consider. It’s possible, however, that the effort can be mitigated by gaining a full understanding of the business needs for the data and considering uncomplicated alternatives. If your organization is looking for guidance when it comes to LIMS data migrations, our team of LIMS consultants can help.