Migrate a huge volume of data to Magento 2

Going Omnichannel is an undeniable trend in recent years as all retail companies wish to provide their customers with the seamless shopping experience across different touch points.

Early 2020, Boyd – CEO of an outdoor sporting brand in the USA came to us and shared that he has a plan to scale the business which required a more complete and centralized management system. However, the current software that he was using from Winward was not connected to the Magento e-commerce platform yet, this issue caused their company at least thousands of dollars per month to cover the human cost of doing manual tasks to update back and forth between systems. Furthermore, sometimes the manual tasks made some mistakes, the staff had to re-done the job, which frustrated both the staff and the job. 

Thus he was urged to find a more suitable solution. After going through different analyses and demonstrations, Boyd decided to move to the Magestore Omnichannel solution. Once the decision had been made, the very first task that we needed to do was how to migrate all data from Winward to Magento.

The simplified business structure

1. The migration process from Winward to Magento

1.1. The first step is to define which data needs to be migrated:

  • Products
  • Orders
  • Customers
  • Stock in physical locations
  • Suppliers
  • Purchase orders
  • Loyalty points
  • Customer level
  • Membership Program

Below is the data volume:

1.2. Migrate data:

  • Learn the data structure in Winward and map them to Magento data 
  • Export the CSV file
  • Build the tool to convert data
  • Import the CSV file to Magento
  • Correct data on Magento

2. Challenges and Solution

In fact, the process did not go well as we expected and we encountered some difficulties, especially the one with the data size. It’s much bigger than we thought. By that time, the development has gathered to figure out the best solution for it.

2.1. Challenges: 

  • Using the csv import function in Magento for the product data takes much time.
  • The customer is unable to identify which field is unique in product data, which leads to duplicated entities. 
  • The size of the csv export file is too huge, which can’t even be open in the online tools (like Google sheet) or available desktop apps (like MS Excel 2003).

2.2. Solutions:

  • Build the tool to convert the data (js is recommended as it’s easy to use and develop further.
  • Find a tool that can handle the large volume of the data. In case of the small volume of data, it can be directly imported via CSDL.

3. Using the new system on Magento

It took us 6 weeks to complete the migration process and set up the Magestore solution to Magento for Boyd’s company.. 

With the new system, Boyd started to implement the Omnichannel strategies as the company’s direction in the next few years.

Riley Dang
Data and process-driven business consultant with a demonstrated history of working with retailers. My aspiration is to help businesses to scale-up and thrive by implementing unified systems.

Leave a Reply

Your email address will not be published. Required fields are marked *

Let's talk

If you want to get a free consultation without any obligations, fill in the form below and we'll get in touch with you.