For the complete documentation index, see llms.txt. This page is also available as Markdown.

Pipeline Operation: Migrate Data

Written for the Old UI

Overview

The Migrate Data operation runs a Data Migrator migration job as a step in a DevOps pipeline. It transfers records between a source and target organization using a Data Migrator template, so you can move data automatically during a pipeline rather than running a migration manually afterward.

Because the migration runs in the Data Migrator app, the operation uses your existing templates. These templates specify which objects and records to transfer, along with any filters and record mappings.

Best Practices

  • Test your Data Migrator template in the Data Migrator app before adding it to a DevOps pipeline.

  • Use this operation to seed a target organization with configuration or reference data after a deployment.

  • Keep records consistent across environments by including a Migrate Data operation.

  • Use Record Mapping in your template for repeated migrations to avoid duplicating data.

  • Apply data masking in your Data Migrator template to replace sensitive data with random seed values, ensuring no real data appears in the target organization.

Data Migrator Prerequisites

Before adding a Migrate Data operation, you must configure several items in the Data Migrator app. The following sections describe these prerequisites and link to related articles.

Template

Every data migration runs from a template that defines which objects and records are migrated, along with any filters and record mappings. Verify that a template copies the correct records before using it in an automated process such as a pipeline. To learn more about Data Migrator templates, review the following articles.

Organizations

A data migration can run only between organizations that have the Data Migrator app assigned. If an organization you want to use is not listed, assign Data Migrator to it in the Global Settings app. For details, see the following article about assigning apps to organizations.

Migrate Data Operation vs Data Migrator Migration

Because the Migrate Data operation uses Data Migrator to create and run the migration, the end results are the same. However, there is an important distinction between the two.

In Data Migrator, a migration is a multi-step process that involves creating, reviewing, and running the migration.

In a DevOps pipeline, the Migrate Data operation is a single process that automates both the creation and the execution of the migration. Because a pipeline is automated, it skips the review step. For this reason, confirm that the migration works correctly before including it in a critical pipeline.

Migrate Data Operation Configuration

The instructions below apply to configuring this pipeline operation. For general instructions on adding a pipeline step and setting the next step, see the following article.

Adding a Pipeline Step

1

Select Operation and Job Name

  1. Click the Operation dropdown menu.

  2. Select Migrate Data.

  3. In the Job name text field, enter a name for the migration job to be run in Data Migrator.

2

Select Source and Target Organizations

Only organizations with the Data Migrator app assigned to them are available as sources and targets for data migration. To migrate data to or from an organization that has only DevOps assigned, first assign Data Migrator to that organization in the Global Settings app. For details, see the following article on assigning apps to organizations.

  1. Click the Source Organization dropdown menu.

  2. Select the organization to use as the source of the migration.

  3. Click the Target Organization dropdown menu.

  4. Select the organization to use as the target of the migration.

3

Select Data Migrator Template

Because you cannot view Data Migrator templates in DevOps, you must know the name of the Data Migrator template you want to use for this migration. If you are unsure, open Data Migrator to find the correct template name. See the following article to learn more about Data Migrator templates.

  1. Click the Template dropdown menu.

  2. Select the template to use for the data migration.

Migrate Data Results

Once a pipeline is started, its progress and results can be viewed from the Activities tab from the branch that ran the pipeline. The instructions below show you how to review the results of this operation. For general instructions on accessing pipeline results from the branch, see the following article.

1

Summary and Log

Once the Migrate Data job is complete, you can view the results summary by selecting the Migrate Data step from the pipeline results.

The Summary of the Migrate Data results displays the following information:

Fields
Description

Status

Completed if the migration job has finished running. A migration is still considered completed even if it has Failed Records.

Exception if an issue prevented the migration job from running. Check the logs to troubleshoot.

Duration

Time the Migrate Data job took to complete, shown in hours, minutes, and seconds.

Date started/finished

The date and time the Migrate Data job started and finished.

Total objects

Total number of objects included in the migration.

Total records

Total number of records in the migration, including successful and failed records.

Failed records

Number of records that failed to migrate.

Log

Log entries for the Migrate Data job. Use the log to review the job's status and any error messages.

Example of a Migrate Data log:

2

See Results

To view more details about the migration, open the results.

  1. Click See result.

  2. A new Data Migrator tab opens and displays the migration job results.

  3. For detailed instructions on reviewing the migration results, see the article below.

You can roll back a data migration from the Data Migrator app. For instructions, see the article below.

Last updated

Was this helpful?