Skip to main content
Version: 2023

Data migration

Introduction to Data migration

A very important step in preparing a database upgrade for your end-application is setting up the data migration between two model versions. This adds information to the upgrade script about renamed and changed tables and columns.

danger

If data migration is not applied correctly, it can ruin the database. In a best-case scenario, the upgrade script will fail, and a backup has to be restored. Yet, in the worst case, the upgrade is carried out without an error message but with data lost.

Source model version

menu Data > Data migration

Always check the selected source model version. This ensures that the new version is created from the correct branch and model version.

You can compare two branches or two points in time within one branch based on the temporal history. If you obtain the latest model version of your end-product, the branch's source branch and model version will be set to that version. This allows easy generation of upgrade scripts for different end-products.

You can set the source set source for data migration and upgrade by obtaining it from the database or manual selection. To prevent problems, this is not possible when jobs for definition generation or validation, or for source code generation are queued or active.

update source Update the previous version

Rebuilding a table

Even if nothing has changed, it can still be desirable to rebuild a complete table. For example, when the structure of a table can no longer be guaranteed.

menu Data > Data migration > tab Tables

  1. Execute the task Rebuild table rebuild.
  2. Answer Yes to the question "Are you sure you want to mark this table to rebuild?".

Table mapping

Update a table mapping

Data migration automatically attempts to map the old tables and new tables based on naming. If the match is successful, From table and To table are filled, and the Status is set to 'Changed'. It is also possible to create your own table mapping.

menu Data > Data migration > tab Tables

  1. Execute the task Update table mapping update.
  2. Where necessary, specify the From table or To table.

update a table mapping Update a table mapping

Disconnect a table mapping

Sometimes a table has the same name as before but a different function. Therefore, the data is no longer correct, and the table needs to be emptied after the upgrade. In this case, it is possible to disconnect a table mapping. The target table will be considered 'New' and not receive data during data migration.

menu Data > Data migration > tab Tables

  1. Execute the task Disconnect table mapping disconnect.
  2. Answer Yes to the question "Are you sure you want to disconnect the mapping of [table]".

Now, the record with a from and to table is split into two records. The to table gets the Status 'New' and the from table gets the Status 'Removed'.

Reset a table mapping

It is possible to reset a specific table for the configured source back to the default mapping. This can only be done when the table mapping has been updated manually (the generated checkbox is cleared).

menu Data > Data migration > tab Tables

  1. Execute the task Reset table mapping disconnect.
  2. Answer Yes to the question "Are you sure you want to reset the mapping of [table]?".

To reset the entire data migration for the configured source back to the default mapping, execute the task Fully reset data migration full reset.

Column mapping

Update a column mapping

To map two columns within a table:

menu Data > Data migration > tab Columns or tab All columns

  1. Execute the task Update column mapping update.
  2. Where necessary, specify the From column and To column.
  3. Optional: set a Default value type. This can be a constant value or an expression. These fields are only mandatory if the column in To column is mandatory. The difference between the default values in data migration and the data model is that the default value in data migration is used only during the upgrade.

map columns Update a column mapping

Disconnect a column mapping

Sometimes a column has the same name as before but a different function. Therefore, the data is no longer correct, and the column needs to be emptied after the upgrade. In this case, it is possible to disconnect a column mapping. The target column will be considered 'New' and not receive data during data migration, unless a default value is configured.

menu Data > Data migration > tab Columns or tab All columns

  1. Execute the task Disconnect column mapping reset.
  2. Answer Yes to the question "Are you sure you want to disconnect the mapping of [column]?".

Now, the record with a from and to column is split into two records. The to column gets the Status 'New' and the from column gets the Status 'Removed'.

Reset a column mapping

It is possible to reset a specific column for the configured source back to the default mapping. This can only be done when the column mapping has been updated manually (the generated checkbox is cleared).

menu Data > Data migration > tab Columns or tab All columns

  1. Execute the task Reset column mapping disconnect.
  2. Answer Yes to the question "Are you sure you want to reset the mapping of [column]?".

Re-apply intermediate mappings

menu Data > Data migration > tab Tables

When you set the source to an earlier version (for example, when releasing a major version), all mapping for the development versions in between will not be included by default. In that case, you can re-apply intermediate mappings reapply that have an earlier configured source or a different branch further down the origin chain. The task shows which intermediate versions contain mapping information that could be applied.

When you execute the task, these mappings will be applied recursively. Mappings configured directly to the target branch will be favored over mappings between a source branch and other intermediate versions in this source branch.

caution

After having re-applied intermediate mappings, always verify the results.

reapply intermediate mappings Re-apply the intermediate mappings

Data migration for complex situations

The data migration settings are intended for simple scenarios, where records are moved from one table to another or columns are filled using default values.

For more complex scenarios, i.e., merging tables, changing primary keys, normalizing or denormalizing, you can add scripts into the upgrade process. See Model version specific update scripts in the Functionality manual.