Automate historic data migration between two dates Skip navigation

Automate historic data migration between two dates

score 25
You have not voted. Active

1. Introduction

1.1. Limitation of most of the job schedulers. Most of the ETL job schedulers have no option to execute/schedule ETL jobs in a controlled fashion to load historic data between two dates. This is a useful feature for data migration projects and requires a lot of manual intervention at present.

 

2. Solution : A simple solution to run a informatica workflow for number of times based on a condition such as till data for all the dates is processed or till start date in a file is equal to end date in that file, is to create two workflows and make them call each other through pmcmd where first workflow will modify start date present in a target file holding two dates i.e. start date and end date, where as other workflow will use this target file as a parameter file to process data for a particular date present as start date in the parameter file.

 

Both the workflows will kick off each other to make the process run in loop. A logic should be placed in workflow modifying the file holding dates to ensure the workflow fails when start date is equal to end date in that file.

 

?idea.JPG

Comments

Vote history