Creation of simple pipeline using Microsoft Azure data factory
First create a azure free account and try this simple pipeline.
To create pipeline which moves file from one blob container to other blob container, we need 2 resources in Microsoft Azure
- Azure storage account creation
- Azure data factory
Azure storage account creation step by step:
Provide necessary details as per your requirement and continue to clicking next button keeping all default settings.
After deployment done then go to the resources
Now create container and upload the source file
Now using same way create target container named demodeb2 and keep it blank where we need to export our source file present in demodeb1 container.
Azure Data factory resource creation step by step:
Provide necessary details as per your requirement and continue to clicking next button .For learning purpose I selected the check box Git configuration later.
After deployment done , then go to resources tab
Creation of Pipeline step by step:
To create pipeline first we need to create linked service for connection to azure blob and then create source dataset and target dataset pointing to the respective container.
Now go through the steps how I create a simple pipeline for moving file from one blob container to other blob container.
creation of linked service:
For storage account name and key we have to go to storage resource and take the following information.
after filling the required details , do test connection .Once done then create the linked service.
creation of pipeline:
Drag and drop the copy data transform in editor and create source and sink dataset.
Now lets see how I created source and sink dataset pointing to different blob container.
Using the same way, create dataset for sink
Now validate the pipeline and publish
Now run the pipeline by using manual trigger.
We can monitor the pipeline using monitor tab
Now after successful run of pipeline , we can see the file already moved to target container.
Wow! we got the file
Keep learning !!
