ADF - Create DataSet with PowerShell - Azure Data Factory

Create a dataset
In this step, you define a dataset that represents the data to copy from a source to a sink. The dataset is of type AzureBlob. It refers to the Azure Storage linked service you created in the previous step. It takes a parameter to construct the folderPath property. For an input dataset, the copy activity in the pipeline passes the input path as a value for this parameter. Similarly, for an output dataset, the copy activity passes the output path as a value for this parameter.

Create a JSON file named BlobDataset.json in the C:\ADFv2QuickStartPSH folder, with the following content:

JSON

Copy
{
"name": "BlobDataset",
"properties": {
"type": "AzureBlob",
"typeProperties": {
"folderPath": "@{dataset().path}"
},
"linkedServiceName": {
"referenceName": "AzureStorageLinkedService",
"type": "LinkedServiceReference"
},
"parameters": {
"path": {
"type": "String"
}
}
}
}
To create the dataset: BlobDataset, run the Set-AzDataFactoryV2Dataset cmdlet.

PowerShell

Copy
Set-AzDataFactoryV2Dataset -DataFactoryName $DataFactory.DataFactoryName `
-ResourceGroupName $ResGrp.ResourceGroupName -Name "BlobDataset" `
-DefinitionFile ".\BlobDataset.json"
Here is the sample output:

console

Copy
DatasetName : BlobDataset

Structure :
Properties : Microsoft.Azure.Management.DataFactory.Models.AzureBlobDataset