dimanche 6 février 2022

ADF Pipeline beginner series : Add list of items to an array variable

 This is a series of tiny articles about some features that are lacking in Azure data factory and how to overcome them :

This follow this post 

As today ( 28/01/2022) Azure Data Factory doesn't allow the Append variable activity to add a list of item to a array variable. The append to array activity state the possibility to add an item to a type array,however, there the two way to deal with it are dead en 

First  if you try to pass a list of item only the first one will be happend to the list.

Second if you try to append an array directly you will reach an error


or



Therefore to achieve the desire outome we can leverage two variable. The first one can be variable1 and the second one _Variable1 and we will use the underscore one like a register.

In this example we looking to add the files name to an existing list of file.

So the first step is we store the actual value of the ListFiles into _ListFiles.


We use a Set variable activity to achieve this :

Second step is to join both item into the array,since we don't have any function to achieve this we need to use text edition command and re-cast as an Object.

One way to achieve this is to use the @union command like

 @union(variables('_ListFiles'),activity('2_6 - Filter only files using -tmpQUeue as source').output.Value)

 this way we add the existing data of ListFiles (which was stored temporarily into
_ListFiles and the output of one activity ( 2_6 on this case )


Aucun commentaire:

Enregistrer un commentaire