AzureFunctions Work Fan-out with Azure Queue in PowerShell
/So I really like using PnP-PowerShell to chain up and perform complex operations in Office 365, and linking them up with AzureFunctions and Flow.
Scenario - scan all tenant's site collections
I bump into another problem today - I needed to scan all my site collections within the tenancy and start a Flow that will notify and apply site closure policy and lock the site.
We can scan all the site collections in a tenant in one request Get-PnPTenantSites - but we want to make sure the job doesn't time out.
So we need to fan-out the workload to a queue, and trigger multiple AzureFunctions to scan each site collection in parallel.
Problem - PoSH Queue binding only 1 output
As soon as I started writing the PoSH - I remembered, with the default PoSH Queue binding - you can only write 1 message to the Queue.
Unlike C# where you could do multiple:
foreach(var message in messages) {
await outQueue.AddAsync(message);
}In PoSH - if you are using the default integration tab to set up an Output Binding to AzureQueue. Then you can only write one message to the Queue.
How to write multiple messages to Queue in PoSH?
It turns out I've already solved this once before in April, but I had completely forgotten this, because I DIDN'T BLOG IT.
Let that be a lesson to all developers - Always blog something cool that you did. Because you will need it in two months when your memory failed you.
# if you have been using the storage in other functions
# you will already have the connection string in your
# function's app settings - reuse it
$storeAuthContext = New-AzureStorageContext -ConnectionString $env:azurefunctions3a585851_STORAGE
$outQueue = Get-AzureStorageQueue –Name 'my-queue-name' -Context $storeAuthContext
if ($outQueue -eq $null) {
$outQueue = New-AzureStorageQueue –Name 'my-queue-name' -Context $storeAuthContext
}
# this example isn't scanning sites - just going through files in a library
$items | % {
$item = @{
source = $_.FieldValues.FileRef;
target = ($destination + "/" + $_.FieldValues.FileLeafRef)
}
# Create a new message using a constructor of the CloudQueueMessage class.
$queueMessage = New-Object `
-TypeName Microsoft.WindowsAzure.Storage.Queue.CloudQueueMessage `
-ArgumentList (ConvertTo-Json $item)
# Add a new message to the queue.
$outQueue.CloudQueue.AddMessage($queueMessage)
}
