Nested Flow (Reusable-Function) cheatsheet for Microsof Flow

To build complex, but maintainable Flows, it is important to be familiar with this next step.  The Flow blog calls this Nested Flows, so I will stick with this term.  What this really is in essence, is how to build Flows as a reusable function or service.  I want to then expand on the official blog post and then go far beyond.

This is the second of the cheatsheet series on Microsoft Flow.

1. JSON cheatsheet for Microsoft Flow
2. Nested-Flow / Reusable-Function cheatsheet for Microsoft Flow (this article)
3. Building non-JSON webservices with Flow
4. One Connection to Proxy Them All - Microsoft Flow with Azure Functions Proxies
5. Building Binary output service with Microsoft Flow

In this post, we will cover:

1. Define a function request
2. Define a function return
3. Call a function with parameter
4. Parse a function return 

Part 1. Define a Nested-Flow like a function

function(word: string, number: number, complete: Boolean)
Let's say we want to define a function with a few arguments.  Start with the Request trigger.  This Flow will run when a request is sent to the Request URL.

We will need to provide the JSON schema that defines the arguments we want.  Anytime we see "Use sample payload to generate schema" immediately click it.  Life is way too short to learn JSON schema.

Type in the JSON we want to send to this end point.

{
    "word": "a word",
    "number": 5,
    "complete": false
}

The JSON schema will be generated for us.  Easy.

Define the return value from Nested-Flow

function() -> { word: string, number: number, complete: Boolean }
Our Flow doesn't need to return a response.  But if we want it to, we can use the Response Action to specify what the response is.
Response Action is a terminating action.  We can't add anymore actions afterwards.

There are many ways to set up the response JSON.  Here we use a variable and initialize it to the values from the Request trigger.

Test Nested-Flow

First, save the Flow - the HTTP Request URL will be generated after save.

Poke the URL with Postman

View fullsize

Note: We have to set content-type to JSON (application/json), otherwise Flow will treat the entire body as plain text.

See how the request is sent to Flow, then returned.  The return type is JSON

View fullsize

So far so good.  Our Flow responds to a HTTP call and returns values.

Technically, there's nothing nested about this Flow, not until we call it from another Flow.  Let's do that next.

Part 2. Call a Nested-Flow

The official blog post covered most of this.  So I'm including some pictures with light commentary.  These steps are pretty easy.

We can call HTTP with a literal JSON in the body.  But I find in reality, we will almost never do this.  We are always constructing a JSON to be passed in.  So I would start with an extra variable.

Parsing the return value from the Nested-Flow

When the HTTP action returns, we will have a body text.  This is not strongly typed and we can't work with this easily.  The magic is to use the action "Data Operations - Parse JSON" to force the body json into a JSON schema and extract strongly typed variables we can then rely on.

We will get used to "use sample payload to generate schema" pretty quick.  Every time we see it we should immediately click it.
Since the Nested-Flow wants the same object that we defined in Part 1 above, we will use the same JSON sample.

Flow then understands we have strongly typed properties to work with.

Test the Run Nested-Flow, Flow

The parent Flow calls the Nested-Flow, and gets the return values.

View fullsize

So now we have four patterns:

1. Define a function request
2. Define a function return
3. Call a function with parameter
4. Parse a function return 

These are the four basic steps in this cheat sheet.  What they unlock.  Is the most powerful workflow product ever.

We need to wrap this up with an ultimate example.

Crazy Example: Fibonnaci Flow

Wait, John, did you say fibonnaci, you mean that recursive function...  

fibonnaci (n)
f(0) => 0
f(1) => 1
f(N) => f(N-2) + f(N-1)

So this is a recursive Flow 

View fullsize

We switch on the 'n' parameter.  If 0,1 we return.  If n, we do n-2/n-1.  
If n is negative, we would be in big trouble ;-)

View fullsize

simple recursion.  fib(2) = 1

View fullsize

deeper recursion.  fib(4) = 3

View fullsize

Always a good reminder that the naive implementation of recursive fibonnaci is very costly in terms of repeated recursion.  fib(4) is 10 runs.

View fullsize

Examining one of the runs - results of fib(3) + fib(2) = 3

Why this is a Game Changer

I hear about how Microsoft Flow is not ready for consumers.  Well, I think they are all wrong really ;-)  Microsoft Flow is amazing.  Yea the UX has bugs.  Sometimes you just need to stand on sharp broken edges to see greatness.  In Australia we say: suck it up princess.

Here's twelve reasons why you need this.  You needed this yesterday, no, I asked for this 5 years ago.

1. You should put repeated steps into a separate function you can call.  This is like basic functionality. 
2. e.g. Email someone
3. e.g. Retrieve an email template from SharePoint then email someone
4. e.g. Retrieve an email template and recipients list from SharePoint/Azure AD then email someone. 
5. e.g. Spam the Team's conversation thread with an urgent message
6. There are concepts very similar to this in various workflow systems I've used - User Defined Actions, WF Custom Workflow Activities, except Nested-Flow just works.
7. If you are chaining Flow Trigger -> Azure Functions -> Flow Delay -> Azure Function.  This gives you ultimate flexibility.  There is no built in "delay" in Azure Function.
8. Have you EVER wish you had SharePoint Workflow that could call itself?  Revolutionary.
9. Your JavaScript can call the Flow - Julie Turner has a great post on this
10. Because Flow supports unwinding a failed action set - you can put fail handler in a … Flow.  It's Exactly like a Flow - try/catch block.
11. Because in one Flow you have a top secret connection that only you can use.  But you want other people to be able to reuse the action, which is configured with your top secret connection.  But you don't want to tell other people what your top secret connection is.
12. Because Flow, on top of LogicApps, belongs to that wonderful category of Azure solutions under the Serverless banner.  Yes no servers.  Few people will care about servers in the future.
13. Because Flow don't let you copy and paste actions.  Are you really going to manually re-create all those steps?

You tell me!  There's got to be a hundred reasons right here.  I'm literally adding new points as I type because there is just so many possibilities.

If you are doing Flow, you need to master this technique.  No exceptions.
 

References

• https://flow.microsoft.com/en-us/blog/build-nested-flows/
• http://julieturner.net/2017/07/sharepoint-flow-azure-functions-launching-a-microsoft-flow-from-client-side-code/

JSON cheatsheet for Microsoft Flow

I have a big blog post in the works for a significant dive in Microsoft Flow functionality.  But I realized that before we get there, we need to GET GOOD at doing JSON in Microsoft Flow.

So this is the cheat sheet.  With all the caveats.  Enjoy!

Update: added a compound JSON object construction.

This is the part of the cheatsheet series on Microsoft Flow.

1. JSON cheatsheet for Microsoft Flow (this article)
2. Nested-Flow / Reusable-Function cheatsheet for Microsoft Flow
3. Building non-JSON webservices with Flow
4. One Connection to Proxy Them All - Microsoft Flow with Azure Functions Proxies
5. Building Binary output service with Microsoft Flow

var X = { "x": 1 }

{ "x": 1 }

And this is the running results

X = JSON.parse("{ 'x': 1 }")

"@json('{\"x\": 2}')"

X is updated.  But the run result panel for Set variable shows incorrect INPUTS value of X

{ ...X, { 'y': 2 } }
jQuery.extend( X, { 'y': 2 } )

"@union(variables('X'), json('{\"y\":2}'))"

Union is great - it'll let you add properties to existing objects easily.

var Y = { ...X, { 'y': 2 } }

Use union to set variable

if ( Y.x != null )

sometimes, you need to check if an object contains a property

sometimes, you suddenly don't need the " double quotes for conditional statements. 
sometimes I feel this is a bug.

The UX can get quite silly.  It forgets how to render the basic view after you save.

(Y.x || Y.y)

"@coalesce(variables('Y').x, variables('Y').y )"

Use Coalesce to pick the first non-null value.  If the property doesn't exist this will error.  Protect the check with contains

X = Y

And we end this cheat sheet on a simple one.  Assigning variables to another.

Note you can't currently use the variable in setting the variable itself.

This would have been REALLY useful...  Because this would have let us keep appending to existing variable.

Finally, object composition
var Z = { z: Y }

"@json( concat( '{ \"z\": ', variables('Y'), ' }'  ))"

The result, see how a compound JSON object has been constructed.

Speaking at Digital Workplace Conference Australia 2017

I'll be speaking at the Digital Workplace Conference Australia!  23-24 August in Sydney.

This is a conference that's near and dear to me - and I've had several opportunities in the past to present at this conference, where I covered Silverlight, JavaScript, TypeScript, Modern Office App-ins and now this year - I plan to present a supercharged talk on running Serverless with Office 365.

Parts of the talk - especially how to get started - may seem familiar to many of you that has started down this journey. 

I wanted to focus a bit less on the technical, and more about how this has changed people. 

Azure Functions democratized 'I need to run a bit of code' to everyone.  Suddenly, the cloud is not this scary place where there are a hundred things we don't know, and don't know where to start.  Suddenly, the toys that seems far out of reach are ours.  Suddenly, a cloud subscription that costs less than a coffee per month is something I don't even think about.

To me, that is the power of AzureFunctions and why Serverless is a game changer. 

Do you know there are now brand new categories of design patterns specifically rewritten for the Serverless world.

I will of course still cover the technical bits - but to see all 20+ demos I have with me, you'll have to come find me in the speaker area for a personal demo :-)

In Digital Workplace Conference 2017, I want to talk about Serverless.

And I want to talk about humans.  Us.

I think the future will be amazing.  I hope to see you at the DWC Australia.  Come and grab me and say hello!

Reusing functions in PowerShell AzureFunctions

This is a pretty simple blog.  Take one of the examples I've been using often: 

https://github.com/johnnliu/azure-functions-o365/blob/master/sharepoint-list-email.ps1

This PowerShell uses PnP-PowerShell to:

• Connect to SharePoint Online
• Pulls a list from Document Library
• Format as HTML Table
• Send to an email with SPO SendMail utility endpoint

One of the most common, repeated step I do in almost every Function is to get credential and authenticate.  So I decided to put it into a shared function.

Refactor this into a separate function

Use the file navigator on the right hand pane - add a new file, call it "shared.psm1" this is a PowerShell Module file.

Create a function get-cred() and put the 4 lines of getting $username and encode PSCredential into this function, then return $creds

Finally, reference the module via:

Import-Module "D:\home\site\wwwroot\get-list-and-email\shared.psm1"

# call get-cred inline here
Connect-PnPOnline -url $siteUrl -Credentials (get-cred)

The path is the name of the function, that points to the current directory.
From now on, in every other function, you can just import that module to share the function.

If you use a separate shared directory under the wwwroot\shared level - that's a good place to put these shared modules too.  But note that you can't access that area via the Files right-hand pane.  You'll need to go there via the Kudu interface.

I consider putting get-cred away as a preparation step for one day in the future where I would replace that function with a call to Azure KeyVault to obtain the PSCredential object. When that refactoring happens, I will only need to update one place.

AzureFunctions Work Fan-out with Azure Queue in PowerShell

So I really like using PnP-PowerShell to chain up and perform complex operations in Office 365, and linking them up with AzureFunctions and Flow.

Scenario - scan all tenant's site collections

I bump into another problem today - I needed to scan all my site collections within the tenancy and start a Flow that will notify and apply site closure policy and lock the site.

We can scan all the site collections in a tenant in one request Get-PnPTenantSites - but we want to make sure the job doesn't time out.

So we need to fan-out the workload to a queue, and trigger multiple AzureFunctions to scan each site collection in parallel.

Problem - PoSH Queue binding only 1 output

As soon as I started writing the PoSH - I remembered, with the default PoSH Queue binding - you can only write 1 message to the Queue.

Unlike C# where you could do multiple:

foreach(var message in messages) {
    await outQueue.AddAsync(message);
}

In PoSH - if you are using the default integration tab to set up an Output Binding to AzureQueue.  Then you can only write one message to the Queue.

How to write multiple messages to Queue in PoSH?

It turns out I've already solved this once before in April, but I had completely forgotten this, because I DIDN'T BLOG IT.
Let that be a lesson to all developers - Always blog something cool that you did.  Because you will need it in two months when your memory failed you.

# if you have been using the storage in other functions 
# you will already have the connection string in your 
# function's app settings - reuse it

$storeAuthContext = New-AzureStorageContext -ConnectionString $env:azurefunctions3a585851_STORAGE 

$outQueue = Get-AzureStorageQueue –Name 'my-queue-name' -Context $storeAuthContext
if ($outQueue -eq $null) {
    $outQueue = New-AzureStorageQueue –Name 'my-queue-name' -Context $storeAuthContext
}

# this example isn't scanning sites - just going through files in a library
$items | % {
    
    $item = @{
        source = $_.FieldValues.FileRef;
        target = ($destination + "/" + $_.FieldValues.FileLeafRef)
    }

    # Create a new message using a constructor of the CloudQueueMessage class.
    $queueMessage = New-Object `
        -TypeName Microsoft.WindowsAzure.Storage.Queue.CloudQueueMessage `
        -ArgumentList (ConvertTo-Json $item)

    # Add a new message to the queue.
    $outQueue.CloudQueue.AddMessage($queueMessage)
}