Setting up MSGraph Webhook with HTTP Action in MicrosoftFlow

/

I've tweeted out several small tidbits of using Microsoft Flow's HTTP action to call the Microsoft Graph.

Hundreds of Graph APIs, dozens of Graph webhooks, one HTTP Action.

This little action continues to amaze me, so I'm putting several examples into this one blog post.

Four Techniques, One Action

  • Connect to Microsoft Graph with ONE HTTP Action 
  • Setting up Microsoft Graph Webhook Subscription
  • Paging
  • Retry Policy

One HTTP Action 

When we set Authentication to "Active Directory OAuth" - we can specify the Client ID / Client Secret in one HTTP action - so we don't need to make two separate calls first to Authenticate and get an access token, then call the flow we want and add the bearer header token.

This one action does it and asks no questions.

  • Tenant, Client ID and Client Secret are 3 strings. 
  • Authority should be set to https://login.microsoftonline.com/
  • Audience (resource) should be set to https://graph.microsoft.com

So yep, it's now easier to call any Microsoft Graph API from Flow than C#, PowerShell or JavaScript.

I'm in love with this, because this is way too amazing.

 

Note - this calls Flow via an App-Only Client ID.  If you are looking for delegate calls, you'll need to set up a Custom Connection swagger file.  Follow @skillriver https://gotoguy.blog/2017/12/17/access-microsoft-graph-api-using-custom-connector-in-powerapps-and-flows/

 

Microsoft Graph Webhook - a dozen new triggers

To set up webhook, we need to set up two Flows.  The First one is the subscriber.

The subscriber should be set up with a recurring 3 day schedule.

The notificationUrl is the HTTP Trigger URL of the second Flow.   

The expiry is 4229 minutes into the future from right now.  The maximum value is 4230 minutes.  If you go over the subscription call will fail.

On success, the subscription is set up and we are now listening to changes in our tenant's groups.

The Second one is the listener.

The listener needs to handle the validationToken that Microsoft Flow will call to test if your webservice follows the specs.

Read the trigger query string to pull out the validationToken.  if the value exists - then this is a set up call.  Respond immediately with text/plain 200 text.

Otherwise, we have a real call.  This is an event where our resource (in my case, I'm listening to Unified Groups being created and modified in my tenant) tells us something is happening. 

I call another Flow to start dealing with the change.  The other Flow does not return a response.  So the HTTP action can happen quickly as a trigger, and I can return 202 accept response to Microsoft Graph quickly.

 

This is the message sent from Microsoft Graph to Flow to tell me I've got a new Group created in my tenant.

Episode III in my blog series on Group Management with Flow will cover the webhook in more detail, combining it with a delta query and figuring out what changed.

 

Pagination

It turns out HTTP Pagination is baked in too.

To set this up - first set a $top in the Microsoft Graph call to artificially limit the number of rows returned.

This will also return a $nextLink

https://developer.microsoft.com/en-us/graph/docs/concepts/paging

Flip to the Settings for the HTTP action by clicking on ...
This lets you turn on Pagination and it will accept up to 5000 items.

The result is that the HTTP action will follow next page links automatically, and return you the entire array of the paged data concatenated together.

Magic.  Still one action.

Retry Policy

When calling HTTP (or other ApiConnections, like SharePoint) the default policy of retry 4 times exponentially means that if your action is going to fail, it will fail four times.

When building a Flow and you want it to fail fast - set the Retry Policy to None.

This is useful for calling AzureFunctions as well.  AzureFunction through host.json lets us control de-queuing speed and concurrency, but that requires you to use a Queue trigger.  When we use HTTP action from Flow - we control the retry / ease off policy through this setting.  In this use case, Flow is the orchestrater.

 

Summary

Calling Microsoft Graph directly with as simple as 1 HTTP request action means that a lot of slightly-more complex task of authenticating, then getting an access token, then calling MSGraph with bearer header becomes a whole lot easier.

And as steps get simpler, we can do a whole lot more.

 

 

Office 365 Groups Management As A Service: Episode II: Know your groups with Flow, MSGraph

/

This is the second post on building a group management tool with Flow and MSGraph.  In this post we talk about how to get a list of all your groups and copy them to a list in SharePoint so you can do more fancy things with them.

Episode I: Create Groups

Plan

  • Call MSGraph with Flow to get a list of all your Office 365 Groups
  • Create a SharePoint list to store them
  • Create/Update SharePoint list items
    (these steps above are good enough, the steps below are bonus points)
     
  • Delta Query
  • More Details
  • Parallel Execution
  • [NEW] Paging (if you have more than 100 groups) - suggestion to this post by @mikaelsvenson

MSGraph to List Groups

Some of you may have seem my love letter tweet.

  • MS Graph endpoint is https://graph.microsoft.com/v1.0/groups - docs 
  • Authority is https://login.microsoftonline.com/
  • Audience is Resource: https://graph.microsoft.com
  • Tenant ID, Client ID and Client Secret you'll need to register an App-Only credential app to get these.  Grant that App permission to Read Groups

 

Create a SharePoint List to store them

Complete the Flow to store Groups to SharePoint

This picture is complex, because it has a lot of tricky parts that's mostly caused by the way Flow's Editor works.  Flow's editor will try to filter the variables available to you by type, the Parse JSON action will parse JSON into a strongly type object following a strict schema.  This lets your subsequent actions work well.

BUT when your action returns null then suddenly the Parse JSON action can fail.  You'll then need to go back to the schema, and change some of the error "types" to "any".  You also may need to remove some of the fields from "required"

A few tweaks to the Parse JSON:

These tweaks are necessary because Parse JSON will fail if one of your groups don't have a createDateTime, classification or displayName.  You need to run this on your environment and check whether the fields need to be required or tweaked.

 

Paging

 

 

 

Delta Query

Because MS Graph supports Delta Queries - we can call the Graph with a Delta Query link, essentially, each time we call it, it'll return a nextLink (or a deltaLink).  So we just remember that, and use it the next time we want to call MS Graph again, and it will tell us just the differences.

See the delta query picked up one new Group I created for testing.

With a delta query - you can reliably set the Flow to run several times a day, and it will only send new/updated Groups back into the SharePoint list.

 

 

More Group Details

Sometimes you want more information on that group.  You can call Get group (on Azure AD connector) to get more information on the group.

Parallel 

We can configure the foreach action in Flow to run in parallel with 20 concurrency.  You'll need to add this to the definition JSON.

parallel.png
"runtimeConfiguration": { "concurrency": { "repetitions": 20 } }
flow-parallel.png

See the task of writing 19 Office 365 groups to sharepoint list was done in 4 seconds.

Result

 

[New] Paging

This section is added as @mikaelsvenson pointed out that I wasn't handling paging for groups.  Mikael also told me to use $top so I don't have to create a few hundred groups for testing.

Always listen to Mikael.

But I don't want to do loops - I've seen a pagination control, I want to know what that does.

 

Go to the settings for HTTP action and the first one is Pagination.
Turn that on, and set the limit to 5000.

  • Pagination controls seems to merge the results of multiple requests into one value array.  This is good.
  • Pagination controls works on Delta query - this is also good.
  • Pagination results does not return the deltaLink for next Delta query.  This is not so good.  So if we are planning to merge both Pagination and Delta Query we'll need to may be make two calls.

Notes

  • Use HTTP with Azure AD Auth to get all my groups in one call
  • Save that to SharePoint
  • Deal with Delta Queries with Flow
  • Configure Parallel execution in Flow so we can do this super fast.  This isn't code that runs one group at a time...  why would we do that when we can hit 20 at a time :-) 

 

 

Difference between beta, edu and v1.0 of MSGraph #microblog

/

I find this interesting - source: digging around MicrosoftTeams powershell and Mikael Svenson's blog post on enabling Teams programmatically. 

What's funny, because what I found strange is when Mikael says this:

The creation of the group itself happens against the /edu/groups which I’ve never seen before, but that’s not interesting.

Because that's the bit I found totally interesting.  Why does the PowerShell need the /edu/ endpoint?
Hop over to Graph Explorer we can play with this:

msgraph-edu.png

 

Comparing different endpoint: v1.0

endpoint: beta

endpoint: edu

It becomes clear why we need the /edu/ endpoint.  It has this bit of information:

"creationOptions": [
    "SkypeSpaces",
    "ExchangeProvisioningFlags:481"
],

The hint that a Team is provisioned seems to be the flag "SkypeSpaces"

Interesting.

Naturally, one would ask.  So if I want to enable Yammer, Planner or PowerBI on an existing Unified Group.  Do I POST an update to creationOptions?

Very.  Interesting.

I'm speaking about Serverless Flow and Azure Functions at Collab365 Free Online Conference

/
collab365-watch-my-session.jpg

Have you heard about the virtual Collab365 Global Conference 2017 that’s streaming online November 1st – 2nd?

Join me and 120 other speakers from around the world who will be bringing you the very latest content around SharePoint, Office 365, Flow, PowerApps, Azure, OneDrive for Business and of course the increasingly popular Microsoft Teams. The event is produced by the Collab365 Community and is entirely free to attend.

Places are limited to 5000 so be quick and register now.

During the conference I'd love you to watch my session which is called : 

'Serverless with Microsoft Flow and Azure Functions'

Level up your mastery of Microsoft Flow. Switch to Azure Functions only when you need to. No doubt there will be many sessions on Microsoft Flow, introducing you to its wonderful merits and rough edges. This session is for the advanced users - we will see what Microsoft Flow really is, and bend it to our will.

If you join me, you will learn:

  • Master JSON in a Flow
  • Combining Azure Functions with Flow
  • Failure Recovery, in a Flow
  • How to handle Binary in a Flow
  • How to write HTML and generate PDF in Flow

Topic(s):

  • Azure Functions
  • Microsoft Flow

Audience :

  • Developer
  • Power User

Time :

  • Thursday, November 2 2017 10:00 AM (UTC)
  • Thursday, November 2 2017 09:00 PM (ADST)

How to attend :

  1. Register here.
  2. At the time listed above go here to watch my session. (you can also add me to your own personal planner from the agenda.
  3. Enjoy the demos and ask me questions, I'll put the templates up for download after the session.

Two ways to convert SharePoint files to PDF via Flow

/

This blog post is divided into three sections: The easy, The Auth and The Complete parts.

Microsoft Flow released a new power to Convert Files to PDF.  This made my October.  So of course we have to play with this.

Part 1. The Easy

Now this work well, but raises a few questions: 

  1. Why do I have to copy to OneDrive for Business?
    Because the Convert File action is also available for OneDrive for consumer, but not SharePoint
     
  2. Can I do this without copying to OneDrive for Business
    Not with the default Actions for now.  There's no Convert File for SharePoint Connector.  And SharePoint Connector's Get File Content action doesn't allow a format parameter.
convert-file-actions.png

And this is the simplest solution.

Warning: Next be dragons (Auth and API)

We are going to dive in to see what API this uses.  And whether we can call the same API on SharePoint library document directly without copying the file to OneDrive first.

This next part is good for you.  But it is heavy and will look complicated.  Brace yourselves.

...So what API does this use?

https://docs.microsoft.com/en-us/onedrive/developer/rest-api/api/driveitem_get_content_format 

GET /drive/items/{item-id}/content?format={format}
GET /drive/root:/{path and filename}:/content?format={format}

Specifically, this uses the Microsoft Graph

Part 2. The Auth

Disclaimer - OAuth looks familiar, but steps are always tricky.  Easy to mess up.  So if you are following this through, walk carefully.

For the next part, we need to connect to MS Graph with AppOnly permissions

In Azure Portal - under Azure AD - create an App Registeration (I'm reusing a powershell-group-app one I had previously baked)

client-id.png

We will be accessing files - so make sure Application Permissions for read files is granted.  This requires admin consent.

client-perms.png

Via the Azure AD portal - hit Grant Permissions to perform admin consent directly.

client-grant.png

Now we are going to write the Flow with HTTP requests

hit the token endpoint for our tenant with a POST message.  The Body must be grant_type=client_credential with client_id, client_secret and the resource is https://graph.microsoft.com

this request if successful will give us back a JSON.  Parse JSON with this schema:

{
    "type": "object",
    "properties": {
        "token_type": {
            "type": "string"
        },
        "expires_in": {
            "type": "string"
        },
        "ext_expires_in": {
            "type": "string"
        },
        "expires_on": {
            "type": "string"
        },
        "not_before": {
            "type": "string"
        },
        "resource": {
            "type": "string"
        },
        "access_token": {
            "type": "string"
        }
    }
}

This gives Flow a variable for access_token for the remainder of the steps to use to call Microsoft Graph

Test this by calling the MS Graph endpoint for SharePoint site

token-test.png

This HTTP request with the Bearer access_token successfully returns SharePoint site data from Microsoft Graph.

 

Part 3.  The Complete Solution to fetch SharePoint document as PDF

Call /content?format=PDF

get-content-format-redirect.png

A few things going on in this result.  

  1. Flow thinks this request has failed - because it doesn't return a 2xx status.  It returns a 302 redirect.
  2. The Response header contains the Location of the redirect, which is where the PDF file is

Parse JSON again on the Response header.  

{
    "type": "object",
    "properties": {
        "Transfer-Encoding": {
            "type": "string"
        },
        "request-id": {
            "type": "string"
        },
        "client-request-id": {
            "type": "string"
        },
        "x-ms-ags-diagnostic": {
            "type": "string"
        },
        "Duration": {
            "type": "string"
        },
        "Cache-Control": {
            "type": "string"
        },
        "Date": {
            "type": "string"
        },
        "Location": {
            "type": "string"
        },
        "Content-Type": {
            "type": "string"
        },
        "Content-Length": {
            "type": "string"
        }
    }
}

We just want Location.  We also need to configure Continue on previous HTTP error.

redirect-continue.png

And finally, retrieve the file via GET again

fetch-return.png

 

When ran, the flow looks like this:

run.png

 

Summary

The complete solution uses HTTP to call MS Graph directly and pulls back the PDF file after a 302 Response.  This is a fairly complex example so please evaluate whether you want the Correct Way or the Easy Way.

Note also that Microsoft Flow has a Premium connector for Azure AD Requests - which will negate the middle part of this blog post re: Auth and let you dive right into MS Graph REST endpoints without worrying about access_tokens.  

Call this Flow request and it downloads the PDF file, converted from a DOCX document in SharePoint team site.

 

Review Special Techniques Invoked:

  • MS Graph Auth
  • The Continue on Error configuration
  • Parse JSON on Response Header