Office 365 Groups Management As A Service: Episode II: Know your groups with Flow, MSGraph

This is the second post on building a group management tool with Flow and MSGraph.  In this post we talk about how to get a list of all your groups and copy them to a list in SharePoint so you can do more fancy things with them.

Episode I: Create Groups

Plan

  • Call MSGraph with Flow to get a list of all your Office 365 Groups
  • Create a SharePoint list to store them
  • Create/Update SharePoint list items
    (these steps above are good enough, the steps below are bonus points)
     
  • Delta Query
  • More Details
  • Parallel Execution
  • [NEW] Paging (if you have more than 100 groups) - suggestion to this post by @mikaelsvenson

MSGraph to List Groups

Some of you may have seem my love letter tweet.

  • MS Graph endpoint is https://graph.microsoft.com/v1.0/groups - docs 
  • Authority is https://login.microsoftonline.com/
  • Audience is Resource: https://graph.microsoft.com
  • Tenant ID, Client ID and Client Secret you'll need to register an App-Only credential app to get these.  Grant that App permission to Read Groups

 

Create a SharePoint List to store them

Complete the Flow to store Groups to SharePoint

This picture is complex, because it has a lot of tricky parts that's mostly caused by the way Flow's Editor works.  Flow's editor will try to filter the variables available to you by type, the Parse JSON action will parse JSON into a strongly type object following a strict schema.  This lets your subsequent actions work well.

BUT when your action returns null then suddenly the Parse JSON action can fail.  You'll then need to go back to the schema, and change some of the error "types" to "any".  You also may need to remove some of the fields from "required"

A few tweaks to the Parse JSON:

These tweaks are necessary because Parse JSON will fail if one of your groups don't have a createDateTime, classification or displayName.  You need to run this on your environment and check whether the fields need to be required or tweaked.

 

Paging

 

 

 

Delta Query

Because MS Graph supports Delta Queries - we can call the Graph with a Delta Query link, essentially, each time we call it, it'll return a nextLink (or a deltaLink).  So we just remember that, and use it the next time we want to call MS Graph again, and it will tell us just the differences.

See the delta query picked up one new Group I created for testing.

With a delta query - you can reliably set the Flow to run several times a day, and it will only send new/updated Groups back into the SharePoint list.

 

 

More Group Details

Sometimes you want more information on that group.  You can call Get group (on Azure AD connector) to get more information on the group.

Parallel 

We can configure the foreach action in Flow to run in parallel with 20 concurrency.  You'll need to add this to the definition JSON.

parallel.png
"runtimeConfiguration": { "concurrency": { "repetitions": 20 } }
flow-parallel.png

See the task of writing 19 Office 365 groups to sharepoint list was done in 4 seconds.

Result

 

[New] Paging

This section is added as @mikaelsvenson pointed out that I wasn't handling paging for groups.  Mikael also told me to use $top so I don't have to create a few hundred groups for testing.

Always listen to Mikael.

But I don't want to do loops - I've seen a pagination control, I want to know what that does.

 

Go to the settings for HTTP action and the first one is Pagination.
Turn that on, and set the limit to 5000.

  • Pagination controls seems to merge the results of multiple requests into one value array.  This is good.
  • Pagination controls works on Delta query - this is also good.
  • Pagination results does not return the deltaLink for next Delta query.  This is not so good.  So if we are planning to merge both Pagination and Delta Query we'll need to may be make two calls.

Notes

  • Use HTTP with Azure AD Auth to get all my groups in one call
  • Save that to SharePoint
  • Deal with Delta Queries with Flow
  • Configure Parallel execution in Flow so we can do this super fast.  This isn't code that runs one group at a time...  why would we do that when we can hit 20 at a time :-) 

 

 

Difference between beta, edu and v1.0 of MSGraph #microblog

I find this interesting - source: digging around MicrosoftTeams powershell and Mikael Svenson's blog post on enabling Teams programmatically

What's funny, because what I found strange is when Mikael says this:

The creation of the group itself happens against the /edu/groups which I’ve never seen before, but that’s not interesting.

Because that's the bit I found totally interesting.  Why does the PowerShell need the /edu/ endpoint?
Hop over to Graph Explorer we can play with this:

msgraph-edu.png

 

Comparing different endpoint: v1.0

endpoint: beta

endpoint: edu

It becomes clear why we need the /edu/ endpoint.  It has this bit of information:

"creationOptions": [
    "SkypeSpaces",
    "ExchangeProvisioningFlags:481"
],

The hint that a Team is provisioned seems to be the flag "SkypeSpaces"

Interesting.

Naturally, one would ask.  So if I want to enable Yammer, Planner or PowerBI on an existing Unified Group.  Do I POST an update to creationOptions?

Very.  Interesting.

I'm speaking about Serverless Flow and Azure Functions at Collab365 Free Online Conference

collab365-watch-my-session.jpg

Have you heard about the virtual Collab365 Global Conference 2017 that’s streaming online November 1st – 2nd?

Join me and 120 other speakers from around the world who will be bringing you the very latest content around SharePoint, Office 365, Flow, PowerApps, Azure, OneDrive for Business and of course the increasingly popular Microsoft Teams. The event is produced by the Collab365 Community and is entirely free to attend.

Places are limited to 5000 so be quick and register now.

During the conference I'd love you to watch my session which is called : 

'Serverless with Microsoft Flow and Azure Functions'

Level up your mastery of Microsoft Flow. Switch to Azure Functions only when you need to. No doubt there will be many sessions on Microsoft Flow, introducing you to its wonderful merits and rough edges. This session is for the advanced users - we will see what Microsoft Flow really is, and bend it to our will.

If you join me, you will learn:

  • Master JSON in a Flow
  • Combining Azure Functions with Flow
  • Failure Recovery, in a Flow
  • How to handle Binary in a Flow
  • How to write HTML and generate PDF in Flow

Topic(s):

  • Azure Functions
  • Microsoft Flow

Audience :

  • Developer
  • Power User

Time :

  • Thursday, November 2 2017 10:00 AM (UTC)
  • Thursday, November 2 2017 09:00 PM (ADST)

How to attend :

  1. Register here.
  2. At the time listed above go here to watch my session. (you can also add me to your own personal planner from the agenda.
  3. Enjoy the demos and ask me questions, I'll put the templates up for download after the session.

Two ways to convert SharePoint files to PDF via Flow

This blog post is divided into three sections: The easy, The Auth and The Complete parts.

Microsoft Flow released a new power to Convert Files to PDF.  This made my October.  So of course we have to play with this.

Part 1. The Easy

Now this work well, but raises a few questions: 

  1. Why do I have to copy to OneDrive for Business?
    Because the Convert File action is also available for OneDrive for consumer, but not SharePoint
     
  2. Can I do this without copying to OneDrive for Business
    Not with the default Actions for now.  There's no Convert File for SharePoint Connector.  And SharePoint Connector's Get File Content action doesn't allow a format parameter.
convert-file-actions.png

And this is the simplest solution.

Warning: Next be dragons (Auth and API)

We are going to dive in to see what API this uses.  And whether we can call the same API on SharePoint library document directly without copying the file to OneDrive first.

This next part is good for you.  But it is heavy and will look complicated.  Brace yourselves.

...So what API does this use?

https://docs.microsoft.com/en-us/onedrive/developer/rest-api/api/driveitem_get_content_format

GET /drive/items/{item-id}/content?format={format}
GET /drive/root:/{path and filename}:/content?format={format}

Specifically, this uses the Microsoft Graph

Part 2. The Auth

Disclaimer - OAuth looks familiar, but steps are always tricky.  Easy to mess up.  So if you are following this through, walk carefully.

For the next part, we need to connect to MS Graph with AppOnly permissions

In Azure Portal - under Azure AD - create an App Registeration (I'm reusing a powershell-group-app one I had previously baked)

client-id.png

We will be accessing files - so make sure Application Permissions for read files is granted.  This requires admin consent.

client-perms.png

Via the Azure AD portal - hit Grant Permissions to perform admin consent directly.

client-grant.png

Now we are going to write the Flow with HTTP requests

hit the token endpoint for our tenant with a POST message.  The Body must be grant_type=client_credential with client_id, client_secret and the resource is https://graph.microsoft.com

this request if successful will give us back a JSON.  Parse JSON with this schema:

{
    "type": "object",
    "properties": {
        "token_type": {
            "type": "string"
        },
        "expires_in": {
            "type": "string"
        },
        "ext_expires_in": {
            "type": "string"
        },
        "expires_on": {
            "type": "string"
        },
        "not_before": {
            "type": "string"
        },
        "resource": {
            "type": "string"
        },
        "access_token": {
            "type": "string"
        }
    }
}

This gives Flow a variable for access_token for the remainder of the steps to use to call Microsoft Graph

Test this by calling the MS Graph endpoint for SharePoint site

token-test.png

This HTTP request with the Bearer access_token successfully returns SharePoint site data from Microsoft Graph.

 

Part 3.  The Complete Solution to fetch SharePoint document as PDF

Call /content?format=PDF

get-content-format-redirect.png

A few things going on in this result.  

  1. Flow thinks this request has failed - because it doesn't return a 2xx status.  It returns a 302 redirect.
  2. The Response header contains the Location of the redirect, which is where the PDF file is

Parse JSON again on the Response header.  

{
    "type": "object",
    "properties": {
        "Transfer-Encoding": {
            "type": "string"
        },
        "request-id": {
            "type": "string"
        },
        "client-request-id": {
            "type": "string"
        },
        "x-ms-ags-diagnostic": {
            "type": "string"
        },
        "Duration": {
            "type": "string"
        },
        "Cache-Control": {
            "type": "string"
        },
        "Date": {
            "type": "string"
        },
        "Location": {
            "type": "string"
        },
        "Content-Type": {
            "type": "string"
        },
        "Content-Length": {
            "type": "string"
        }
    }
}

We just want Location.  We also need to configure Continue on previous HTTP error.

redirect-continue.png

And finally, retrieve the file via GET again

fetch-return.png

 

When ran, the flow looks like this:

run.png

 

Summary

The complete solution uses HTTP to call MS Graph directly and pulls back the PDF file after a 302 Response.  This is a fairly complex example so please evaluate whether you want the Correct Way or the Easy Way.

Note also that Microsoft Flow has a Premium connector for Azure AD Requests - which will negate the middle part of this blog post re: Auth and let you dive right into MS Graph REST endpoints without worrying about access_tokens.  

Call this Flow request and it downloads the PDF file, converted from a DOCX document in SharePoint team site.

 

Review Special Techniques Invoked:

  • MS Graph Auth
  • The Continue on Error configuration
  • Parse JSON on Response Header