Office 365 Groups management as a service - Flow, Functions and MSGraph

Because Office 365 Groups is a key component of group membership in Office 365, there will always be an evolving story on how to manage it, extend it, report on it and automate it.

Microsoft will continue to add more complex features to Azure AD Premium.  And this will be an enterprise grade solution to many customers that wants these features.  But there is a lot of room for partners to build Groups Management solutions.  

Ultimately, we have realized this: Our client's organization has unique rules, and there's a need to customize, fortunately, we have all the tools at our disposal, and they are not hard to do.

Group Management Life Cycle

This is a post in a series about Office 365 Groups management life cycle.  This is the first post - we will discuss the history Office 365 Groups creation.

Office 365 Groups Creation

There have been many blog posts about how we can automate Groups creation.  So this discussion is about understanding the underlying pieces, and how we must see these pieces as the building blocks to solve all your future problems.

View fullsize

Timeline of availability of O365 Groups creation methods

2015

• EXO Remote PowerShell - I find this post from 2015 by Wictor Wilen truly a trend setter.  http://www.wictorwilen.se/office-365-groups-for-admins-creating-groups

2016

• MS Graph API for group creation available
• PnP Sample Solution - Vesa demo'ed in late 2016 with bots and Azure App Service
• Azure Function enabled very simple hosting of small code

 2017

• January 2017 PowerShell Azure Functions with resource owner grant
• Feb 2017 PowerShell Azure Functions with AppOnly
• March 2017 PnP-PowerShell further simplified to two lines by @mikaelsvenson

    Connect-PnPMicrosoftGraph (PR by Mikael Svenson Feb 2017)
    New-PnPUnifiedGroup (available October 2016)

• http://blog.atwork.at/post/2017/09/28/Provisioning-an-Office-365-group-with-an-approval-flow-and-Azure-functions-part-1 by @magrom
• October 2017 Microsoft Flow HTTP Request to create groups by @sohailmerchant
• October 2017 Microsoft Flow added an Azure AD connector that can create Office 365 Group

View fullsize

One Flow Connector to rule them all

I only know this: Everything we know got simplified

So here we are.  At the end of the first blog post on Groups Management - we need to understand the trend:

First, we have API
Then we have PowerShell, we have AzureFunctions for code or HTTP Request in Flow
Eventually we have a Flow Connector

We must see this pattern.  Everything we want to build is in one of these stages.

When we understand these pieces - there's nothing we can't build, for practically free.  The question isn't "No, this is too hard" or even "This should be free or out of box".

The question we should all be asking is "Where do I find a piece that fits here, right now" because I have a crazy customization need.

In 2017, everything got abstracted and simplified.  This trend will continue into the future - there will be more Flow Connectors.  Azure Functions will get more upgrades - durable functions are absolutely amazing.  MS Graph will get more API endpoints.  Life will be even more amazing.

Future of this series

I am not alone in what we are building for our customers - many consultants, partners and ISVs have already moved on to more complex issues:

• Creation - Complex approval chains for Groups creation
• Creation - Which products to enable on Groups creation
• Creation - Post-Groups creation SharePoint site templating
• Maintain - Scheduled Groups compliance audit reporting 
• Maintain - Owner leaving organization scenario
• Maintain - Members changed roles scenario
• Closure - Expiration policies
• Closure - Group archiving and closure

There are a lot of solutions to solve.  I want to cover more of Office 365 Groups life cycle management, with Flow and Functions, all on top of what MS Graph gives us already.

If you are interested in this topic or have shared some of the ideas you are working on - please share them with me and I would be happy to link to your work.

Difference between beta, edu and v1.0 of MSGraph #microblog

I find this interesting - source: digging around MicrosoftTeams powershell and Mikael Svenson's blog post on enabling Teams programmatically. 

What's funny, because what I found strange is when Mikael says this:

The creation of the group itself happens against the /edu/groups which I’ve never seen before, but that’s not interesting.

Because that's the bit I found totally interesting.  Why does the PowerShell need the /edu/ endpoint?
Hop over to Graph Explorer we can play with this:

Comparing different endpoint: v1.0

endpoint: beta

endpoint: edu

It becomes clear why we need the /edu/ endpoint.  It has this bit of information:

"creationOptions": [
    "SkypeSpaces",
    "ExchangeProvisioningFlags:481"
],

The hint that a Team is provisioned seems to be the flag "SkypeSpaces"

Interesting.

Naturally, one would ask.  So if I want to enable Yammer, Planner or PowerBI on an existing Unified Group.  Do I POST an update to creationOptions?

Very.  Interesting.

I'm speaking about Serverless Flow and Azure Functions at Collab365 Free Online Conference

Have you heard about the virtual Collab365 Global Conference 2017 that’s streaming online November 1st – 2nd?

Join me and 120 other speakers from around the world who will be bringing you the very latest content around SharePoint, Office 365, Flow, PowerApps, Azure, OneDrive for Business and of course the increasingly popular Microsoft Teams. The event is produced by the Collab365 Community and is entirely free to attend.

Places are limited to 5000 so be quick and register now.

During the conference I'd love you to watch my session which is called : 

'Serverless with Microsoft Flow and Azure Functions'

Level up your mastery of Microsoft Flow. Switch to Azure Functions only when you need to. No doubt there will be many sessions on Microsoft Flow, introducing you to its wonderful merits and rough edges. This session is for the advanced users - we will see what Microsoft Flow really is, and bend it to our will.

If you join me, you will learn:

• Master JSON in a Flow
• Combining Azure Functions with Flow
• Failure Recovery, in a Flow
• How to handle Binary in a Flow
• How to write HTML and generate PDF in Flow

Topic(s):

• Azure Functions
• Microsoft Flow

Audience :

• Developer
• Power User

Time :

• Thursday, November 2 2017 10:00 AM (UTC)
• Thursday, November 2 2017 09:00 PM (ADST)

How to attend :

1. Register here.
2. At the time listed above go here to watch my session. (you can also add me to your own personal planner from the agenda.
3. Enjoy the demos and ask me questions, I'll put the templates up for download after the session.

From Office 365 to Azure to Minecraft, connected with Flow

"John, what's this headline."
This is just a love serenade to Microsoft's many engineers and teams.  Thank you, for making these products that makes these things look easy.

Thank you for showing the world what One Microsoft might look like when everything works together.

Plan

1. Minecraft Windows 10 edition in the latest 1.2 (better together update) included web sockets previously only in the Minecraft Education version.  Web Sockets lets you connect to a Minecraft game, and remotely execute commands.
2. Minecraft Code Connection is an external application that hosts a friendly REST API and translates JSON to web socket.  Previously this was only for Minecraft Education Edition.  An update in early October allows this to work with Minecraft Windows 10.
   https://makecode.com/blog/minecraft/10-18-2017
3. Microsoft Data Gateway allows Flow, PowerApps, PowerBI to talk to on-premises environments.  It also recently gained the ability to execute custom connections.  It is essentially, an enterprise data gateway / reverse proxy that connects your local environment to an Azure Service Bus.  The ability to call your custom local REST endpoint was released in September.
   https://flow.microsoft.com/en-us/blog/q3-2017-update/
4. Cloud based services like Flow/LogicApps, PowerApps, and PowerBI can talk to this connection online, via the magic of Azure Service Bus.
5. And because I'm a SharePoint MVP - we are triggering this Flow from SharePoint Online.  Because the world needs this.

Setting up Minecraft & Code Connection

https://www.microsoft.com/en-au/store/p/minecraft-for-windows-10/9nblggh2jhxj?rtc=1 
Buy or download Minecraft for Windows 10

https://education.minecraft.net/get-started/download/
Download Minecraft Code Connection

https://education.minecraft.net/support/knowledge-base/code-connection-api-documentation/
The API for the JSON messages to send to Code Connection is here.
 

View fullsize

Code Connection uses Minecraft commands - the option is called "activate cheats" - do this in a creative world.

Code Connection is a separate executable.  Run it outside of Minecraft - it'll ask you to enter the command into Minecraft

Use Postman to test your localhost:8080 and send a REST request.  You'll see your agent bot move.

Congrats - you now have a REST endpoint that can send game commands to your Minecraft game.

Setting up Flow Data Gateway

https://flow.microsoft.com/en-us/documentation/gateway-reference/ 

The data gateway also calls your local REST endpoints via a custom connector.  This was a feature that was silently released in the deluge of news from MSIgnite.  I need to thank @pratapladhani (PowerApps PM) for sending me the link of the announce.  It is REALLY obscure.

https://flow.microsoft.com/en-us/blog/q3-2017-update/

“On-premises connectivity to any HTTP API - Finally, users can now connect their own on-premises APIs with Custom Connectors - leveraging the On-Premises Data Gateway. For example, if you have a service that’s only available on your local network, you can now author a custom connector that can read data from or push data to that local service.”

Congrats - this is your reverse proxy.

Setting up custom connection via this Swagger File I've prepared for you

https://github.com/johnnliu/flow/blob/master/mc-cc.swagger.json 

Grab this swagger.json and add a custom connection to Flow

Note the host is localhost:8080
Tick: Connect via on-premise data gateway

Note the custom connection swagger file points to localhost:8080 - this will work if your data gateway is on the same machine as the Code Connection exe.  Otherwise, you should use the LAN IP Address of the Code Connection server.

Add a  connection, and select On-Premises data gateway

Note the connection type is "On-premises"

Write my Flow

These wonderful methods with dropdowns are defined in Swagger file.  It's hard work.  Contributions welcome!

Trigger it from Office 365 / SharePoint

Hi bot agent

Uploaded by John Liu on 2017-10-25.

Code-Less

I want you to understand, all these are done, without me writing a single line of C# or JavaScript.

Cost?

Office 365 E3+ license allows you to have:

• 2000 Flow runs per user
• 1 Custom Connection
• On-Premises data gateway allowed

https://australia.flow.microsoft.com/en-us/pricing/ 

So there's no extra cost for an Office 365 customer E3+

Summary Key Points

• Minecraft Win10 has websockets
• Code Connection does JSON to websocket
• Data Gateway is amazing, also calls your local REST endpoints
• Microsoft Flow just unlocks it to the rest of the world, if you like, LogicApps will work just as well.
• You can have a PowerApp calling the function via the Custom Connection, without Flow.
• Extremely code-less

Where do we go next?

Well, when I detect an Active Directory User deletion via webhook - I'm going to teleport this Zombie Villager into lava.

Update: I can't see a use case for this

So, some people have commented to me directly - John this is very cute, but there's no business use case for this.

Let me explain the head-fake

Flow can reach into your organization and call any of your REST endpoints with no code

Microsoft Flow blog posted https://flow.microsoft.com/en-us/blog/on-premise-apis/ explaining how you can build your own on-premises API service and call it from Flow via the data gateway.  I was bothering them with explanations for the data gateway and I think they were ready to publish anyway, they told me they'll share it in a few days.

I beat them slightly to publish since I worked out how to do the Minecraft bit.

I also think they need more imagination.  Minecraft is a hundred times better example and I didn't even have to write a REST service ;-)

<3

Auto-Classify Images in SharePoint Online library via Flow for Free

Microsoft Flow's most recent update added ability to query and update SharePoint File property.  This is actually really timely, as I wanted to combine this with a few other techniques and built a Document Library Image Auto-Classifier Flow.

Is that a clickbait headline?  Well it's totally real, and we'll build it in a moment.

Steps:

1. Set up your cognitive service account (understand the free bucket)
2. Set up a SharePoint Online document library with Categories
3. Set up the Flow file loop
4. Do a fancy JSON array to concatenated string projection operation with Select and Join
5. Viola, no code.  And pretty much *free*

This is part of a series on Microsoft Flow

Set up your Azure Cognitive Service instance

Follow these simple steps to create an Computer Vision API Cognitive Service in your Azure.  Computer Vision API has a free tier.

1. Create Computer Vision API

View fullsize

2. Scroll down and hit Create

View fullsize

3. Give this service a name, set up the region and select Free pricing tier

4. You need the endpoint url here

5. Also, copy the Name and key 1

You will need the "Name" and a "Key" for the next step.

The free tier of Computer Vision API - first 5000 transactions free per month.

Note the service isn't available in all regions.  Most of my stuff is in Australia East, but for the Cognitive Service API it has to be hosted in Southeast Asia.  YMMV.

View fullsize

Then we need to set up the connection in Flow

1. Find the Computer Vision API action

View fullsize

2. Enter service name, key and the root site url to set up the initial connection

View fullsize

3. Created correctly, you get an action like this

Set up the SharePoint Document Library

My SharePoint document library is very simple - it is just a basic document library, but I added an extra site column "Categories". This is an out of the box field, and is just a simple text field.

This is a simple step

Set up the Flow

I trigger the flow with a Scheduled Recurrence that runs once per day.
Using the new Get Files (properties only), I grab a list of all the files in a document library.
I then run for-each on the list of files.

View fullsize

Inside the for-each, I have a condition that checks if the Categories field is null.  If you type null directly into the field, you will get the string 'null'. 

Tip: To actually get the formulat/expression null, select Expressions and type null there.

View fullsize

If the Categories is null, then we proceed.

Grab the file content via Get file content
Call Computer Vision API with the image content.  Select the Image Source to binary, instead of URL.

Tip: I use a compose to see the debug results

I'll explain the array projection in the next section.

View fullsize

Select projection: JSON array to String array

We have an array of JSON objects:

[{
     'name': 'foo'
},
{
    'name': 'bar'
}]

This default UI maps to:

tags -> [{ specified properties }…]

The result is that we would end up with a new array of (simpler) JSON objects.
Hit advanced text mode.

Here, we can use Expression to say item('Tag_Image')?.name

In this case the UI is smart enough to show Tag.Name as a dynamic content (as well as the Tag.ConfidenceScore property).  So we can select that.

This performs a projection of

tags -> [ names… ]

We now have an array of strings.  Combine them via Join with a comma (,) separator.
Update the file properties with this string.

Lets see the results

I uploaded a few images to the library.
Note the categories field is blank.

View fullsize

Running the Flow

View fullsize

When it finishes, I'm checking the JSON - the picture is identified with a "person" with 99% confidence.
The combined string "person,young,posing" is updated into the File property.

View fullsize

The documents are updated.  When Flow runs tomorrow it will skip them.

View fullsize

The Final Flow

View fullsize