Make a wishing wand - why we need a MicrosoftFlow Custom Connector for MSGraph $Batch

/

A question that has been in the back of my mind since my late night demo of building a Microsoft Flow Custom Connector for Microsoft Graph is this.

I focused on the "how" - because I wanted to show that the whole thing can be done in about 10 minutes.  And I didn't want to skip steps like "oh here's how you register an Azure AD app".

 

But (and this is my fault, I did only ask for 10 minutes) I should have spent some time to talk about the "why"

Why do we need this?  What problem does this solve?

Why does John think this is super special?

This is special because, a $Batch connector is a "Send HTTP Request to MS Graph" as me.

It is a wishing wand that grants wishes. Creating MSTeam is a wish. We didn't build a wish. We built a wishing wand. Then we waved it around.  MS Teams API currently is delegate permission only, so we call it through a $batch connector.

Planner API is delegate only.  Any time you see a delegate permission MSGraph you remember you have a wishing wand.

The Ingredients:

Microsoft Flow and Logic Apps has concept of connectors (these are essentially API definitions) and connections (these are instances of connectors bound to a resource as well as cached automatic authentication)

A Flow is a JSON description of how actions and triggers on the connection are chained together to perform automation.

We do have a general purpose HTTP action, and it works very well for app-only permissions that needs client-id/client-secret or client-cert.  But HTTP action is not backed by a connectors framework so it can't remember credentials.

To build a general purpose tool and remember credentials for Microsoft Graph, we have to build a custom connection.  This is basically a swagger API file that tells Flow and LogicApps what APIs exist on the resource.  In the case of MSGraph, that list is enormous.

The Recipe:

The crazy part though, is MSGraph provides a $batch endpoint, so the recipe for building the magic wand is simply this:

  1. Build a custom connection with one method - $batch - this is a POST
  2. Pre-authenticate it with a delegate permission
  3. Wrap requests (1 or many) as a JSON array and call them via the $batch connection.
  4. We have a magic wand

Agenda for the September community call included Microsoft Graph updates, Microsoft Graph and Microsoft Teams, and custom connectors for Microsoft Graph $Batch

Where do we go from here?

I demoed this with a Manual button trigger.  Jeremy mentioned (in the video) that this can easily be a SharePoint trigger.

This can also be a HTTP Request trigger, turning your Teams Provisioning into one simple HTTP Webservice.  You can hook that up to external systems, PowerApp or a bot.

A technical note

Sometimes you discover you need extra permissions on that Azure AD app.  We do this by going back to Azure AD portal and add the extra permissions.  You can "grant permission" through the AAD portal.

If the cached connection in Flow refuses to work, I find that we may need to right-click on the custom connection and re-authenticate.  Sometimes this requires us to switch to a different user then switch back.  Because Azure AD dynamic consent is just so hard to do.

But it will work.  Just need to toggle a few times.  I do this a lot.

Resources

 

Office 365 Saturday Adelaide - 2018 September 15

/
sps-adelaide.png

So it is said - there's always a SharePoint or Office 365 Saturday every Saturday around the world.

 

O365 Saturday Adelaide / SPS Saturday Adelaide

In less than two weeks, we'll converge in Adelaide Australia for September 15 2018 SharePoint and Office 365 Saturday.

This is a free event (held at Microsoft Adelaide this year) for Office 365 and SharePoint experts, developers, admin and powerusers to converge and talk about Office 365.

https://www.meetup.com/O365-Saturday/events/254139366/

This year I'm presenting an ever evolving talk around Microsoft Flow and what it can (and SharePoint's old Workflow platform can't do).

Hope to see you in Adelaide!

A small favour - share this with two people

Every year we reach out and people that come to the event tells us it's a great event but they wish they know about it earlier.  So here's our small ask:

If you are in Adelaide, please come, and please tell two other colleagues about this.
If you aren't in Adelaide - but you have colleagues in Adelaide, please tell them about this event.

 

 

 

Microsoft Graph Community Call September 2018 - $Batch and Flow

/

I wanted to write a quick blog post letting everyone know I've put my hands up to present a 10 minute demo on using a Custom Connection for Microsoft Graph $Batch in Microsoft Flow.

Custom Connection = Swagger = WSDL

In Microsoft Flow, Power Apps or Logic Apps, a Custom Connection is a Swagger/Open API definition that allows these low-code solutions access to a well defined API, so it knows what to call and how to call them.

For old API guys that knew what a SOAP Webservice looked like, this is the WSDL that tells other systems how to use this service.

In general, calling Microsoft Graph with App-Only Permissions requires only a client-id and client-secret pair, and we can do it directly via the HTTP action without needing a custom connection.

When we do need to create a custom connection, that's usually because we need to call something with Delegate permissions

Why Flow - Connectors Framework

One of the hidden supper power of Flow and Logic Apps is the connectors framework, which cache, renew and manages connections across various different APIs and auth methods.  

This means - you actually don't need to write any of this code.  Repeat - as a developer, you don't have to write code to manage auth.  Auth is encapsulated away and auth code is no longer your concern.

Why $Batch

You'll have to see the demo (I'll update this later).  Hey, I can't spill all the beans in one blog post.  This is a tease.

The result

If you want to automate any delegate-permission API calls on Microsoft Graph, then Microsoft Flow is the easiest way to do it - far easier than writing code, and this is considered a low-code solution, it's crazy nuts.

Resource

Link to the recording will be updated here once presentation is done and uploaded.

The future of Flow Studio - Trial

/

Hi everyone.  This is somewhat serious, and I really need your feedback.

I wanted to write this down and project my intentions clearly and well in advance.  This is something I really would want feedback regarding the future of Flow Studio.

 

Flow Studio today

I love working on Flow Studio, and I heard feedback all the time that there's more I can do with it, there is so much more.  It is always my intention that if I can make it into a product offering - it can supplement my income and I can wholly devote more working days to it.  It is very fulfilling work but very very tiring to work weekends and evenings and have a head full of grey hair from lack of sleep.

A few weeks ago - I implemented Google Analytics to try to figure out which feature area people are using that I need to improve on.  If you are interested I can share the results.  But (to my surprise), I'm now tracking 40 active users per week.  When I last guesstimated 3 weeks ago before I implemented analytics - I thought I had 10 recurring.

The most used activity is Expand - of course we all want to see our Flows, what's been running, and more importantly, what's been failing.

 

Immediately: announcing a new web-3.0 URL

Here's a new thing.  You can use this shiny new URL that makes this a web 3.0 application.  A new URL means we are in serious business.

https://flowstudio.app 

The old address will continue to work, and they both map to the same application at the moment, in the future I consider switching the old URL to an insider/test build.

 

Flow Studio tomorrow

Flow Studio is free.  Whether you add a subscription or not.

The current functionality, navigation, filter, sort, tag will all sit in the Flow Studio base product and provided for free indefinitely.

For the subscription, my thinking is to chase after two particular Stories:

The Flow Studio Professional scenario and the Flow Studio Administrator scenario

Flow Studio will exit Open-Alpha status today, from tomorrow it will be Open-Trial status.  Everyone is on trail status.  There's no subscriptions built yet so it's just a name that implies there's more stuff later.  I will also start to label functions as Pro or Admin.

 

Flow Studio Pro

Flow Studio Pro is a subscription offered for Professionals, Consultants, Makers.  The main work here is to help you make more Flows, quicker and better Flows.

  • I am thinking to move the Edit JSON capability here - because I think the users that really benefit from this feature are super Flow users and will be happy to pay a small subscription.  This is also an area that I think I'll need to build more protection to help people feel safer modifying their Flows and not worry about breaking a Flow.
     
  • I'm thinking to offer a rudimentary backup/restore to give people some assurances.
     
  • The ultimate goal is to provide more Power User focused features.  Do you know you can rename actions with Flow Studio?  (this feature will also move under PRO)
     
  • I am thinking to charge US$5/m per account (email) or $10/m for 3 emails (I think the 3x offering is better suited for teams or for consultants that work on multiple projects - you can choose which 3 emails as you please)

    I have no idea if that's too little or too much.  You have to tell me.  I also think it is a subscription you can turn off and on as you like - if you aren't using Flow anymore on your project just stop the subscription and resume it later. 
     
  • Flow Studio doesn't go into your environment and delete anything when you unsubscribe.  Don't worry!
     
  • Without a subscription the Pro and Admin functions are disabled.  Flow Studio will continue to function.

 

Flow Studio Admin

The Flow Admin scenario is a subscription offered for Administrators - the main work here is to help you see all the Flows within your organization, set up triggers, reports and governance policies when things happen.  Turn off Flows when it exceeds run limits, when it uses specific connectors, what happens to the Flow when its owner leaves the organization etc etc.  What happens when a Flow fail?  When it fails a lot?

  • I am thinking $30/m for Flow Studio Admin - the Flow Admin may need Flow Plan 2 for access to the /admin REST APIs.  So I'm cautious what is coming out of the box with Plan 2, and what I plan to build as extras on top.
     
  • Ultimately, I want Admins to have better visibility into what they have running in their environment, help them build triggers and governance flows (even if they unsubscribe later) and help them feel comfortable configuring Flow to suit their business.
     
  • There will be trials for all these subscriptions so you can see the functionality.  Flow Studio Admin is aimed to be purchased by an IT department and has visibility into thousands of Flows.

 

Variables

US$5, $10 and $30 are monthly recurring subscription costs.  You can switch them on or off from month to month.

But I really want to know what you think.  Ask yourself this: if Flow Studio is $5/m will you buy it now, or will you not.  Tell me what you think.

 

Please tell me what you think

This is really important to me.  Is it too cheap.  Is it too expensive.  Does this help you.  Does this not help you.  What are your worries.

If you are a fan of Flow Studio or a fan of building start ups, I want to hear from you.

If you think Flow is wonderful but nobody should have to learn expressions - I really want to hear from you.  How can I help you, for only $5/month.

My lines are all open:

https://twitter.com/johnnliu
https://www.linkedin.com/in/johnnliu/ (please mention re: flow studio when connecting)
https://github.com/johnnliu/Flow-Studio-Issues/issues

or comment below

 

 

 

Post-DWCAU Conference - 7* flows in 1 hour

/

I'm composing this post on a sleepy Friday morning, still in Melbourne, on the day after the Digital Workplace Conference (Also Australian SharePoint Conference).  It is still my favourite conference.  This year, I had a session at the end, and helped out in a hackathon-styled PowerApps training.

Thank You

Thank you - the attendees that came to the sessions and the hackathon. 
Thank you to Debbie and Mark for putting together another great conference.
Thank you for the many speakers that flew in to present.  It is always such a thrill and pleasure to meet everyone down under.  
Thank you to Paul and Ashlee for running a PowerApps (with a bit of Flow) hackathon.

No thank-you to the LEGO Voltron I didn't win.  The guilty vendor-name is withheld you know who you are.

 

Presentation

I presented a talk on "Functions and Flow" - the plan was to cover increasing our serverless toolkit with both Azure Functions and Microsoft Flow.  But because I see my time is slotted near the end of the late afternoon, I didn't really want to do slides - so I just went with lets do fun demos.

There is a roadmap to the list of Flows that I demo'ed in 1 hour.  In retrospection, I should have left a slide up while I do my demos of this roadmap. 

So imagine this list is that missing slide, giving you the overview of what we are covering as we progress.  Shockingly, the Flows aren't just 'randomly selected' they actually follow a story.

This is the roadmap, with links to each Flow:

  1. The Only "Workflow" in the demo
    How to do parallel approvals with time-out escalation and manager lookup.  The pattern is more complex than one single action with settings, but gives you a LOT of flexibility to twist how you want to design your approval conditions.
    https://github.com/johnnliu/flow/blob/master/Approvals-Demo_20180818224437.zip

    My example is inspired by Serge Luca's "Escalate Approvals" posts on approval scenarios
    https://powerusers.microsoft.com/t5/Microsoft-Flow-Community-Blog/Microsoft-Flow-How-to-Escalate-Approvals/ba-p/93773

    Also:  time patterns:
    PT1M = 1 minute
    P1M = 1 month (remember Flow time out in 31 days, so this pattern won't be too useful, unless you want to trigger a new Flow when this one time out)
    P14D = 14 days
     
  2. Flow does binary natively
    Get SharePoint File from HTTP Request
    https://github.com/johnnliu/flow/blob/master/SPFlowProxy_20180818225415.zip
     
  3. Making new Files in Flow
    Generate PDF from SharePoint files, including pictures
    http://johnliu.net/blog/2017/10/generate-any-pdf-documents-from-html-with-flow
     
  4. Making Actions in Flow
    Azure Function and Flow to run any PnP-PowerShell (350 cmdlets)
    http://johnliu.net/blog/2018/4/run-any-pnp-powershell-in-one-azurefunction-from-microsoft-flow

    This can be deployed via 1 ZIP file
    http://johnliu.net/blog/2018/4/run-any-pnp-powershell-via-drag-and-drop-zip-to-azurefunctions 
     
  5. Making Actions in Flow (make Teams)
    This requires a Batch Custom Connection - I'll be demo'ing this in a future MSGraph call
     
  6. Making Triggers in Flow
    Convert MSGraph webhook to an automatic trigger (this one is a pair of 2 flows, but I count this as one demo)
    http://johnliu.net/blog/2018/1/setting-up-msgraph-webhook-with-http-action-in-microsoftflow
     
  7. Sometimes, we just want an over the top demo for On-Premises data gateway.
    http://johnliu.net/blog/2017/10/from-office-365-to-minecraft-connected-with-flow
     

Questions

  1. A question about on-premises PowerShell
    The best solution is to look at Azure Automation's Hybrid Runbooks
     
  2. A question about very complex approval sequences - I suggested implementing a State Machine described by Serge Luca
    https://powerusers.microsoft.com/t5/Microsoft-Flow-Community-Blog/Simulate-state-machines-with-Microsoft-Flow/ba-p/92211

 

Flow Studio

Because I'm a terrible salesman, I forgot to mention anything about my Flow utility app "Flow Studio" - I actually used it to select a subset of the Flows I present.  So you were looking at it, I just didn't say anything about it.

Try it free: https://flow-studio.azurewebsites.net

Flow Studio is a utility toolbox of features I added to help me manage, organize and build more Flows.  It is in Open-Alpha status.  If you want to get some great ideas about what's going on with ALL your Flows and not just individual ones, Flow Studio is really good at that.

Summary

I have done several talks in Microsoft Flow - and every time we increase in speed and complexity of the demos.  Because Flow is connecting boxes - once we all get past "action" connects to another "action", suddenly we find ourselves talking about connecting HTTP API and Webhook triggers.

That is actually the final thought I wanted to end on.  Flow makes things deceptively simple, but they aren't steps that seems impossible.  They look like tiny steps.  They represent giant leaps.

We step off the cliff, and we fly.

 

Connect

 

A Request

I didn't have any pictures from this session!  I forgot!!  If you have some could you tweet them at me (or send them through LinkedIn)?  I'm looking particularly for a picture that shows the size of the crowd that afternoon.