John Liu .NET

View Original

One Flow to handle them all - how to subscribe to multiple SharePoint lists with one Flow

John Liu

One Flow to handle them all - how to subscribe to multiple SharePoint lists with one Flow

At some point - we think hmm how do we save a Flow as a template and deploy it with all our SharePoint site collections…

and did we just create a massive problem down the road where we have all these duplicate Flows…

and how do we manage versioning and changes to them hmm lets have a think and talk about this.

So instead of all that copying. Let’s try a different idea - what if we have One Flow that Rules them all. It handles events from All the Sites and Libraries, all within one Flow.

Depending on what our Flow actually do - this might be a better approach.

Live Tweet

This is a long overdue write up of an idea of two Flow done as a hack over twitter.  https://twitter.com/johnnliu/status/1002747531506245632

Multi-Part Series: Deploy My Flow

This is a multi-part series called Deploy My Flow. I want to publish discussing the various ways to deploy Flows across SharePoint and Office 365, between dev, test and production environments. Across lists, sites, and tenants.

  • Export and Import

  • Automate - PowerShell, Flow Management and Flow Studio

  • Power Platform Solutions

  • Subscribe to multiple SharePoint Lists with One Flow

Part I - SharePoint Webhooks

A Flow can only listen/trigger from one list, so while in SharePoint as we provision sites - we consider having a way to provision templated Flows to handle each list. 

But let's turn the question around.  Can we have just one Flow to handle multiple lists?
Is it possible to have Flow connect directly to SharePoint's Webhooks and manage events directly with Flow?

 

Why this approach?

  • The Flow triggers listen to and raises events for each individual list or document library. So, out of the box, we can't create one Flow that listens to every list or library.

  • Cloning Flows multiple times has its own challenges:

    • 250 Flow limit (This limit is now 500, you can request a service ticket to raise this number) per account.

    • How do we manage multiple Flows, do we have one master? Does it get exported and imported over existing Flows when we upgrade? What happens with the connections?

    • Can we do these automatically?

  • There’s a lot to discuss, but for this blog post we will focus on a simple pattern

  1. Create a Handler Flow that handles list change events.

  2. Create a second Scheduler Flow that subscribes webhooks from multiple SharePoint lists onto the first Flow.

Reference

We don't start from scratch, we have the excellent materials on docs.microsoft.com from the SharePoint PnP team describing how to attach to SharePoint webhooks via code.

https://docs.microsoft.com/en-us/sharepoint/dev/apis/webhooks/overview-sharepoint-webhooks

We only need to translate this to Flow.

First Create the Scheduler Flow

First - here I’m reading all the lists from SharePoint that’s a document library. Then read the subscriptions on each document library. This end point shows me SharePoint’s webhooks.

TIP: Because I use odata=nometadata a LOT - I end up putting that header into a header_nometa JSON variable and insert it in every Send HTTP Request action.


Create Subscription and handle Resubscription

Next, we check if the webhooks contains has a clientState with the current Flow’s name (Flow names are unique guids).

If not, we create one. The check and filter for ClientState makes sure we don’t subscribe the same event handler to the same document library multiple times. It also makes this “scheduler” flow re-run safe. We can pretty much change the trigger to a recurrence trigger that runs as often as you’d like - say daily.

workflow()?['name']

The Handler

We need a new Flow to handle the document updating. This is the One Flow that does all the work.

Save the Handler Flow and take the HTTP Reques URL back to our Subscriber Flow

And this is the final result - when we run the Subscriber Flow - it iterates through all my document libraries and connects them all to the handler Flow.

Now when documents are updated in these 12 document libraries, my “Handler Flow” will be triggered by SharePoint.

SharePoint webhook subscriptions are valid for 180 days. So at some point our Scheduler Flow will re-run on recurrence and re-subscribe the document libraries. (We can also handle the True condition in the Scheduler Flow to renew webhooks).

Part 2 - Get Changes

We need to go deeper into Get Changes. Your webhook fired, now you need to figure out what changed.

Follow Part 2 here

http://johnliu.net/blog/2019/5/one-flow-to-handle-them-all-part-2-figuring-out-the-changes

@ISSPDEV couldn’t wait for the next part he went ahead with this. And because he wrote in so much detail - I will also link to his work for Part 2.

Apologies

This was a live hack in 06/2018 - I actually have three draft versions of this blog post never finished so never previously published. So moving that forward with this first post. 9 months late is still better than never.