One Flow to handle them all - how to subscribe to multiple SharePoint lists with one Flow

One Flow to handle them all - how to subscribe to multiple SharePoint lists with one Flow

At some point - we think hmm how do we save a Flow as a template and deploy it with all our SharePoint site collections…

and did we just create a massive problem down the road where we have all these duplicate Flows…

and how do we manage versioning and changes to them hmm lets have a think and talk about this.

So instead of all that copying. Let’s try a different idea - what if we have One Flow that Rules them all. It handles events from All the Sites and Libraries, all within one Flow.

Depending on what our Flow actually do - this might be a better approach.


Live Tweet

This is a long overdue write up of an idea of two Flow done as a hack over twitter.  https://twitter.com/johnnliu/status/1002747531506245632

Multi-Part Series: Deploy My Flow

This is a multi-part series called Deploy My Flow. I want to publish discussing the various ways to deploy Flows across SharePoint and Office 365, between dev, test and production environments. Across lists, sites, and tenants.

  • Export and Import

  • Automate - PowerShell, Flow Management and Flow Studio

  • Power Platform Solutions

  • Subscribe to multiple SharePoint Lists with One Flow

Part I - SharePoint Webhooks

A Flow can only listen/trigger from one list, so while in SharePoint as we provision sites - we consider having a way to provision templated Flows to handle each list. 

But let's turn the question around.  Can we have just one Flow to handle multiple lists?
Is it possible to have Flow connect directly to SharePoint's Webhooks and manage events directly with Flow?

 

Why this approach?

  • The Flow triggers listen to and raises events for each individual list or document library. So, out of the box, we can't create one Flow that listens to every list or library.

  • Cloning Flows multiple times has its own challenges:

    • 250 Flow limit (This limit is now 500, you can request a service ticket to raise this number) per account.

    • How do we manage multiple Flows, do we have one master? Does it get exported and imported over existing Flows when we upgrade? What happens with the connections?

    • Can we do these automatically?

  • There’s a lot to discuss, but for this blog post we will focus on a simple pattern

  1. Create a Handler Flow that handles list change events.

  2. Create a second Scheduler Flow that subscribes webhooks from multiple SharePoint lists onto the first Flow.

Reference

We don't start from scratch, we have the excellent materials on docs.microsoft.com from the SharePoint PnP team describing how to attach to SharePoint webhooks via code.

https://docs.microsoft.com/en-us/sharepoint/dev/apis/webhooks/overview-sharepoint-webhooks

We only need to translate this to Flow.

First Create the Scheduler Flow

First - here I’m reading all the lists from SharePoint that’s a document library. Then read the subscriptions on each document library. This end point shows me SharePoint’s webhooks.

TIP: Because I use odata=nometadata a LOT - I end up putting that header into a header_nometa JSON variable and insert it in every Send HTTP Request action.


Create Subscription and handle Resubscription

Next, we check if the webhooks contains has a clientState with the current Flow’s name (Flow names are unique guids).

If not, we create one. The check and filter for ClientState makes sure we don’t subscribe the same event handler to the same document library multiple times. It also makes this “scheduler” flow re-run safe. We can pretty much change the trigger to a recurrence trigger that runs as often as you’d like - say daily.

workflow()?['name']

The Handler

We need a new Flow to handle the document updating. This is the One Flow that does all the work.

Save the Handler Flow and take the HTTP Reques URL back to our Subscriber Flow

And this is the final result - when we run the Subscriber Flow - it iterates through all my document libraries and connects them all to the handler Flow.

Now when documents are updated in these 12 document libraries, my “Handler Flow” will be triggered by SharePoint.

SharePoint webhook subscriptions are valid for 180 days. So at some point our Scheduler Flow will re-run on recurrence and re-subscribe the document libraries. (We can also handle the True condition in the Scheduler Flow to renew webhooks).

Part 2 - Get Changes

We need to go deeper into Get Changes. Your webhook fired, now you need to figure out what changed.

Follow Part 2 here

http://johnliu.net/blog/2019/5/one-flow-to-handle-them-all-part-2-figuring-out-the-changes

@ISSPDEV couldn’t wait for the next part he went ahead with this. And because he wrote in so much detail - I will also link to his work for Part 2.

Apologies

This was a live hack in 06/2018 - I actually have three draft versions of this blog post never finished so never previously published. So moving that forward with this first post. 9 months late is still better than never.

I hope team Office give Delve a second chance

Photo by Jay Wennington on Unsplash

Office Delve was the first product to use the Office Graph – an attempt to surface the nodes and edges of documents and people to build some intelligence.

Office Graph became Microsoft Graph and found lots of developer love.

Delve had to choose how to continue to exist – I suspect the team looked at the usage of how people that actually used Delve and studied how they used Delve.

Disclaimer

Please note - I have no particular inside knowledge of what Team Office is working on regarding Delve. I just wanted to write this, in 2019, because a good friend asked me - hey do you use Delve, do you like it? I am also NOT implying team Office is cutting Delve - as far as I know, they are maintaining it as it currently is. In fact, if I had known what they are working on - I wouldn’t have been able to write this post.

I do know from SharePoint uservoice - that there seems to be some work happening on taxonomy and metadata. So I figure I should write this blog post now.

So how does Delve work?

So a few years ago, I suspect they settled on what they thought was the killer idea – Delve shows you “people around you” and “documents around you” 

They found that if they could show YOU the documents that PEOPLE AROUND YOU are working on – that must be documents you want to see.

Today, that’s pretty much the only scenario that Delve works really well for - if I am looking for the document that I know my colleague was working on last Friday but I don’t know where it was saved.  Delve will tell me.

There was a second feature of delve - Boards and favourites. But as the product evolved around how it is being used - the boards idea takes a dramatic backseat.

How I think Delve should have worked

it should have been the cross Office 365 #tags management system, powered by Search and Graph. Providing a one-stop definition for folksonomy and taxonomy as both a destination, but also the interface where a process librarian owns and curates their tag collection.

 

There was a secondary idea – Delve gave us “Boards” where people can add documents to a board – think Pinterest, but for documents, and people, and office objects. 

So now imagine #usergroup tag shows you emails, documents and people involved with running the monthly company usergroup.

That’s damn useful. Surface it in SPFx search webpart. Use it to show a most recently added documents list or appear in a mega menu. 

What I think what team Office should have built, is to improve and combine

 

  • Improve Delve Boards and combine

  • SP Document Metadata taxonomy tags

  • SP Folksonomy tags

  • Yammer hashtags

  • Teams hashtags

  • Exchange/Outlook hashtags and categories

  • Tags across Todo, Planner

  • Tags across Stream

 

This way – we can surface a collection of documents explicitly labelled with tags – across the entire company. 

Say, if you tag #induction and this article will automatically appear in an #induction board in Delve.  That would have been more magical. 

Or if an OHS #worksafety email was sent and tagged - it would appear in the Delve Board. 

Appoint specific users to own the boards – so they are the tag librarian making sure unsuitable links is hidden from that tag. They own the boards - so they are the curators. They can decide if objects are automatically included, how the objects are sorted, and which objects should be pinned to the top.

I think this isn’t a simple goal - this is the longer path – requiring Team Office to work on some sort of inter-office tag/hashtag/metadata system.   I don’t believe it is possible to implement this across every product and force everyone to use the same list - but I do believe it is possible to aggregate them at the Search results stage and control collections via Delve Boards.

I think it would give Delve a strong reason and a decent second chance at being a unique Office 365 go to destination.  And tie up a lot of loose ends at the same time.

It’s 2019, my hope with Delve Next

is that we will see Me, then Tags (then People) - and the tags are curated boards of content collected across all of Office 365 via every tag and hashtag system we have.

Two free tickets to great Office 365 and SharePoint events in Sydney in the next month

Photo by Fancycrave on Unsplash

Photo by Fancycrave on Unsplash

I wanted to write about two free upcoming events regarding Office 365 and SharePoint happening very soon in Sydney Australia, both events are free, but you’ll need to register.



First: Office 365 Saturday Sydney is this Saturday October 13!

on October 13 - that is this upcoming Saturday! We will be gathering at the Microsoft Sydney Reactor which is located above Wynyard station in the city. This would be… the 8th Saturday event we have ran since the earliest SharePoint Saturday.

This is happening this Saturday! Register here.
https://www.meetup.com/en-AU/O365-Saturday/events/255042254/

O365 Saturday Sydney 2018

Sat., 13 Oct. 2018, 9:00 am: Welcome to the 2018 edition of SharePoint & Office 365 Saturday Sydney! This is a free event where we learn and celebrate Office 365. We have a dozen local, national and international speakers...


Second: Office Developer Bootcamp is Friday November 2

This was an oversubscribed event in 2017 - join Microsoft evangelists and Office 365 Development MVPs for a day of catching up to the latest state of Office APIs and wizardry. We are on the ground covering your questions from Azure Functions, Flow, Microsoft Graph, SPFx, new SharePoint and Teams APIs to obscure API webhooks and cheapest ways to make Office 365 work for you and light up like a Christmas tree.

This is a full day Friday event held at Microsoft North Ryde, so you need to talk to your manager to get the day off and bring your laptop for a day of hacking.


https://www.eventbrite.com.au/e/global-office-365-developer-bootcamp-sydney-2018-tickets-48062636640

Global Office 365 Developer Bootcamp - Sydney 2018

The Global Office 365 Developer Bootcamp is a free, one-day training event led by Microsoft MVPs with support from Microsoft and local community leaders. The bootcamps will provide hands-on labs for deep learning, and a comprehensive view of all key technologies and products on the Office 365 platform.

Whether we see you on the Saturday or the Friday, come visit and drop in and say hello. This is absolutely the best time to meet most of the Office 365 experts in Sydney and wrap up 2018.

I want to end this announcement with a lit Christmas tree. Hope to see you very soon.

Photo by Brigitte Tohm on Unsplash

Office 365 Saturday Canberra 2018 #O365CBR #SPSCBR

This Saturday - 14 July, we'll be converging on SharePoint and Office 365 Saturday Canberra.  I'm planning to take my time and drive down for the weekend.

028Canberra0021.jpg

https://www.meetup.com/en-AU/O365-Saturday/events/252110689/ 

The event is held at Microsoft office this time (so not at the previous Clifton's venue).  We will be covering a mix of Office 365, modern SharePoint, and security topics.

I'll be around all day as well - ask me anything about SharePoint, Flow, PowerApps, Office Development, Todo, Xbox and everything!

I believe we have an Xbox One and a bunch of swag to give away too!  But you are definitely coming for the wide range of content and topics, and definitely not for the swag.

We hope to see you there!

 

Run Any PnP-PowerShell via Drag and Drop Zip to AzureFunctions

pexels-birthday-box-celebration-45238.jpg

So I previously asked aloud "Can we simplify this further" - It turns out you can.  You can deploy a PnP-PowerShell from scratch in 1 minute.

And since we are targetting completely non-devs.  We do this via dragging and dropping one Zip file.

 

Plan

In my mind - yes.  Yes that installation could get simpler.  App Services (because of the Kudu backend) supports Zip Deploy.

https://docs.microsoft.com/en-us/azure/azure-functions/deployment-zip-push

But a quick read of the article it targets DevOps doing zipdeploy with Azure CLI, REST or PowerShell.  There's nothing for a yet-to-be-developer.  What I wanted to do was drag and drop zip deploy (so I quickly reached out to Ling Toh and turns out she says she does it all the time).

Steps in Pictures

1. Grab this ZIP file 

https://github.com/johnnliu/azure-functions-o365/blob/master/Run-Any-PnP-PowerShell.zip

2. Create a new Functions App

3. Drag and Drop zip file into Kudu

4. Configure App Settings

Back to Functions page under Azure Portal.  Refresh and you'll see the function appear.
Go to app settings to set up the USER/PW variables we need for the test.

5. Test it

Summary

That's it really.  Drag and drop deploy of a function via ZIP file.
Feel free to open up the zip and see how all the pieces are laid out, including the entire PnP-PowerShell module dependency.