I'm speaking about Serverless Flow and Azure Functions at Collab365 Free Online Conference

Have you heard about the virtual Collab365 Global Conference 2017 that’s streaming online November 1st – 2nd?

Join me and 120 other speakers from around the world who will be bringing you the very latest content around SharePoint, Office 365, Flow, PowerApps, Azure, OneDrive for Business and of course the increasingly popular Microsoft Teams. The event is produced by the Collab365 Community and is entirely free to attend.

Places are limited to 5000 so be quick and register now.

During the conference I'd love you to watch my session which is called : 

'Serverless with Microsoft Flow and Azure Functions'

Level up your mastery of Microsoft Flow. Switch to Azure Functions only when you need to. No doubt there will be many sessions on Microsoft Flow, introducing you to its wonderful merits and rough edges. This session is for the advanced users - we will see what Microsoft Flow really is, and bend it to our will.

If you join me, you will learn:

• Master JSON in a Flow
• Combining Azure Functions with Flow
• Failure Recovery, in a Flow
• How to handle Binary in a Flow
• How to write HTML and generate PDF in Flow

Topic(s):

• Azure Functions
• Microsoft Flow

Audience :

• Developer
• Power User

Time :

• Thursday, November 2 2017 10:00 AM (UTC)
• Thursday, November 2 2017 09:00 PM (ADST)

How to attend :

1. Register here.
2. At the time listed above go here to watch my session. (you can also add me to your own personal planner from the agenda.
3. Enjoy the demos and ask me questions, I'll put the templates up for download after the session.

From Office 365 to Azure to Minecraft, connected with Flow

"John, what's this headline."
This is just a love serenade to Microsoft's many engineers and teams.  Thank you, for making these products that makes these things look easy.

Thank you for showing the world what One Microsoft might look like when everything works together.

Plan

1. Minecraft Windows 10 edition in the latest 1.2 (better together update) included web sockets previously only in the Minecraft Education version.  Web Sockets lets you connect to a Minecraft game, and remotely execute commands.
2. Minecraft Code Connection is an external application that hosts a friendly REST API and translates JSON to web socket.  Previously this was only for Minecraft Education Edition.  An update in early October allows this to work with Minecraft Windows 10.
   https://makecode.com/blog/minecraft/10-18-2017
3. Microsoft Data Gateway allows Flow, PowerApps, PowerBI to talk to on-premises environments.  It also recently gained the ability to execute custom connections.  It is essentially, an enterprise data gateway / reverse proxy that connects your local environment to an Azure Service Bus.  The ability to call your custom local REST endpoint was released in September.
   https://flow.microsoft.com/en-us/blog/q3-2017-update/
4. Cloud based services like Flow/LogicApps, PowerApps, and PowerBI can talk to this connection online, via the magic of Azure Service Bus.
5. And because I'm a SharePoint MVP - we are triggering this Flow from SharePoint Online.  Because the world needs this.

Setting up Minecraft & Code Connection

https://www.microsoft.com/en-au/store/p/minecraft-for-windows-10/9nblggh2jhxj?rtc=1 
Buy or download Minecraft for Windows 10

https://education.minecraft.net/get-started/download/
Download Minecraft Code Connection

https://education.minecraft.net/support/knowledge-base/code-connection-api-documentation/
The API for the JSON messages to send to Code Connection is here.
 

View fullsize

Code Connection uses Minecraft commands - the option is called "activate cheats" - do this in a creative world.

Code Connection is a separate executable.  Run it outside of Minecraft - it'll ask you to enter the command into Minecraft

Use Postman to test your localhost:8080 and send a REST request.  You'll see your agent bot move.

Congrats - you now have a REST endpoint that can send game commands to your Minecraft game.

Setting up Flow Data Gateway

https://flow.microsoft.com/en-us/documentation/gateway-reference/ 

The data gateway also calls your local REST endpoints via a custom connector.  This was a feature that was silently released in the deluge of news from MSIgnite.  I need to thank @pratapladhani (PowerApps PM) for sending me the link of the announce.  It is REALLY obscure.

https://flow.microsoft.com/en-us/blog/q3-2017-update/

“On-premises connectivity to any HTTP API - Finally, users can now connect their own on-premises APIs with Custom Connectors - leveraging the On-Premises Data Gateway. For example, if you have a service that’s only available on your local network, you can now author a custom connector that can read data from or push data to that local service.”

Congrats - this is your reverse proxy.

Setting up custom connection via this Swagger File I've prepared for you

https://github.com/johnnliu/flow/blob/master/mc-cc.swagger.json 

Grab this swagger.json and add a custom connection to Flow

Note the host is localhost:8080
Tick: Connect via on-premise data gateway

Note the custom connection swagger file points to localhost:8080 - this will work if your data gateway is on the same machine as the Code Connection exe.  Otherwise, you should use the LAN IP Address of the Code Connection server.

Add a  connection, and select On-Premises data gateway

Note the connection type is "On-premises"

Write my Flow

These wonderful methods with dropdowns are defined in Swagger file.  It's hard work.  Contributions welcome!

Trigger it from Office 365 / SharePoint

Hi bot agent

Uploaded by John Liu on 2017-10-25.

Code-Less

I want you to understand, all these are done, without me writing a single line of C# or JavaScript.

Cost?

Office 365 E3+ license allows you to have:

• 2000 Flow runs per user
• 1 Custom Connection
• On-Premises data gateway allowed

https://australia.flow.microsoft.com/en-us/pricing/ 

So there's no extra cost for an Office 365 customer E3+

Summary Key Points

• Minecraft Win10 has websockets
• Code Connection does JSON to websocket
• Data Gateway is amazing, also calls your local REST endpoints
• Microsoft Flow just unlocks it to the rest of the world, if you like, LogicApps will work just as well.
• You can have a PowerApp calling the function via the Custom Connection, without Flow.
• Extremely code-less

Where do we go next?

Well, when I detect an Active Directory User deletion via webhook - I'm going to teleport this Zombie Villager into lava.

Update: I can't see a use case for this

So, some people have commented to me directly - John this is very cute, but there's no business use case for this.

Let me explain the head-fake

Flow can reach into your organization and call any of your REST endpoints with no code

Microsoft Flow blog posted https://flow.microsoft.com/en-us/blog/on-premise-apis/ explaining how you can build your own on-premises API service and call it from Flow via the data gateway.  I was bothering them with explanations for the data gateway and I think they were ready to publish anyway, they told me they'll share it in a few days.

I beat them slightly to publish since I worked out how to do the Minecraft bit.

I also think they need more imagination.  Minecraft is a hundred times better example and I didn't even have to write a REST service ;-)

<3

Auto-Classify Images in SharePoint Online library via Flow for Free

Microsoft Flow's most recent update added ability to query and update SharePoint File property.  This is actually really timely, as I wanted to combine this with a few other techniques and built a Document Library Image Auto-Classifier Flow.

Is that a clickbait headline?  Well it's totally real, and we'll build it in a moment.

Steps:

1. Set up your cognitive service account (understand the free bucket)
2. Set up a SharePoint Online document library with Categories
3. Set up the Flow file loop
4. Do a fancy JSON array to concatenated string projection operation with Select and Join
5. Viola, no code.  And pretty much *free*

This is part of a series on Microsoft Flow

Set up your Azure Cognitive Service instance

Follow these simple steps to create an Computer Vision API Cognitive Service in your Azure.  Computer Vision API has a free tier.

1. Create Computer Vision API

View fullsize

2. Scroll down and hit Create

View fullsize

3. Give this service a name, set up the region and select Free pricing tier

4. You need the endpoint url here

5. Also, copy the Name and key 1

You will need the "Name" and a "Key" for the next step.

The free tier of Computer Vision API - first 5000 transactions free per month.

Note the service isn't available in all regions.  Most of my stuff is in Australia East, but for the Cognitive Service API it has to be hosted in Southeast Asia.  YMMV.

View fullsize

Then we need to set up the connection in Flow

1. Find the Computer Vision API action

View fullsize

2. Enter service name, key and the root site url to set up the initial connection

View fullsize

3. Created correctly, you get an action like this

Set up the SharePoint Document Library

My SharePoint document library is very simple - it is just a basic document library, but I added an extra site column "Categories". This is an out of the box field, and is just a simple text field.

This is a simple step

Set up the Flow

I trigger the flow with a Scheduled Recurrence that runs once per day.
Using the new Get Files (properties only), I grab a list of all the files in a document library.
I then run for-each on the list of files.

View fullsize

Inside the for-each, I have a condition that checks if the Categories field is null.  If you type null directly into the field, you will get the string 'null'. 

Tip: To actually get the formulat/expression null, select Expressions and type null there.

View fullsize

If the Categories is null, then we proceed.

Grab the file content via Get file content
Call Computer Vision API with the image content.  Select the Image Source to binary, instead of URL.

Tip: I use a compose to see the debug results

I'll explain the array projection in the next section.

View fullsize

Select projection: JSON array to String array

We have an array of JSON objects:

[{
     'name': 'foo'
},
{
    'name': 'bar'
}]

This default UI maps to:

tags -> [{ specified properties }…]

The result is that we would end up with a new array of (simpler) JSON objects.
Hit advanced text mode.

Here, we can use Expression to say item('Tag_Image')?.name

In this case the UI is smart enough to show Tag.Name as a dynamic content (as well as the Tag.ConfidenceScore property).  So we can select that.

This performs a projection of

tags -> [ names… ]

We now have an array of strings.  Combine them via Join with a comma (,) separator.
Update the file properties with this string.

Lets see the results

I uploaded a few images to the library.
Note the categories field is blank.

View fullsize

Running the Flow

View fullsize

When it finishes, I'm checking the JSON - the picture is identified with a "person" with 99% confidence.
The combined string "person,young,posing" is updated into the File property.

View fullsize

The documents are updated.  When Flow runs tomorrow it will skip them.

View fullsize

The Final Flow

View fullsize

Speaking at Digital Workplace Conference Australia 2017

I'll be speaking at the Digital Workplace Conference Australia!  23-24 August in Sydney.

This is a conference that's near and dear to me - and I've had several opportunities in the past to present at this conference, where I covered Silverlight, JavaScript, TypeScript, Modern Office App-ins and now this year - I plan to present a supercharged talk on running Serverless with Office 365.

Parts of the talk - especially how to get started - may seem familiar to many of you that has started down this journey. 

I wanted to focus a bit less on the technical, and more about how this has changed people. 

Azure Functions democratized 'I need to run a bit of code' to everyone.  Suddenly, the cloud is not this scary place where there are a hundred things we don't know, and don't know where to start.  Suddenly, the toys that seems far out of reach are ours.  Suddenly, a cloud subscription that costs less than a coffee per month is something I don't even think about.

To me, that is the power of AzureFunctions and why Serverless is a game changer. 

Do you know there are now brand new categories of design patterns specifically rewritten for the Serverless world.

I will of course still cover the technical bits - but to see all 20+ demos I have with me, you'll have to come find me in the speaker area for a personal demo :-)

In Digital Workplace Conference 2017, I want to talk about Serverless.

And I want to talk about humans.  Us.

I think the future will be amazing.  I hope to see you at the DWC Australia.  Come and grab me and say hello!

Are you Cloud-Curious or Cloud-Serious? Azure Functions in DWCNZ 2017

I had a fantastic time at Digital Workplace Conference in NZ.

Highlight Sessions

• Keynote by Sarah Bowden on Digital Transformation
• And Christian Buckley's 3 stories and 4 lessons
• The Foodstuffs Journey 
• Steve Knutson's PowerApps and Flow  
• Jeff Willinger's intensive Power BI talk

There are many other great sessions, I wasn't able to be in multiple places at once!

My Own Session

I presented Azure Functions in Office 365 - Building Serverless Solutions

There were a few things that I didn't managed to get through.  I wanted to list them here, and hope you will accept my apologies.  I've had several conversations with you all over the two days of the conference, many wanted deeper details into certain aspects of using Azure Functions.

 

Demo: Timer Based Alert with Email

https://github.com/johnnliu/azure-functions-o365/blob/master/sharepoint-list-email.ps1

This demo outlines a very simple script that will connect to a SharePoint list (or document library), query and fetch list items, format them into HTML and email to user from the System Account.

Combined with a schedule, this is an extremely common scenario in SharePoint Online: you want to schedule a smart alert email once a week, based on a filter to a list.

 

Using Recurring event in Flow instead of Azure Functions Timer-Trigger

While you can schedule tasks in Azure Function via a Timer Trigger, Microsoft Flow's recurrent trigger has several benefits:

• You can create a Team Flow - so multiple users can be owners and configure the recurrence trigger.
• The UI for setting up a time for the trigger is more obvious for power users.
• You can easily see past runs from within Flow
• You can easily re-run a Flow

 

The Severless "Specturm"

From my own experiences and from reading and understanding the greater scope of Serverless solutions that are being designed in the world, I wanted to present the spectrum of Serverless solutions.  We start on one side - from the Cloud-Curious, to the experts - the Cloud-Serious.

Cloud Curious

The majority of the presentation is pitched for the cloud-curious.  You have heard of Azure Functions and Serverless.  The demos presented how to get going really quickly.

Functions are thus:

1. Micro (web) services for everyone.  So many people I talked to has given up on programming, thinking writing microservices or complex architecture isn't for them.  It's for the young'in dev teams now.
   AzureFunctions, especially with PowerShell - flipped the whole thing upside down.  Now, many 'ex-developers' suddenly find themselves build amazing service end points, connecting them to webhooks and Azure Blob Queues.  It is an amazing resurgence and move to microservices.  And everyone's having fun playing with really cool new toys.
    
2. Use your favourite language!
   C#?  JS?  PoSH?  F#?  You can even use TS or VB.NET compiled.  Nobody can tell you what language you can and can't use.
    
3. Perfect solution for many problems in SharePoint customizations
   Elevate permissions
   Webhook and event receivers
   Timer Jobs
   Extending Flow (as custom workflow action)
   
   If you are bringing customizations in SharePoint On-Premises to SharePoint Online - Azure Functions is a solution that must be evaluated.  It fits so many scenarios that you need to bring your On-Premises customizations forward, without breaking the bank, or needing complex re-development.

Cloud Serious

For the cloud serious - you are already using simple functions.  You want to know what's next.

1. "Idempotent" - this is the keyword that will define the entire Serverless Framework.  You want to design functions that has no side-effects if you rerun.  A function can fail, it will automatically retry until success.  Your function must be built to be retry-safe.
    
2. Use message queues and service bus to scale your Function.
   In Serverless, you are bound by duration.  You are not bound by parallel compute.
   To scale your long running process, split into a Queue and spawn infinite parallel compute.
    
3. In a serial code, we wanted to catch all our exceptions to speed up long running tasks.  When we convert to parallel compute - we no longer really care about exceptions.  If you fail, you want to fail fast.  Throw exceptions freely and as fast as possible.  Terminate the function.  Let Queue retry automatically.
    
4. With the new Azure Functions Proxies, we can create Serverless Web Applications - which is essentially combining a CDN to host static resources, and Functions to run server side code.
   
   Future of Serverless web apps is basically: CDN + Functions
   Both scale in parallel infinitely, by default, by design.  But is easy to understand and accept in concept.
   
   You do no worry about scaling VMs, AppPools, IIS, WebJobs, WebSites... 

Your solutions sits on top of all of those things - but there is no fear.  A fast messaging queue with built-in retries and a thousand atomic hammers will carry your workload from now to infinity.  And it'll cost less than your coffee.

Slide downloads

https://github.com/johnnliu/pptx