From Office 365 to Azure to Minecraft, connected with Flow

/

"John, what's this headline."
This is just a love serenade to Microsoft's many engineers and teams.  Thank you, for making these products that makes these things look easy.

Thank you for showing the world what One Microsoft might look like when everything works together.

Plan

  1. Minecraft Windows 10 edition in the latest 1.2 (better together update) included web sockets previously only in the Minecraft Education version.  Web Sockets lets you connect to a Minecraft game, and remotely execute commands.
  2. Minecraft Code Connection is an external application that hosts a friendly REST API and translates JSON to web socket.  Previously this was only for Minecraft Education Edition.  An update in early October allows this to work with Minecraft Windows 10.
    https://makecode.com/blog/minecraft/10-18-2017
  3. Microsoft Data Gateway allows Flow, PowerApps, PowerBI to talk to on-premises environments.  It also recently gained the ability to execute custom connections.  It is essentially, an enterprise data gateway / reverse proxy that connects your local environment to an Azure Service Bus.  The ability to call your custom local REST endpoint was released in September.
    https://flow.microsoft.com/en-us/blog/q3-2017-update/
  4. Cloud based services like Flow/LogicApps, PowerApps, and PowerBI can talk to this connection online, via the magic of Azure Service Bus.
  5. And because I'm a SharePoint MVP - we are triggering this Flow from SharePoint Online.  Because the world needs this.

Setting up Minecraft & Code Connection

https://www.microsoft.com/en-au/store/p/minecraft-for-windows-10/9nblggh2jhxj?rtc=1 
Buy or download Minecraft for Windows 10

https://education.minecraft.net/get-started/download/
Download Minecraft Code Connection

https://education.minecraft.net/support/knowledge-base/code-connection-api-documentation/
The API for the JSON messages to send to Code Connection is here.
 

Code Connection uses Minecraft commands - the option is called "activate cheats" - do this in a creative world.

cc-1.png

Code Connection is a separate executable.  Run it outside of Minecraft - it'll ask you to enter the command into Minecraft

mc-2.png
cc-2.png
cc-post-1.png

Use Postman to test your localhost:8080 and send a REST request.  You'll see your agent bot move.

Congrats - you now have a REST endpoint that can send game commands to your Minecraft game.

 

Setting up Flow Data Gateway

https://flow.microsoft.com/en-us/documentation/gateway-reference/ 

The data gateway also calls your local REST endpoints via a custom connector.  This was a feature that was silently released in the deluge of news from MSIgnite.  I need to thank @pratapladhani (PowerApps PM) for sending me the link of the announce.  It is REALLY obscure.

https://flow.microsoft.com/en-us/blog/q3-2017-update/

“On-premises connectivity to any HTTP API - Finally, users can now connect their own on-premises APIs with Custom Connectors - leveraging the On-Premises Data Gateway. For example, if you have a service that’s only available on your local network, you can now author a custom connector that can read data from or push data to that local service.”

 

 

data-gateway.png
gateway-2.png

Congrats - this is your reverse proxy.

 

Setting up custom connection via this Swagger File I've prepared for you

https://github.com/johnnliu/flow/blob/master/mc-cc.swagger.json 

Grab this swagger.json and add a custom connection to Flow

custom-connection-0.png

Note the host is localhost:8080
Tick: Connect via on-premise data gateway

custom-connection.png

Note the custom connection swagger file points to localhost:8080 - this will work if your data gateway is on the same machine as the Code Connection exe.  Otherwise, you should use the LAN IP Address of the Code Connection server.

Add a  connection, and select On-Premises data gateway

data-gateway-2.png
connection.png

Note the connection type is "On-premises"

 

Write my Flow

flow.png

These wonderful methods with dropdowns are defined in Swagger file.  It's hard work.  Contributions welcome!

Trigger it from Office 365 / SharePoint

flow-run-1.png
flow-run-2.png

 

Hi bot agent

Uploaded by John Liu on 2017-10-25.

 

Code-Less

I want you to understand, all these are done, without me writing a single line of C# or JavaScript.

Cost?

Office 365 E3+ license allows you to have:

  • 2000 Flow runs per user
  • 1 Custom Connection
  • On-Premises data gateway allowed

https://australia.flow.microsoft.com/en-us/pricing/ 

So there's no extra cost for an Office 365 customer E3+

Summary Key Points

  • Minecraft Win10 has websockets
  • Code Connection does JSON to websocket
  • Data Gateway is amazing, also calls your local REST endpoints
  • Microsoft Flow just unlocks it to the rest of the world, if you like, LogicApps will work just as well.
  • You can have a PowerApp calling the function via the Custom Connection, without Flow.
  • Extremely code-less

Where do we go next?

Well, when I detect an Active Directory User deletion via webhook - I'm going to teleport this Zombie Villager into lava.

 

Update: I can't see a use case for this

So, some people have commented to me directly - John this is very cute, but there's no business use case for this.

Let me explain the head-fake

Flow can reach into your organization and call any of your REST endpoints with no code

Microsoft Flow blog posted https://flow.microsoft.com/en-us/blog/on-premise-apis/ explaining how you can build your own on-premises API service and call it from Flow via the data gateway.  I was bothering them with explanations for the data gateway and I think they were ready to publish anyway, they told me they'll share it in a few days.

I beat them slightly to publish since I worked out how to do the Minecraft bit.

I also think they need more imagination.  Minecraft is a hundred times better example and I didn't even have to write a REST service ;-)

<3

Flow Admin Center - Do a spring cleaning of your Flows

/

I spewed coffee all over my screen.

flow-admin-quota.png

Each user in Office 365 gets 2000 Flow runs pooled.  This is my personal playground tenant, I've got 10 users licenses but it's really just me running around sending Flows to myself, and I.

So, I'm looking around Flow Admin center.  I see a tab Quotas.  I clicked on it and sees oh it's got a bar telling me how many runs left.  That's good.

Then I see the number: 6500+ runs.

I chocked.  What.  I run a number of Flows but really NOT that many.  Is that lifetime total?  It's per month.  OK.  We need to interrupt everything and sort this out.

So hmm, it does look peculiar.  (BTW, I don't know how 8000+4000 = 6500)

But I do want to go into these two examples:

First Bad Thing: Scheduled Recurrence

flow-recurrence-trigger.png

Sometimes, I use a Recurrence trigger to test something - and it default to 1 per minute.  I had changed this to 5, but this was way too often.  

TIP: Always run test Flows via a HTTP Request Trigger - poke it with Postman instead.

 

Second Bad Thing: Infinity Loops

This one is can be in different forms.  But the idea is simple - you trigger from a SharePoint list item, and you update that same item.

The more tricker version to spot is if the update was hidden within a Conditional Block, so it only "sometimes" trigger, but not all the time.  The problem is, that sometimes started happening, and now it's looping endlessly without telling you about it.

You get the idea.  This second one was worse than the first problem - this one clocked in 8000 runs on its own.

Only you can save Yourself (and your Free runs)

I'm sure Flow product team will be thrilled if you don't check your Flows.

But you might not be.  So you need to proactively do the next step.

We create a new recurring monthly Flow that creates a TODO task to review the Flow Admin Quotas.

This logic is perfect*
If you see the error, let me know in the comments ;-)


I've been working on an epic Flow story and that's gonna take some time.  But this short episode was too good not to write up.  It is now Friday 7 PM so I'm signing off have a great weekend everyone.

Serverless connect-the-dots: MP3 to WAV via ffmpeg.exe in AzureFunctions, for PowerApps and Flow

/

There's no good title to this post, there's just too many pieces we are connecting.  

So, a problem was in my todo list for a while - I'll try to describe the problem quickly, and get into the solution.

  • PowerApps Microphone control records MP3 files
  • Cognitive Speech to Text wants to turn WAV files into JSON
  • Even with Flow, we can't convert the audio file formats. 
  • We need an Azure Function to gluethis one step
  • After a bit of research, it looks like FFMPEG, a popular free utility can be used to do the conversion

Azure Functions and FFMPEG

So my initial thought is that well, I'll just run this utility exe file through PowerShell.  But then I remembered that PowerShell don't handle binary input and output that well.  A quick search nets me several implementations in C# one of them catches my eye: 

Jordan Knight is one of our Australian DX Microsoftie - so of course I start with his code

It actually was really quick to get going, but because Jordan's code is triggered from blob storage - the Azure Functions binding for blob storage has waiting time that I want to shrink, so I rewrite the input and output bindings to turn the whole conversion function into an input/output HTTP request.

https://github.com/johnnliu/function-ffmpeg-mp3-to-wav/blob/master/run.csx

#r "Microsoft.WindowsAzure.Storage"

using Microsoft.WindowsAzure.Storage.Blob;
using System.Diagnostics;
using System.IO;
using System.Net;
using System.Net.Http.Headers;

public static HttpResponseMessage Run(Stream req, TraceWriter log)
{

    var temp = Path.GetTempFileName() + ".mp3";
    var tempOut = Path.GetTempFileName() + ".wav";
    var tempPath = Path.Combine(Path.GetTempPath(), Guid.NewGuid().ToString());

    Directory.CreateDirectory(tempPath);

    using (var ms = new MemoryStream())
    {
        req.CopyTo(ms);
        File.WriteAllBytes(temp, ms.ToArray());
    }

    var bs = File.ReadAllBytes(temp);
    log.Info($"Renc Length: {bs.Length}");


    var psi = new ProcessStartInfo();
    psi.FileName = @"D:\home\site\wwwroot\mp3-to-wave\ffmpeg.exe";
    psi.Arguments = $"-i \"{temp}\" \"{tempOut}\"";
    psi.RedirectStandardOutput = true;
    psi.RedirectStandardError = true;
    psi.UseShellExecute = false;
    
    log.Info($"Args: {psi.Arguments}");
    var process = Process.Start(psi);
    process.WaitForExit((int)TimeSpan.FromSeconds(60).TotalMilliseconds);


    var bytes = File.ReadAllBytes(tempOut);
    log.Info($"Renc Length: {bytes.Length}");


    var response = new HttpResponseMessage(HttpStatusCode.OK);
    response.Content = new StreamContent(new MemoryStream(bytes));
    response.Content.Headers.ContentType = new MediaTypeHeaderValue("audio/wav");

    File.Delete(tempOut);
    File.Delete(temp);
    Directory.Delete(tempPath, true);    

    return response;
}
Trick: You can upload ffmpeg.exe and run them inside an Azure Function

https://github.com/johnnliu/function-ffmpeg-mp3-to-wav/blob/master/function.json

{
  "bindings": [
    {
      "type": "httpTrigger",
      "name": "req",
      "authLevel": "function",
      "methods": [
        "post"
      ],
      "direction": "in"
    },
    {
      "type": "http",
      "name": "$return",
      "direction": "out"
    }
  ],
  "disabled": false
}

Azure Functions Custom Binding

Ling Toh (of Azure Functions) reached out and tells me I can try the new Azure Functions custom bindings for Cognitive Services directly.  But I wanted to try this with Flow.  I need to come back to custom bindings in the future.

https://twitter.com/ling_toh/status/919891283400724482

Set up Cognitive Services - Speech

In Azure Portal, create Cognitive Services for Speech

Need to copy one of the Keys for later

Flow

Take the binary Multipart Body send to this Flow and send that to the Azure Function

base64ToBinary(triggerMultipartBody(0)?['$content'])

Take the binary returned from the Function and send that to Bing Speech API

Flow returns the result from Speech to text which I force into a JSON

json(body('HTTP_Cognitive_Speech'))

Try it:

Swagger

Need this for PowerApps to call Flow
I despise Swagger so much I don't even want to talk about it (the Swagger file takes 4 hours the most problematic part of the whole exercise)

{
  "swagger": "2.0",
  "info": {
    "description": "speech to text",
    "version": "1.0.0",
    "title": "speech-api"
  },
  "host": "prod-03.australiasoutheast.logic.azure.com",
  "basePath": "/workflows",
  "schemes": [
    "https"
  ],
  "paths": {
    "/xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx/triggers/manual/paths/invoke": {
      "post": {
        "summary": "speech to text",
        "description": "speech to text",
        "operationId": "Speech-to-text",
        "consumes": [
          "multipart/form-data"
        ],
        "parameters": [
          {
            "name": "api-version",
            "in": "query",
            "default": "2016-06-01",
            "required": true,
            "type": "string",
            "x-ms-visibility": "internal"
          },
          {
            "name": "sp",
            "in": "query",
            "default": "/triggers/manual/run",
            "required": true,
            "type": "string",
            "x-ms-visibility": "internal"
          },
          {
            "name": "sv",
            "in": "query",
            "default": "1.0",
            "required": true,
            "type": "string",
            "x-ms-visibility": "internal"
          },
          {
            "name": "sig",
            "in": "query",
            "default": "4h5rHrIm1VyQhwFYtbTDSM_EtcHLyWC2OMLqPkZ31tc",
            "required": true,
            "type": "string",
            "x-ms-visibility": "internal"
          },
          {
            "name": "file",
            "in": "formData",
            "description": "file to upload",
            "required": true,
            "type": "file"
          }
        ],
        "produces": [
          "application/json; charset=utf-8"
        ],
        "responses": {
          "200": {
            "description": "OK",
            "schema": {
              "description": "",
              "type": "object",
              "properties": {
                "RecognitionStatus": {
                  "type": "string"
                },
                "DisplayText": {
                  "type": "string"
                },
                "Offset": {
                  "type": "number"
                },
                "Duration": {
                  "type": "number"
                }
              },
              "required": [
                "RecognitionStatus",
                "DisplayText",
                "Offset",
                "Duration"
              ]            
            }
          }
        }
      }
    }
  }
}

Power Apps

Result

Summary

I expect a few outcomes from this blog post.

  1. ffmpeg.exe is very powerful and can convert multiple audio and video datatypes.  I'm pretty certain we will be using it a lot more for many purposes.
  2. Cognitive Speech API doesn't have a Flow action yet.  I'm sure we will see it soon.
  3. PowerApps or Flow may need a native way of converting audio file formats.  Until such an action is available, we will need to rely on ffmpeg within an Azure Function
  4. The problem of converting MP3 to WAV was raised by Paul Culmsee - the rest of the blog post is just to connect the dots and make sure it works.  I was also blocked on an error on the output of my original swagger file, which I fixed only after Paul sent me a working Swagger file he used for another service - thank you!

 

 

Generate Any PDF Documents from HTML with Flow

/

This is a crazy one, and if you have read ALL my Microsoft Flow blog posts, you'll be familiar with all these pieces.

We are going to connect them a different way though, and the result is still awesome.  Lets begin.

In this blog post:

  • Try to convert image files to PDF
  • Use a workaround to convert image files to PDF
  • Ultimate power: create Any HTML and convert to PDF. 
  • Effectively, we arrive at PDF-gen with templating.

Try to convert Image Files to PDF with Flow

  • Get File Content (Binary) from SharePoint. 
  • Write to OneDrive for Business
  • Use Convert to PDF to convert JPG to PDF.

This doesn't actually work.  But it's good to try and see the error.  Not Acceptable.

This isn't the end though, we have workarounds.

Use a workaround to convert image files to PDF

So even though Convert File doesn't work on image files directly, it does work on HTML.  And this is the heart of our workaround.

Remember several blog posts ago we used dataUriToBinary() to convert PowerApps camera image to a Binary file to store into SharePoint?

Today, we are going backwards.  We are taking a SharePoint image (as Binary), and converting that to dataUri format.  Lets put that in a variable dataUriJPG

We then create another string for <img src=variable(dataUriJPG) />.  You'll need concat() expression to combine the string and variable.

concat('<img src="', variable('dataUriJPG'), '" />')

See, browsers can render images with dataUri directly embedded.  Lets write that to a HTML file (pic.html)

  1. Get File Content from SharePoint
  2. Convert to dataUri string
  3. Concat within an HTML IMG tag
  4. Write to a HTML file
  5. Convert File from HTML to PDF

And hit it again with convert-file to PDF

Ta-da!

jpg-to-pdf-around-result.png

 

End with the ultimate power: create Any HTML and convert to PDF. 

  1. Try multiple headings,
  2. And repeat that image twice.
concat('<html>', '<h1>heading 1</h1>', variables('html'), '<h1>heading 2</h1>', variables('html'), '</html>')

Read and include some live data.

  1. Use SharePoint List Folder action and get a list of all the files
  2. Use Data Operations - Select to remap just the Path and Size properties (this is the same as Array.Map)
  3. Create HTML Table - with automatic columns.  We only have two fields.
  4. Concat into our HTML


Effectively, PDF-gen with templating

In this blog post, we covered some new and old techniques:

  • DataUri is our friend again
  • Convert-File to PDF
  • Select, Create HTML table

I leave the last example as a thought exercise for you, the reader.

  1. Store the HTML template in a HTML file.  You really don't want to be typing HTML in concat within a tiny formula box.
  2. Use Replace expression to replace REPLACE_ME words with values you plan to fill from a live data source.
  3. Insert images as DataUri strings to easily get your logo, headers into the PDF report.
  4. Consider using PDF to snapshot list item upon workflow completion, and then email that as PDF attachment to the manager.

Thank you for reading

I have a favour to ask.  See, I told @paulculmsee sneak peek about this post and he's like oh that's it?
If you think gosh this one's awesome, please tell him he's wrong :-)

 

Two ways to convert SharePoint files to PDF via Flow

/

This blog post is divided into three sections: The easy, The Auth and The Complete parts.

Microsoft Flow released a new power to Convert Files to PDF.  This made my October.  So of course we have to play with this.

Part 1. The Easy

Now this work well, but raises a few questions: 

  1. Why do I have to copy to OneDrive for Business?
    Because the Convert File action is also available for OneDrive for consumer, but not SharePoint
     
  2. Can I do this without copying to OneDrive for Business
    Not with the default Actions for now.  There's no Convert File for SharePoint Connector.  And SharePoint Connector's Get File Content action doesn't allow a format parameter.
convert-file-actions.png

And this is the simplest solution.

Warning: Next be dragons (Auth and API)

We are going to dive in to see what API this uses.  And whether we can call the same API on SharePoint library document directly without copying the file to OneDrive first.

This next part is good for you.  But it is heavy and will look complicated.  Brace yourselves.

...So what API does this use?

https://docs.microsoft.com/en-us/onedrive/developer/rest-api/api/driveitem_get_content_format 

GET /drive/items/{item-id}/content?format={format}
GET /drive/root:/{path and filename}:/content?format={format}

Specifically, this uses the Microsoft Graph

Part 2. The Auth

Disclaimer - OAuth looks familiar, but steps are always tricky.  Easy to mess up.  So if you are following this through, walk carefully.

For the next part, we need to connect to MS Graph with AppOnly permissions

In Azure Portal - under Azure AD - create an App Registeration (I'm reusing a powershell-group-app one I had previously baked)

client-id.png

We will be accessing files - so make sure Application Permissions for read files is granted.  This requires admin consent.

client-perms.png

Via the Azure AD portal - hit Grant Permissions to perform admin consent directly.

client-grant.png

Now we are going to write the Flow with HTTP requests

hit the token endpoint for our tenant with a POST message.  The Body must be grant_type=client_credential with client_id, client_secret and the resource is https://graph.microsoft.com

this request if successful will give us back a JSON.  Parse JSON with this schema:

{
    "type": "object",
    "properties": {
        "token_type": {
            "type": "string"
        },
        "expires_in": {
            "type": "string"
        },
        "ext_expires_in": {
            "type": "string"
        },
        "expires_on": {
            "type": "string"
        },
        "not_before": {
            "type": "string"
        },
        "resource": {
            "type": "string"
        },
        "access_token": {
            "type": "string"
        }
    }
}

This gives Flow a variable for access_token for the remainder of the steps to use to call Microsoft Graph

Test this by calling the MS Graph endpoint for SharePoint site

token-test.png

This HTTP request with the Bearer access_token successfully returns SharePoint site data from Microsoft Graph.

 

Part 3.  The Complete Solution to fetch SharePoint document as PDF

Call /content?format=PDF

get-content-format-redirect.png

A few things going on in this result.  

  1. Flow thinks this request has failed - because it doesn't return a 2xx status.  It returns a 302 redirect.
  2. The Response header contains the Location of the redirect, which is where the PDF file is

Parse JSON again on the Response header.  

{
    "type": "object",
    "properties": {
        "Transfer-Encoding": {
            "type": "string"
        },
        "request-id": {
            "type": "string"
        },
        "client-request-id": {
            "type": "string"
        },
        "x-ms-ags-diagnostic": {
            "type": "string"
        },
        "Duration": {
            "type": "string"
        },
        "Cache-Control": {
            "type": "string"
        },
        "Date": {
            "type": "string"
        },
        "Location": {
            "type": "string"
        },
        "Content-Type": {
            "type": "string"
        },
        "Content-Length": {
            "type": "string"
        }
    }
}

We just want Location.  We also need to configure Continue on previous HTTP error.

redirect-continue.png

And finally, retrieve the file via GET again

fetch-return.png

 

When ran, the flow looks like this:

run.png

 

Summary

The complete solution uses HTTP to call MS Graph directly and pulls back the PDF file after a 302 Response.  This is a fairly complex example so please evaluate whether you want the Correct Way or the Easy Way.

Note also that Microsoft Flow has a Premium connector for Azure AD Requests - which will negate the middle part of this blog post re: Auth and let you dive right into MS Graph REST endpoints without worrying about access_tokens.  

Call this Flow request and it downloads the PDF file, converted from a DOCX document in SharePoint team site.

 

Review Special Techniques Invoked:

  • MS Graph Auth
  • The Continue on Error configuration
  • Parse JSON on Response Header