Implementing a fast sort with Microsoft Flow using Parallel Compute

This is #FlowNinja hack 112. Parallel Compute Sort.

I had written about how to sort with a variable (this is insertion sort) back in 2018 How to implement sort with Microsoft Flow-in-3-actions-within-a-loop

But in this previous method, the use of variable means we can’t run apply to each in parallel, so this method was always slow when array is large. Today, while chatting with Hiro - I had a sudden idea to revisit the pattern and see if I can make this quicker.

Photo by Amy Shamblen on Unsplash

Photo by Amy Shamblen on Unsplash


The problem

To create a sort, using parallel apply to each, and get a sorted array at the end.



In 2018, I didn’t have many of the patterns I need to make this new 2020 sort method. Firstly, to get results from parallel apply to each, we need Pieter’s Method (Compose apply to each inner output) to fan-in after parallel fan-in.

Second, we need to sort the actual array, and I came up with a pretty interesting method.



How this works

Consider array [ “d”, “e”, “c”, “b”, “a” ]
If we say for each character, filter array for items that are < than the current item, we’d get:

3:d, 4:e, 2:c, 1:b, 0:a

Then, if we consider, hey, we have 5 items

[0, 1, 2, 3, 4] => map to this dictionary, we’d get [ “a”, “b”, “c”, “d”, “e” ]



Side Story

I actually was thinking about this pattern while driving home, once it clicked I had to pull over the side, park the car, take out my laptop and write this Flow, and after I saw it work I drove home.



Steps

Some additional considerations


If the original array has duplicates

[ “a”, “b”, “b”, “c” ]
0:a, 1:b, 3:c

We’ll see 2 is missing. This is not end of the world, but when we do the final map of

[0,1,2,3] => [ “a”, “b”, null, “c” ]

Observation

I was worried that JSON('{ "0": "a", "0":"a" }') would give an error, but it seems like the duplicate key is ignored. This could be an interesting way to detect duplicates in the future by building a dictionary.





Flow - Format Number advanced tips and tricks

I put together a video to celebrate the new update in Microsoft Flow (Power Automate) and Logic Apps - formatNumber()

The video explains 3 tips and 2 gotchas.

  • Tip 1: Use action in prod right now

  • Tip 2: Use ### and 000 patterns in formatNumber() expressions

  • Gotcha 2.1: May be don't use of $### - use $##0

  • Gotcha 2.2: May be don't use $ - use C or C2

  • Tip 3: Use formatNumber in collections with Select/Create HTML

If you want the clipboard paste of the Format Number action to use, use this.

{
    "id": "c1fa8a84-e2c0-4be0-823e-3a45-1e6834aa",
    "brandColor": "#a098e6",
    "connectionReferences": {},
    "icon": "https://psuxaustralia.azureedge.net/Content/Images/DesignerOperations/numberfunctions.png",
    "isTrigger": false,
    "operationName": "Format_number",
    "operationDefinition": {
        "type": "Expression",
        "kind": "FormatNumber",
        "inputs": {
            "number": 12345,
            "format": "C4"
        },
        "runAfter": {
            "Compose": [
                "Succeeded"
            ]
        }
    }
}

True Governance of the Power Platform

track-2.jpg

It’s very early Sunday morning and I’ve just returned from an red-eye flight home from a week spent in Perth with Paul, Ashlee and Terrie Culmsee, meeting their many clients, participating in Perth App in a Day, and talking about PowerApps, Flow and Power Platform Governance.

It is testament to Paul’s nurturing of his clients that every company that I talk to on this trip have the same forward thinking mentality.

Many of the companies I had talked to online, or here in Sydney also shares these same traits. They want to implement proper governance of the power platform.

You might be on the same journey - I wish you the best, and I want to support you in this quest.

True Governance is about creating a partnership

The scenario is the same, over and over and over. Business wants agility, they want applications that traditional IT struggles to deliver.

IT already struggle with supporting existing applications created by Business - they worry this is yet another thing they need to support - these “citizen developed apps” will fall into their laps, not following best practices, not documented, and yet highly business critical.

Power Platform is an opportunity, to ride this wave and fix the partnership problem in your business.

True leadership, are the managers seizing this unique opportunity to (re)engage Business and IT

Because traditional IT has become the department of “No” - modern business units and IT doesn’t really want to talk. And in the last few years, many businesses rode the Microsoft Power Platform submarine to get applications developed under the radar to reach critical mass first. Always easier to ask for forgiveness than blessing.

In this paragraph, I want to speak to you - the business. You must understand, there is an virtual ceiling you are about to hit. Let me explain.

See - your empowered citizen developers are learning as they go, their first apps may not be that fantastic, while they solve critical business problems - there may be bugs and we see there will always be more incremental improvements needed.

As the number of Power applications you create increases, your limited number of citizen developer will reach capacity. Without being able to properly support these apps, they can’t make more apps.

You still need more apps. But your citizen devs’ hands are now full. Worse, what happens if they take a role elsewhere? Who can support these apps?

The correct answer is, business should take a cost code to IT and ask IT to support these apps. IT should be paid to build internal capacity to understand, administer, and govern the Power Platform.

Ideally, someone that understands the Power Platform should lead this team and effort. They are the bridge that talks to Business and IT. The ultimate navigator that will unite the business and IT.

Look across to any successful enterprises now scaling to hundreds and thousands of Power Apps and Flows. This pattern is the same. Over and over.

IT must realize that part of governance is this great opportunity - go talk to your business. Find the super maker that is helping with adoption, and able to put together business plans that will include a maintenance support fee to IT. Stop being “free support” (see my previous blog post) and start being part of value generation in your business.

An emerging trend is that the internal super Power Platform maker joins the IT side of the business to be that navigator. Governance and adoption tools (I have a list near the end) support the quest of this navigator. Business pays IT to support and maintain more Power Apps and Flows.

Do you think this is not possible? This is a fairy tale? I saw these relationships, these bridges forming everywhere. Is this not your experience? Have a look around.

If you are an IT manager, rather than look at the Power Platform with fear, close your eyes, and reopen them and see it as the biggest opportunity you have ever had in the last decade.

A low code platform the Business wants, with tools and reports for IT to properly function and support this platform. Don’t squander this and drive your business away to some real Shadow IT platform and now you can’t even monitor that.

Governance tools are available - but true governance is People willing to talk

Business and IT not talking to each other is a problem. Here, is an opportunity to solve that. Take your off-shored IT back in-house. Be part of value generation.

If you want partnerships, I will help. There are many in the community that want to help you. Many Power platform champions are literally living this reality and working in this role. Borrow their job template and make it yours, initate the conversation - take it to your manager in business or in IT and say, hey, let’s do this.

I wanted to list a series of tools available. Some has (custom) tag and would need self assembly.

Build tools, build bridges

Disclaimer - I build Power Studio and Power Clarity. The point is, there are tools to provide governance on this platform. This is FAR better than choosing a different tool that IT has no means of providing any meaningful governance.

It would be somewhat easy for me to run purely on fear and say buy “my tools” - they will lock everything down and give you governance. Build a wall. You need a wall. Buy my wall.

If some snake oil salesman comes to you and say that, I want you to understand, real governance is not a wall. There is already enough of a chasm between business and IT.

Real governance is an opportunity to take this platform and transform your apps, yourself, your business and the relationships in your business. It’s hard, potentially very rewarding work.

What do you build your governance for?

Today, governance is a word thrown around without a care of what it actually means. We need to do governance - for what? No that’s all, we just need to do governance.

I wanted to share an article from Paul Culmsee. Those that sees governance as the goal, and not the means to an end, there’s something we all have to let go. Why do we implement governance? What outcome do we desire?

https://medium.com/@paulculmsee/how-to-doom-office365-governance-over-and-over-again-9feede8ef14a

If you have never thought about why you are implementing governance, and you don’t know what your end goal is, then take this quest:

We build governance, because our IT and Business must form a partnership. Our entire governance strategy is to make that partnership work.

The choices are yours, but I think you shouldn’t take forever to think about them

Do the right thing. Chances like this don’t come that often. Don’t look back and think, ah, I didn’t take that opportunity and initiate that conversation, and a decade later, I’m still only a cost center, still fighting shadow IT, and the board still wants to offshore my team.

That’d be unfortunate.

Presentations at the Digital Workplace Conference Australia 2019 I’m looking forward to

It’s really exciting for me to be back at the Digital Workplace Conference again in 2019, this is one of the first major conferences that I was very fortunate to be able to present at, and I have always looked forward to attend, to learn and to give back and present the latest happenings in SharePoint, Office 365, Azure and now also Power Platform.

The Digital Workplace Conference will be on August 6-7 in Sydney.

DWCAU-temp-dates.png


This year I’m presenting a topic (and sorry for the slightly concatenated topic name):

A quick introduction to Microsoft Flow, but rapidly takes audience deeper to see the possibilities of the types of solutions possible


There probably should be a line-break in there somewhere. You have to use your imagination for this title for that missing character, sorry!

Lets try that again!

An introduction to Microsoft Flow: Office 365 - Automated

This is an introduction session on Microsoft Flow, but my aim isn’t to show you a simple alert email. My aim is to explain that we have far transcended “Workflows”. It’s not just workflow. We are way beyond that - what we really have is a full Automation Engine. It is Office 365 - Automated.

Be very careful, writing Microsoft Flow is completely addictive.


The Sessions I want to see!

  • Become The Expert of You – Power Skills for Personal Development - Heather Newman

  • Trust-Based Corporate Culture: How to Kick Fear and Toxicity Out of the Workplace - Heather Newman

  • The Fruit Salad Formula for Implementing Ofice 365 - Debbie Ireland

  • Microsoft Teams Deep Dive – 29 Practical Tips and Tricks - Lee Stephens

  • Making Teams work without understanding Information Architecture - Alistair Pugin

  • A Zero-Hype Introduction to Artifical Intelligence and Machine Learning - Kilash Awati

  • Adding voice interaction to your apps - Brian Farnhill

  • Give your PowerApps and Flows some Vision, Language and Feeling - Rich Burdes

  • How to build a Project Hub with Hubsites and Sitedesign and Sitescripts - Knut Relbe-Moe

  • Success Factors in a Thriving Yammer Network - Rebecca Jackson

  • How to Govern PowerApps and Flow - Paul Culmsee

  • Governance & Adoption for Microsoft 365: Making the Marriage work - Megan Strant

But that’s not all

I’m at the conference both days and plan to hang around for the workshop day too. So if you have any questions on

  • Microsoft Flow, PowerApps, Power Platform Governance solutions

  • Azure LogicApps, Azure Functions

  • SharePoint, PnP, Microsoft Graph, SPFx

  • Flow Studio startup journey

  • The latest imaginations in the mind of a mad man

And whatever in between…
I’m available for a chat!

The value of a Physical Conference

I always feel the top value of a physical conference is being able to stash away two or three days to focus and immerse in an environment with like-minded people, we talk tech, have the same challenges, but may have solved some of them with our own personal takes of that experience.

We are very much alike - when we are together in the same conference - we are introduced to new ideas that gives us a much broader perspective than we would usually see when we are so focused on our immediate day to day business problems.

I treat it as a relaxed edu-vacation. Seek out the sessions that broadens my capacities. And grab hold of the presenters and get a deeper discussion to take that back with me.

I would love to see you (see you again) at the Digital Workplace Conference 2019. If you see me you should say Hi John - I want to ask / I want to tell you about <THIS THING>

Presenting a roadmap to learning and mastery of Microsoft Flow at Collab365 Microsoft Flow Virtual Summit

All the Flow you can eat!

c365-fvs-john-liu.png

https://events.collab365.community/microsoft-flow-virtual-summit-agenda/

I managed to squeeze in last minute with a session on “A roadmap to learning and mastery of Microsoft Flow” this is an advanced session aiming to cover a wide range of techniques and pointing out the pitfalls along these milestones.

The session is near the very end of the Flow Virtual Summit. So I hope you would join me live. (As of this post - the Flow Virtual Summit is on right now). If you want to catch the video on demand after the session, you will need to purchase an All Access Pass.

The review of milestones and roadmap is suitable for learning or navigating our learning along Microsoft Flow.


Affiliate Discount for Flow Virtual Summit All-Pass users

Flow Studio is affiliated with Flow Virtual Summit to offer an additional $20 off the first year of your subscription if you have an Flow Virtual Summit All Access Pass. So this is how it will work.

First year - US$80
Subsequent years - US$100

After July, where Flow Studio’s price increases to US$200

First year - US$180
Subsequent years - $200

So if you have purchased an Flow Virtual Summit All-Pass and is interested in purchasing a Flow Studio subscription - let me know!

Enjoy the show

https://events.collab365.community/microsoft-flow-virtual-summit-agenda/