Power Apps API changes - no longer accept Azure Management token

/

There has been recently a major change to the API and authorization that Flow Studio uses to access the Power Platform. As we have just pushed the update to production, we want to take a bit of time to write this blog post - why we are doing this, and how it might affect you.


Flow Studio uses a mixture of APIs available through Azure, Microsoft Graph, and Power Platform to provide a tool that lets us focus on our flows, and make sure they continue running successfully. Since the very beginning in 2018, Flow Studio does this by requesting an Azure authorization token when the user logs in. Since the Power Platform APIs sit on top of the Azure platform, this token was sufficient to access across multiple APIs.

In 2021 there was a big update with Power Automate APIs and now in 2023 there's a second update with Power Apps APIs. As a result, we made the decision to switch to granular, Power Platform specific permissions.

Scope, Permissions

"read flows", "manage flows" are required permissions to access Power Automate - see and update your flows, shared flows or solution flows.

"read activities" allows Flow Studio to read recent Power Automate events - we use this to figure out which flows have been running recently.

"read approvals" and "manage approvals" are related to Power Automate's approval feature. Flow Approvals is not an area we currently have major feature development in, but it is a tab that we have available, so we ask for this permission to maintain the feature.  Since Power Automate approvals v2, some of the data is also available via the default solution DataVerse entities. So if you are keen to do some approvals reporting, you can access them that way.

"Power Apps Service API" is for reading Power Apps and Power Platform connections.

"basic profile" is what most apps request to read the user's email address and display name.

"maintain access" is the "offline" access - since Flow Studio is a single web application we store your token in the browser's web storage temporarily (this is standard MSAL functionality).



Trust and Verified Publisher


Flow Studio is a trusted, verified publisher. This tick is provided by Microsoft, and tells our customers that we aren’t some random new app that’s popped up now asking for your permissions. We have been around since 2018, and have operated under Microsoft publisher guidelines in our use of the APIs and the care we have for our customers. If we do a bad thing, Microsoft knows how to find us.

We are a legal company registered in Australia, our office operates out of Sydney.



Path forward for Flow Studio

So from Flow Studio v1.1.41 or later - users will need to re-login and grant the new set of permissions to continue using Flow Studio.

Flow Studio for Teams and Enterprise is updated as of v0.1.070


Other users might be affected by this

  • If you use older versions of Power Platform Power Shell

  • If you have very old Power Apps connectors created from a long time ago

  • If you are using Office CLI to perform some actions on the Power Platform

You may see specific errors referring to "The received access token has been obtained from the wrong audience or resource".





Turning a new page

/

Wanted to write again, and let everyone know what I’ve been up to. I ended up taking a break through most of 2021 and 2022 simply resting, recovering, and doing light work from home.

2023 resumed with a big bang, I found motivation and drive to dive back into the many projects I’ve temporarily shelved in the last two years. I’ve also became pretty handy with a bunch of home DIY projects. It was a big change to the old me that only knew how to do digital projects but not physical projects. Perhaps more on that in a future post.

Flow Studio

We’ve had several Flow Studio fixes in the last two months

  • there was an API pagination fix since the API no longer accepts 250 records at once and restricts us to only 50. (That means more pages and API call takes longer)

  • API auth fix relating to Power Apps is in-progress.

  • There’s a second API skip/continuation token fix.

  • We’ve also tweaked the way trail is applied when anyone wants to try Flow Studio pro - you can sign up and trial will be available for two weeks - you can cancel the subscription before the trial end date to avoid being charged, if Flow Studio isn’t suitable for you.

Clarity / Flow Studio for Teams and Enterprise

We’ve had renewed interest in Flow Studio for Teams and Enterprise (Clarity) through the last few years.

  • Flow Studio for Teams will be tweaked to focus on monitoring critical flows and alerting users when their business critical flows fail. This will be priced simply and does not offer governance capabilities.

  • Flow Studio for Enterprise will be focused on the turnkey Power Platform Governance story, adding new features to scan more areas of the Power Platform, and integrate with CoE starter kit.

  • So far this year, we’ve added BYO Azure Storage. There’s been a lot of fixes to API breakages in Power Apps area this year.

Contract Work

I started a regular part time contract work in Sydney CBD, so if you are local, hit me up for a coffee.

  • I’m working with a lot of Power BI reports

  • There’s a lot of Power Automate doing the heavy lifting as well.

  • We are also talking about adding some Power Apps visuals to allow executive comments to be collected during a report presentation.

Community

Several of the meetups, conferences and events that I used to participate in are becoming active again. I hope to see more of the community not just virtually, but physically as well. I hope to be able to grab a coffee with you soon.

Our Legacy

/

This blog post is quite personal and doesn’t have much to do with tech. So if that’s not what you are here for, wait for the next blog post. This is personal to me though, so if you want to read about me, everything with flaws and all, then follow along.

Why am I writing this here on my blog?

I’ve been watching (I know, late) Hamilton the musical, and the ending “who lives, who dies, who tells your story” really startled me. In the story, the widow lives another 50 years after him and did all sort of things to tell his legacy. It really got me thinking - what about my wife, who left us so young, so soon, who will tell the story of this quiet girl that was never in the public spotlight, but fun, loving, always happy and full of kindness. Who else, but me? The person that loved and knew her the most.

Where do I even write this? Hamilton, being a founding father of America - has scores of historians looking up all the letters, correspondences and essays that he wrote during his time. There’s nothing like that for Lina Abidin, she didn’t write many things, and there will be no historians to talk about her one day, but we do live in the Internet age so I will write about her, on the platform that will be archived for the world to remember, on my public blog.

What is our legacy?

John as a young man would be quite similar to Hamilton - young, hungry and scrappy. He believed he is pretty smart and can do anything he put his mind to. As he grew in his career - he dreamed of starting his own company and start up.

Lina as a young lady did not have grand aspirations (or grand delusions?) as John, but she always saw herself as someone that will use her financial skills and sharp wits to help her husband. So in 2018, when we finally started my own company Flow Studio - Lina was delighted. She didn’t need us to be rich beyond our wildest dreams - she just wanted a comfortable living with me and our kids.

What is Lina’s skill? She is extremely talented at finding and understanding numbers, and how to calculate best outcomes. Tax and accounting comes naturally to her, financial modelling is her forte, and pretty much every computer game we played together she has a fully working Excel spreadsheet powering the calculations and the shortest paths to the best rewards in the game. She has more excel files than games in her Steam library.

Hey kids, it’s actually all about you two

As I looked back in the final period of our lives together, her concerns were increasingly focused on our children. She wanted them to eat the yummy food that she could cook. She wanted to knit more articles of clothing and make crochet toys for them. She wanted to sort out her will so that they will be protected for the kids so even John can’t screw it up.

Ultimately, I had thought our legacy would be some sort of wealth that we’ve built together. But when it all comes down to the end, it was our two lovely kids that’s the only thing on her mind.

We thought perhaps she can pre-record some things to say to them when they come of age, and we tried, we really tried - but Lina could not stop crying when we try to record these, she says she wanted to see their wedding, their partners and their children and how unfair it all seems that she would not be there to see them graduate, or hit any of these life milestones.

I didn’t have anything to say to her, I can only hug her and say “hey, if anything feels too hard, let me take care of it, OK?”.
“OK, she said”.

With that she was able to stop worrying at the end and enter into rest.

Future

What’s next for us, for the kids, for our company, for our livelihood? I have no idea. But I will do my absolute best to raise her legacy. I was very much reminded of this this week, and I wanted to commit my promises to her, to words.

May the Internet and our kids be my witness.

Research: Power Automate comments are stored in Dataverse

/

Out of curiosity, and because I think listing outstanding comments might be a product feature in Flow Studio, I did some research, experimenting with REST query and general exploring on a late Friday night. This was done on Twitter, a bunch of people chipped in but also we ended up with pictures all over the place. This blog post is to collect everything in one place as a reference.

The record is stored as a top level Container (with an artifactid)

Then top level records (kind=Threads) with Container(commentid = containerid)
Then records (kind=Reply)

We can query Dataverse to get the rows back. Here’s how to do it in one request.


We can query Dataverse comments from WITHIN the flow about the current flow (via workflow().name expression)

I think this fulfills some sort of inception criteria

This also qualifies this research to be #FlowNinja hack 125

Here is a Reply record.

Comments are tied to an Anchor - which is an action within the Flow.
The corresponding Flow has metadata.operationMetadataId created as a reference when this happens.

Comments can be resolved (state =1, Resolved) or deleted (statuscode = 2, Inactive).

I expect the same data structure design will be suitable for other Power Platform products, so I’m keen to see it.

Parse CSV through Code in Power Automate Custom Connection

/

I was inspired reading Alex Shlega and Hiroaki Nagao ’s posts on using code with custom connections. So I set out to give it a go and work on another common problem I have: Parse CSV

First a picture showing you how it works.

Give it text, the action returns array of arrays.

Microsoft’s docs are here.
Write code in a custom connector | Microsoft Docs

And particularly, I need to parse CSV without using additional libraries, and using only the existing libraries available here. I noted that we do have access to System.Text.RegularExpressions, so I started my planning there.

Because parsing CSV correctly is a problem best sorted via use of a tokenizer, I went looking for a regular expression pattern that treats each line as a series of tokens. There are many patterns, but I like this one that I found on stackoverflow the best for my needs. https://stackoverflow.com/a/48806378

Code

So the code takes all the content of the body and splits by line breaks, then the regular expression is run over every line using Matches (this method returns multiple matches giving us a MatchCollection of tokens). In each match, I look for Group[2] which is the value without quotes “ and “. But if failing that match, we take Group[1] value.
We do not take the Match.Value because that would include the comma.

/end of regular expression explanation.

We cast the matches back to array via Linq and then back to JArray and return that back to Flow.

public class Script : ScriptBase
{
    public override async Task<HttpResponseMessage> ExecuteAsync()
    {
        if (this.Context.OperationId == "csv")
        {
            return await this.HandleCSV().ConfigureAwait(false);
        }

        HttpResponseMessage response = new HttpResponseMessage(HttpStatusCode.BadRequest);
        response.Content = CreateJsonContent($"Unknown operation ID '{this.Context.OperationId}'");
        return response;
    }

    private async Task<HttpResponseMessage> HandleCSV()
    {
        var contentAsString = await this.Context.Request.Content.ReadAsStringAsync().ConfigureAwait(false);

        // (?:,|\n|^)("(?:(?:"")*[^"]*)*"|[^",\n]*|(?:\n|$))
        // https://stackoverflow.com/a/48806378

        var re = new Regex("(?!$)(\"((?:(?:\"\")*[^\"]*)*)\"|[^\",\r\n]*)(?:,|$)");
        var lines = Regex.Split(contentAsString, "\r\n|\r|\n");

        var result = new JArray(lines.Select(line=>{ 
            var matches = re.Matches(line);
            return new JArray(matches.Cast<Match>().Select(match => {
                return match.Groups[2].Success ?
                match.Groups[2].Value :
                match.Groups[1].Value;
            } ).ToArray());
        }).ToArray());

        var response = new HttpResponseMessage(HttpStatusCode.OK);
        response.Content = CreateJsonContent(result.ToString());
        return response;
    }
}

explain the regex and match groups

Swagger

This is the custom connection swagger YAML file.

swagger: '2.0'
info: {title: CustomCode, description: Custom Code, version: '1.0'}
host: johnliu.net
basePath: /
schemes: [https]
consumes: []
produces: []
paths:
  /Csv:
    post:
      responses:
        default:
          description: default
          schema:
            type: array
            items: {}
            description: Array
            title: Array
      summary: Parse CSV
      description: Parse CSV
      operationId: csv
      parameters:
      - name: value
        in: body
        required: true
        schema: {type: string, description: Text, title: value}
      x-ms-visibility: important
definitions: {}
parameters: {}
responses: {}
securityDefinitions: {}
security: []
tags: []

I want to add more parameters over time, and that will involve a tweak to the input parameters on the Swagger definition. But that’s probably a task for another day.

Links:

Write code in a custom connector | Microsoft Docs

C# code in Power Automate: let’s sort a string array? | It Ain't Boring (itaintboring.com)

Calculate Sum & Average in Power Automate using C# code in a custom connector - MoreBeerMorePower (hatenablog.com)