Turning a new page

/

Wanted to write again, and let everyone know what I’ve been up to. I ended up taking a break through most of 2021 and 2022 simply resting, recovering, and doing light work from home.

2023 resumed with a big bang, I found motivation and drive to dive back into the many projects I’ve temporarily shelved in the last two years. I’ve also became pretty handy with a bunch of home DIY projects. It was a big change to the old me that only knew how to do digital projects but not physical projects. Perhaps more on that in a future post.

Flow Studio

We’ve had several Flow Studio fixes in the last two months

  • there was an API pagination fix since the API no longer accepts 250 records at once and restricts us to only 50. (That means more pages and API call takes longer)

  • API auth fix relating to Power Apps is in-progress.

  • There’s a second API skip/continuation token fix.

  • We’ve also tweaked the way trail is applied when anyone wants to try Flow Studio pro - you can sign up and trial will be available for two weeks - you can cancel the subscription before the trial end date to avoid being charged, if Flow Studio isn’t suitable for you.

Clarity / Flow Studio for Teams and Enterprise

We’ve had renewed interest in Flow Studio for Teams and Enterprise (Clarity) through the last few years.

  • Flow Studio for Teams will be tweaked to focus on monitoring critical flows and alerting users when their business critical flows fail. This will be priced simply and does not offer governance capabilities.

  • Flow Studio for Enterprise will be focused on the turnkey Power Platform Governance story, adding new features to scan more areas of the Power Platform, and integrate with CoE starter kit.

  • So far this year, we’ve added BYO Azure Storage. There’s been a lot of fixes to API breakages in Power Apps area this year.

Contract Work

I started a regular part time contract work in Sydney CBD, so if you are local, hit me up for a coffee.

  • I’m working with a lot of Power BI reports

  • There’s a lot of Power Automate doing the heavy lifting as well.

  • We are also talking about adding some Power Apps visuals to allow executive comments to be collected during a report presentation.

Community

Several of the meetups, conferences and events that I used to participate in are becoming active again. I hope to see more of the community not just virtually, but physically as well. I hope to be able to grab a coffee with you soon.

Our Legacy

/

This blog post is quite personal and doesn’t have much to do with tech. So if that’s not what you are here for, wait for the next blog post. This is personal to me though, so if you want to read about me, everything with flaws and all, then follow along.

Why am I writing this here on my blog?

I’ve been watching (I know, late) Hamilton the musical, and the ending “who lives, who dies, who tells your story” really startled me. In the story, the widow lives another 50 years after him and did all sort of things to tell his legacy. It really got me thinking - what about my wife, who left us so young, so soon, who will tell the story of this quiet girl that was never in the public spotlight, but fun, loving, always happy and full of kindness. Who else, but me? The person that loved and knew her the most.

Where do I even write this? Hamilton, being a founding father of America - has scores of historians looking up all the letters, correspondences and essays that he wrote during his time. There’s nothing like that for Lina Abidin, she didn’t write many things, and there will be no historians to talk about her one day, but we do live in the Internet age so I will write about her, on the platform that will be archived for the world to remember, on my public blog.

What is our legacy?

John as a young man would be quite similar to Hamilton - young, hungry and scrappy. He believed he is pretty smart and can do anything he put his mind to. As he grew in his career - he dreamed of starting his own company and start up.

Lina as a young lady did not have grand aspirations (or grand delusions?) as John, but she always saw herself as someone that will use her financial skills and sharp wits to help her husband. So in 2018, when we finally started my own company Flow Studio - Lina was delighted. She didn’t need us to be rich beyond our wildest dreams - she just wanted a comfortable living with me and our kids.

What is Lina’s skill? She is extremely talented at finding and understanding numbers, and how to calculate best outcomes. Tax and accounting comes naturally to her, financial modelling is her forte, and pretty much every computer game we played together she has a fully working Excel spreadsheet powering the calculations and the shortest paths to the best rewards in the game. She has more excel files than games in her Steam library.

Hey kids, it’s actually all about you two

As I looked back in the final period of our lives together, her concerns were increasingly focused on our children. She wanted them to eat the yummy food that she could cook. She wanted to knit more articles of clothing and make crochet toys for them. She wanted to sort out her will so that they will be protected for the kids so even John can’t screw it up.

Ultimately, I had thought our legacy would be some sort of wealth that we’ve built together. But when it all comes down to the end, it was our two lovely kids that’s the only thing on her mind.

We thought perhaps she can pre-record some things to say to them when they come of age, and we tried, we really tried - but Lina could not stop crying when we try to record these, she says she wanted to see their wedding, their partners and their children and how unfair it all seems that she would not be there to see them graduate, or hit any of these life milestones.

I didn’t have anything to say to her, I can only hug her and say “hey, if anything feels too hard, let me take care of it, OK?”.
“OK, she said”.

With that she was able to stop worrying at the end and enter into rest.

Future

What’s next for us, for the kids, for our company, for our livelihood? I have no idea. But I will do my absolute best to raise her legacy. I was very much reminded of this this week, and I wanted to commit my promises to her, to words.

May the Internet and our kids be my witness.

Research: Power Automate comments are stored in Dataverse

/

Out of curiosity, and because I think listing outstanding comments might be a product feature in Flow Studio, I did some research, experimenting with REST query and general exploring on a late Friday night. This was done on Twitter, a bunch of people chipped in but also we ended up with pictures all over the place. This blog post is to collect everything in one place as a reference.

The record is stored as a top level Container (with an artifactid)

Then top level records (kind=Threads) with Container(commentid = containerid)
Then records (kind=Reply)

We can query Dataverse to get the rows back. Here’s how to do it in one request.


We can query Dataverse comments from WITHIN the flow about the current flow (via workflow().name expression)

I think this fulfills some sort of inception criteria

This also qualifies this research to be #FlowNinja hack 125

Here is a Reply record.

Comments are tied to an Anchor - which is an action within the Flow.
The corresponding Flow has metadata.operationMetadataId created as a reference when this happens.

Comments can be resolved (state =1, Resolved) or deleted (statuscode = 2, Inactive).

I expect the same data structure design will be suitable for other Power Platform products, so I’m keen to see it.

Parse CSV through Code in Power Automate Custom Connection

/

I was inspired reading Alex Shlega and Hiroaki Nagao ’s posts on using code with custom connections. So I set out to give it a go and work on another common problem I have: Parse CSV

First a picture showing you how it works.

Give it text, the action returns array of arrays.

Microsoft’s docs are here.
Write code in a custom connector | Microsoft Docs

And particularly, I need to parse CSV without using additional libraries, and using only the existing libraries available here. I noted that we do have access to System.Text.RegularExpressions, so I started my planning there.

Because parsing CSV correctly is a problem best sorted via use of a tokenizer, I went looking for a regular expression pattern that treats each line as a series of tokens. There are many patterns, but I like this one that I found on stackoverflow the best for my needs. https://stackoverflow.com/a/48806378

Code

So the code takes all the content of the body and splits by line breaks, then the regular expression is run over every line using Matches (this method returns multiple matches giving us a MatchCollection of tokens). In each match, I look for Group[2] which is the value without quotes “ and “. But if failing that match, we take Group[1] value.
We do not take the Match.Value because that would include the comma.

/end of regular expression explanation.

We cast the matches back to array via Linq and then back to JArray and return that back to Flow.

public class Script : ScriptBase
{
    public override async Task<HttpResponseMessage> ExecuteAsync()
    {
        if (this.Context.OperationId == "csv")
        {
            return await this.HandleCSV().ConfigureAwait(false);
        }

        HttpResponseMessage response = new HttpResponseMessage(HttpStatusCode.BadRequest);
        response.Content = CreateJsonContent($"Unknown operation ID '{this.Context.OperationId}'");
        return response;
    }

    private async Task<HttpResponseMessage> HandleCSV()
    {
        var contentAsString = await this.Context.Request.Content.ReadAsStringAsync().ConfigureAwait(false);

        // (?:,|\n|^)("(?:(?:"")*[^"]*)*"|[^",\n]*|(?:\n|$))
        // https://stackoverflow.com/a/48806378

        var re = new Regex("(?!$)(\"((?:(?:\"\")*[^\"]*)*)\"|[^\",\r\n]*)(?:,|$)");
        var lines = Regex.Split(contentAsString, "\r\n|\r|\n");

        var result = new JArray(lines.Select(line=>{ 
            var matches = re.Matches(line);
            return new JArray(matches.Cast<Match>().Select(match => {
                return match.Groups[2].Success ?
                match.Groups[2].Value :
                match.Groups[1].Value;
            } ).ToArray());
        }).ToArray());

        var response = new HttpResponseMessage(HttpStatusCode.OK);
        response.Content = CreateJsonContent(result.ToString());
        return response;
    }
}

explain the regex and match groups

Swagger

This is the custom connection swagger YAML file.

swagger: '2.0'
info: {title: CustomCode, description: Custom Code, version: '1.0'}
host: johnliu.net
basePath: /
schemes: [https]
consumes: []
produces: []
paths:
  /Csv:
    post:
      responses:
        default:
          description: default
          schema:
            type: array
            items: {}
            description: Array
            title: Array
      summary: Parse CSV
      description: Parse CSV
      operationId: csv
      parameters:
      - name: value
        in: body
        required: true
        schema: {type: string, description: Text, title: value}
      x-ms-visibility: important
definitions: {}
parameters: {}
responses: {}
securityDefinitions: {}
security: []
tags: []

I want to add more parameters over time, and that will involve a tweak to the input parameters on the Swagger definition. But that’s probably a task for another day.

Links:

Write code in a custom connector | Microsoft Docs

C# code in Power Automate: let’s sort a string array? | It Ain't Boring (itaintboring.com)

Calculate Sum & Average in Power Automate using C# code in a custom connector - MoreBeerMorePower (hatenablog.com)

Tiny forward steps for me and Flow Studio

/
miriam-eh-kWWeA1DVQxY-unsplash (1).jpg

I’ve been taking little tiny steps forward, this is a good time to share what I’ve been up to.

Flow Studio

There are several Flow Studio updates in the works. v1.1.00 is coming out very soon, and there are several important updates:

  • The development build is on https://dev.flowstudio.app/

  • The major update are changes to the backend APIs that flow studio calls. I am really thankful for the extra time given for me to work on this.

  • In subscriptions link there is now a way to see and manage your own subscriptions, using the Stripe billing support page.

  • There are a number of UX fixes to improve performance and bug fixes since last update.

Please let me know if you have any issues with Flow Studio. Bug fixes are priority.

Branding and Product Offerings

I renamed from Flow Studio to Power Studio last year, and introduced a new product Power Clarity. This has created a lot of additional branding work that’s time consuming to manage. So I’ve decided to simplify everything.

Original Updated Notes
Power Studio Flow Studio
Flow Studio Flow Studio We are going back to our original name
Power Studio Free   Flow Studio Free The freemium offering
Power Studio Pro Flow Studio Pro US$20 Monthly / US$200 Yearly
Power Clarity Flow Studio Teams US$2000 Yearly
Power Clarity Flow Studio Enterprise   Enquire

Flow Studio Free is for everyone

  • quickly see all your own flows

  • manage them using a tool designed for bulk operations.

Flow Studio Pro is for power users who needs more operations

  • Editing Flow JSON

  • Migrate flows

  • Bulk cancellation

  • Export flow history

  • Restore Flow to earlier versions.

  • Administrators that wants to see all the flows in their environment

Flow Studio Teams is for teams to manage all the flows used by their team

  • See all your flows, across multiple accounts

  • Manage them in bulk

  • Continuous monitoring of flow runs

  • Continuous backups

Flow Studio Enterprise is for IT and compliance departments that wants to have full visibility to understand, manage and utilize all the flows used within an organization.

  • Licensing assistance

  • Backup, migrate flows between accounts

  • DLP policy assistance and violations

  • Continuous reports

  • Cross tenancy support


How are you doing, John?

I’m OK, thank you for reaching out. Some days are really good, other days I can’t do it and just take a break.

It is currently school holidays in Sydney and we are in a new lockdown, so stuck at home. Let me know if you want to chat, I can find time to chat (remotely is OK).