Our Legacy

/

This blog post is quite personal and doesn’t have much to do with tech. So if that’s not what you are here for, wait for the next blog post. This is personal to me though, so if you want to read about me, everything with flaws and all, then follow along.

Why am I writing this here on my blog?

I’ve been watching (I know, late) Hamilton the musical, and the ending “who lives, who dies, who tells your story” really startled me. In the story, the widow lives another 50 years after him and did all sort of things to tell his legacy. It really got me thinking - what about my wife, who left us so young, so soon, who will tell the story of this quiet girl that was never in the public spotlight, but fun, loving, always happy and full of kindness. Who else, but me? The person that loved and knew her the most.

Where do I even write this? Hamilton, being a founding father of America - has scores of historians looking up all the letters, correspondences and essays that he wrote during his time. There’s nothing like that for Lina Abidin, she didn’t write many things, and there will be no historians to talk about her one day, but we do live in the Internet age so I will write about her, on the platform that will be archived for the world to remember, on my public blog.

What is our legacy?

John as a young man would be quite similar to Hamilton - young, hungry and scrappy. He believed he is pretty smart and can do anything he put his mind to. As he grew in his career - he dreamed of starting his own company and start up.

Lina as a young lady did not have grand aspirations (or grand delusions?) as John, but she always saw herself as someone that will use her financial skills and sharp wits to help her husband. So in 2018, when we finally started my own company Flow Studio - Lina was delighted. She didn’t need us to be rich beyond our wildest dreams - she just wanted a comfortable living with me and our kids.

What is Lina’s skill? She is extremely talented at finding and understanding numbers, and how to calculate best outcomes. Tax and accounting comes naturally to her, financial modelling is her forte, and pretty much every computer game we played together she has a fully working Excel spreadsheet powering the calculations and the shortest paths to the best rewards in the game. She has more excel files than games in her Steam library.

Hey kids, it’s actually all about you two

As I looked back in the final period of our lives together, her concerns were increasingly focused on our children. She wanted them to eat the yummy food that she could cook. She wanted to knit more articles of clothing and make crochet toys for them. She wanted to sort out her will so that they will be protected for the kids so even John can’t screw it up.

Ultimately, I had thought our legacy would be some sort of wealth that we’ve built together. But when it all comes down to the end, it was our two lovely kids that’s the only thing on her mind.

We thought perhaps she can pre-record some things to say to them when they come of age, and we tried, we really tried - but Lina could not stop crying when we try to record these, she says she wanted to see their wedding, their partners and their children and how unfair it all seems that she would not be there to see them graduate, or hit any of these life milestones.

I didn’t have anything to say to her, I can only hug her and say “hey, if anything feels too hard, let me take care of it, OK?”.
“OK, she said”.

With that she was able to stop worrying at the end and enter into rest.

Future

What’s next for us, for the kids, for our company, for our livelihood? I have no idea. But I will do my absolute best to raise her legacy. I was very much reminded of this this week, and I wanted to commit my promises to her, to words.

May the Internet and our kids be my witness.

Research: Power Automate comments are stored in Dataverse

/

Out of curiosity, and because I think listing outstanding comments might be a product feature in Flow Studio, I did some research, experimenting with REST query and general exploring on a late Friday night. This was done on Twitter, a bunch of people chipped in but also we ended up with pictures all over the place. This blog post is to collect everything in one place as a reference.

The record is stored as a top level Container (with an artifactid)

Then top level records (kind=Threads) with Container(commentid = containerid)
Then records (kind=Reply)

We can query Dataverse to get the rows back. Here’s how to do it in one request.


We can query Dataverse comments from WITHIN the flow about the current flow (via workflow().name expression)

I think this fulfills some sort of inception criteria

This also qualifies this research to be #FlowNinja hack 125

Here is a Reply record.

Comments are tied to an Anchor - which is an action within the Flow.
The corresponding Flow has metadata.operationMetadataId created as a reference when this happens.

Comments can be resolved (state =1, Resolved) or deleted (statuscode = 2, Inactive).

I expect the same data structure design will be suitable for other Power Platform products, so I’m keen to see it.

Parse CSV through Code in Power Automate Custom Connection

/

I was inspired reading Alex Shlega and Hiroaki Nagao ’s posts on using code with custom connections. So I set out to give it a go and work on another common problem I have: Parse CSV

First a picture showing you how it works.

Give it text, the action returns array of arrays.

Microsoft’s docs are here.
Write code in a custom connector | Microsoft Docs

And particularly, I need to parse CSV without using additional libraries, and using only the existing libraries available here. I noted that we do have access to System.Text.RegularExpressions, so I started my planning there.

Because parsing CSV correctly is a problem best sorted via use of a tokenizer, I went looking for a regular expression pattern that treats each line as a series of tokens. There are many patterns, but I like this one that I found on stackoverflow the best for my needs. https://stackoverflow.com/a/48806378

Code

So the code takes all the content of the body and splits by line breaks, then the regular expression is run over every line using Matches (this method returns multiple matches giving us a MatchCollection of tokens). In each match, I look for Group[2] which is the value without quotes “ and “. But if failing that match, we take Group[1] value.
We do not take the Match.Value because that would include the comma.

/end of regular expression explanation.

We cast the matches back to array via Linq and then back to JArray and return that back to Flow.

public class Script : ScriptBase
{
    public override async Task<HttpResponseMessage> ExecuteAsync()
    {
        if (this.Context.OperationId == "csv")
        {
            return await this.HandleCSV().ConfigureAwait(false);
        }

        HttpResponseMessage response = new HttpResponseMessage(HttpStatusCode.BadRequest);
        response.Content = CreateJsonContent($"Unknown operation ID '{this.Context.OperationId}'");
        return response;
    }

    private async Task<HttpResponseMessage> HandleCSV()
    {
        var contentAsString = await this.Context.Request.Content.ReadAsStringAsync().ConfigureAwait(false);

        // (?:,|\n|^)("(?:(?:"")*[^"]*)*"|[^",\n]*|(?:\n|$))
        // https://stackoverflow.com/a/48806378

        var re = new Regex("(?!$)(\"((?:(?:\"\")*[^\"]*)*)\"|[^\",\r\n]*)(?:,|$)");
        var lines = Regex.Split(contentAsString, "\r\n|\r|\n");

        var result = new JArray(lines.Select(line=>{ 
            var matches = re.Matches(line);
            return new JArray(matches.Cast<Match>().Select(match => {
                return match.Groups[2].Success ?
                match.Groups[2].Value :
                match.Groups[1].Value;
            } ).ToArray());
        }).ToArray());

        var response = new HttpResponseMessage(HttpStatusCode.OK);
        response.Content = CreateJsonContent(result.ToString());
        return response;
    }
}

explain the regex and match groups

Swagger

This is the custom connection swagger YAML file.

swagger: '2.0'
info: {title: CustomCode, description: Custom Code, version: '1.0'}
host: johnliu.net
basePath: /
schemes: [https]
consumes: []
produces: []
paths:
  /Csv:
    post:
      responses:
        default:
          description: default
          schema:
            type: array
            items: {}
            description: Array
            title: Array
      summary: Parse CSV
      description: Parse CSV
      operationId: csv
      parameters:
      - name: value
        in: body
        required: true
        schema: {type: string, description: Text, title: value}
      x-ms-visibility: important
definitions: {}
parameters: {}
responses: {}
securityDefinitions: {}
security: []
tags: []

I want to add more parameters over time, and that will involve a tweak to the input parameters on the Swagger definition. But that’s probably a task for another day.

Links:

Write code in a custom connector | Microsoft Docs

C# code in Power Automate: let’s sort a string array? | It Ain't Boring (itaintboring.com)

Calculate Sum & Average in Power Automate using C# code in a custom connector - MoreBeerMorePower (hatenablog.com)

Tiny forward steps for me and Flow Studio

/
miriam-eh-kWWeA1DVQxY-unsplash (1).jpg

I’ve been taking little tiny steps forward, this is a good time to share what I’ve been up to.

Flow Studio

There are several Flow Studio updates in the works. v1.1.00 is coming out very soon, and there are several important updates:

  • The development build is on https://dev.flowstudio.app/

  • The major update are changes to the backend APIs that flow studio calls. I am really thankful for the extra time given for me to work on this.

  • In subscriptions link there is now a way to see and manage your own subscriptions, using the Stripe billing support page.

  • There are a number of UX fixes to improve performance and bug fixes since last update.

Please let me know if you have any issues with Flow Studio. Bug fixes are priority.

Branding and Product Offerings

I renamed from Flow Studio to Power Studio last year, and introduced a new product Power Clarity. This has created a lot of additional branding work that’s time consuming to manage. So I’ve decided to simplify everything.

Original Updated Notes
Power Studio Flow Studio
Flow Studio Flow Studio We are going back to our original name
Power Studio Free   Flow Studio Free The freemium offering
Power Studio Pro Flow Studio Pro US$20 Monthly / US$200 Yearly
Power Clarity Flow Studio Teams US$2000 Yearly
Power Clarity Flow Studio Enterprise   Enquire

Flow Studio Free is for everyone

  • quickly see all your own flows

  • manage them using a tool designed for bulk operations.

Flow Studio Pro is for power users who needs more operations

  • Editing Flow JSON

  • Migrate flows

  • Bulk cancellation

  • Export flow history

  • Restore Flow to earlier versions.

  • Administrators that wants to see all the flows in their environment

Flow Studio Teams is for teams to manage all the flows used by their team

  • See all your flows, across multiple accounts

  • Manage them in bulk

  • Continuous monitoring of flow runs

  • Continuous backups

Flow Studio Enterprise is for IT and compliance departments that wants to have full visibility to understand, manage and utilize all the flows used within an organization.

  • Licensing assistance

  • Backup, migrate flows between accounts

  • DLP policy assistance and violations

  • Continuous reports

  • Cross tenancy support


How are you doing, John?

I’m OK, thank you for reaching out. Some days are really good, other days I can’t do it and just take a break.

It is currently school holidays in Sydney and we are in a new lockdown, so stuck at home. Let me know if you want to chat, I can find time to chat (remotely is OK).

2021 - broken, I am missing you Lina Abidin

/

I told myself this eventually would happen, but I still am not prepared for it when it did. How can anyone be prepared for what happens next.

On March 25th, a couple of hours past midnight, the love of my life Lina passed away peacefully in her sleep.

We had been fighting cancer for the last 5 years, it really hurts that we couldn’t beat it, and she has to go so young. Lina barely made it past her birthday in February, she was 41. We met each other when we were 20s and she’s been grinning at me for the last 21 years.

Lina is someone I don’t write a lot about, but has always been the driving force of many things that I did (or rather, we did together). She understood what I do - she was the first to cheer for my MVP award (we went to celebrate with yummy food!), she took care of the kids, she took care of our business, she’s always got my back, she made sure I smile - she is someone that is always smiling.

P1010506.JPG

So now I have the near impossible task of raising our two young kids and I really don’t know how I’m going to do this alone.

Everywhere I look, I see her. I wish, I had more time to tell her how much I’m missing her.

20171022_132038.jpg
141752196_10158882493054220_7250287982114968951_o.jpg

In my January post I said I would probably take time off until April. I’m going to need more time. Right now the kids are in school holidays for two weeks, and I’m taking the time to care for them. As school resumes next semester, I’m going to see if I can ease myself back into work.

  • There’s work to do to update Flow Studio.

  • Have some ideas and plans for Clarity - but more importantly, I’m really keen to hear from friends and teams that needs automated flow monitoring. Let me know if that is you.

  • I also have consulting work to resume with a few clients.

Lots to do, but some days, I crash and can’t find the ability to pull myself together to work on anything.

In her final days, she’d grab my face to tell me:

Remember to smile, don’t bottle everything in your heart.

I don’t know how to smile anymore, I cry and wail, as someone whose heart is utterly broken does.