Thankful for MVP 2017 and 2016 in Review

I've just received email that I've been rewarded with Microsoft MVP for 2017.  I am thankful to be counted with some of the smartest people that does what we do.  I still stay up and wait for this email.  It is still very exciting :-)

2016 was a wonderful year

It was a wonderfully busy year for me personally.  (I say personal, because learning lots of new JavaScript is nothing but personal).

In the beginning of the year, we* were coming off a new Angular module that was built in 2015, and with that fresh in our minds, marching into 2016 with a focus on modernizing our Javascript stack (to Angular - which seems like the only choice at the time).

This was also the time where we talked about building Angular applications for Office and stepping out of not just Intranets but even into Office client applications.

May the 4th 2016 was amazing with Jeff Teper stepping out and stamped SharePoint back into the minds of many unbelievers who had brushed it off.  We had some inking this was coming - but it was hard to tell what effect this would have set off in the SharePoint world - 7 months later, I think the SharePoint team has successfully followed up with several big hits.

https://blogs.office.com/2016/05/04/the-future-of-sharepoint/

About midway through the year, we started working with Webpack and realizing that it will revolutionize our entire build pipeline.  (As a bonus, it unifies our work across different frameworks - some KnockoutJS, some DurandalJS and the new modules in Angular).

If you aren't sure whether to learn React or Angular - Learn Webpack.

https://webpack.js.org/

There's going to be lots of JavaScript...

In the meantime, our paths crossed with Nora at Telerik (Progress) who was looking for someone to contribute to a Office 365 or SharePoint based whitepaper.  There were several synergies in our meeting - we wanted to explore deeply using an existing KendoUI framework with SharePoint, we know SharePoint Framework was coming and there's a lot of talks of the toolchain needing to be refreshed.

http://developer.telerik.com/featured/how-to-prepare-your-toolbox-for-the-sharepoint-framework/

We also saw how Webpack fitted in all the pieces, and picked up PnP JS Core - new at the time, but rapidly gaining momentum with several major releases through the year.

https://github.com/SharePoint/PnP-JS-Core

Our whitepaper was finished in August, but took a while to come out as it passes though several reviews.  It was of great interest to me personally how many libraries we talked about had a new version while the whitepaper was being reviewed:  Angular went from 1.5/2.0rc5 to release.  JS Core went from 1.02 to 1.0.4.  Webpack released 2.0 end of the year, and SPFx went through 5 revisions.

http://www.telerik.com/campaigns/kendo-ui/sharepoint-framework-ui-customization-angular

We asked John Bristowe to come down and visit and recorded a podcast with Andrew Coates for the Dev Office Podcast (recorded in person and in studio!)

 

https://blogs.office.com/2016/12/15/episode-111-john-bristowe-john-liu-office-365-development-kendoui-angular2-office-365-developer-podcast/

We hit our Melbourne Cup with a major rewrite of our Melbourne Cup app (it was already in Angular, but now it is Webpack'd and deployed through Azure CDN - taking the actual download of the package from 10mb down to 2mb).

https://sharepointgurus.net/melbournecup/

We started experimenting with Azure Functions - first with Javascript/Node.  We wanted perhaps to get PnP JS Core to run in a Function.

http://johnliu.net/blog/2016/5/azure-functions-js-and-app-only-updates-to-sharepoint-online

We then looked into PnP PowerShell and saw how easy it was to run that within Azure Function - this will revolutionize IT Pros when we connect it with Flow.

http://johnliu.net/blog/2016/11/build-your-pnp-site-provisioning-with-powershell-in-azure-functions-and-run-it-from-flow

 

2017 - what's Your Stack

Looking towards 2017, we are marching ahead on both fronts - our Frontend work is backed by Angular, Webpack and Azure CDN, ready to go with SPFx.  Our backend work is backed by PnP PowerShell or JS Core and Azure Functions, ready to go with Flow.

I have an upcoming session in Australian Ignite on Azure Functions

https://msftignite.com.au/sessions/session-details/1988/serverless-in-office-365-build-services-with-azure-functions-

 

*We refers to me and various colleagues in SharePoint Gurus or otherwise.  Sometimes it might be just the royal 'we'.  Because we aren't consistent.

 

Running OfficeDev PnP cmdlets in 32bit AzureFunctions

Some recent changes with AzureFunction created a small problem - x64 functions are not available in the starter-consumption plans.

The situation is not game over.

Since all the assemblies are not compiled for x64 only.  So just open the psd1 file and change 'Amd64' to 'None'.  They actually run just fine.

 

Build your PnP Site Provisioning with PowerShell in Azure Functions and run it from Flow

What if I tell you - you can build your own Azure Function and use PnP to provision SharePoint Online sites without firing up Visual Studio?

What if I tell you - we will follow the best practices from PnP.  And You can do this all within 10 minutes (OK I timed it - if you don't take any breaks it'll be 10 minutes), and we'll do it with PnP PowerShell.

You will think - this is magic.  You will probably say, this is all crazy.

What if I tell you - you can connect this to the newly released Microsoft Flow.  So you don't even need to learn WebHooks (which is awesome and you should learn it).

What if I tell you - the whole thing will cost you a little bit.  It's not totally free.  It will cost you may be two cents.  ($0.02)

You will say, get out.  I need to lie down.

Learning Software pieces like LEGO blocks

I see software pieces as I see LEGO blocks.  PnP Provisioning is a block (a .NET library written in C#).  PowerShell cmdlets are a block.  And Azure Functions is a block.  Flow is a block.

What would happen if you connect them together?

 

Setting up PowerShell from PnP

First, this is how we set up in our local machine:

Install-Module SharePointPnPPowerShellOnline

Why PowerShell? 

Why PowerShell, you ask - John you are a developer.  You are a .NET developer before you became a JavaScript developer.  Why PowerShell suddenly?

The answer is simple.  It is because everybody understand this:

# connect to the SPWeb context
Connect-SPOnline -url "https://sharepointgurus365.sharepoint.com/sites/projects/template"
# export (copy) a template
Get-SPOProvisioningTemplate -force -out project-template.xml

# connect to new blank SPWeb
Connect-SPOnline -url "https://sharepointgurus365.sharepoint.com/sites/projects/tea"
# paste the template
Apply-SPOProvisioningTemplate -path project-template.xml

Four lines - you have just copied a template from /projects/template and stamp it over to /projects/tea - this is the power and the hardwork of the PnP provisioning team.  They make this magic look simple.

PnP was updated about a week after the blogpost and several PnP commands have been renamed to avoid confusion with SharePoint admin commands. See PnP-PowerShell documentations.

Connect-SPOnline is now Connect-PnPOnline
Get-SPOProvisioningTemplate is Get-PnPProvisioningTemplate 
etc

A way to verify the powershell is to run it first on your local machine and make sure it works first, before uploading the same DLL modules and script to Azure Function.

Special thanks to @pskelly

 

But you can do this in C#

Of course, you can also do this in C#, it looks like this:

This shouldn't be too scary to a developer.  But everyone sees the bubbles and the flow chart and just stop trying.  It "looks" hard.

Additionally, in CSOM - everything is asynchronous.  C# makes this simpler with the await syntax, but in the PowerShell cmdlets, every request is essentially serialized - so they are easy to understand, even if they aren't the fastest way. 

If you want to see what this looks like in NodeJS - I have several blogs.  It's great to learn.  If you don't like JavaScript to begin with - then you'll probably run to PowerShell with open arms.

OK enough chatter - let's go back to the demo.

 

Setting up Azure Functions

Create a new HTTP Request function - select PowerShell as the language.

It says Experimental - what they really should say is "The World's Not Ready For This".

PowerShell Azure Functions looks like this.  The request is passed in as raw JSON content - which is parsed into an object.

Output from the function by writing out to the $res file.

 

Install-Module

install-module doesn't work.  But we still can have modules.

http://stackoverflow.com/questions/37724769/how-to-install-a-powershell-module-in-an-azure-function/39985646#39985646  - The steps are explained by Azure Function team's @tohling

Essentially, we can take a PowerShell Module, find all its component script and DLL, and copy them to a modules/ subfolder within the Azure Function.  It will be loaded when the function needs to run.

In Functions - go to Kudu

Navigate to your new function's directory, and add a modules subfolder

In your PC:

Navigate to

C:\Program Files\WindowsPowerShell\Modules\SharePointPnPPowerShellOnline\2.8.1610.1\

(if you don't have this folder - you missed Install-Module SharePointPnPPowerShellOnline from earlier)

You will also need Microsoft.IdentityModel

C:\Windows\assembly\GAC_MSIL\Microsoft.IdentityModel
C:\Windows\Microsoft.NET\assembly\GAC_MSIL\Microsoft.IdentityModel.Extensions

Grab all these DLLs and PowerShell script and throw them all inside modules.

 

Change Azure Function to x64

One more thing.  The PowerShell cmdlets prefer x64 architecture.  Azure Functions provide 32 bit by default.  Switch this in function settings > application settings > Platform

 

Setup Done!

Now we can write fun functions!  Go back to our site provisioning function.

$requestBody = Get-Content $req -Raw | ConvertFrom-Json

# {
#     "source": "https://sharepointgurus365.sharepoint.com/sites/demo02/projects/pst",
#     "destination": "https://sharepointgurus365.sharepoint.com/sites/demo02/projects/tea"
# }

$source = $requestBody.source
$destination = $requestBody.destination

$secpasswd = ConvertTo-SecureString $env:SPO_P -AsPlainText -Force
$mycreds = New-Object System.Management.Automation.PSCredential ($env:SPO_U, $secpasswd)

Connect-SPOnline -url $source -Credentials $mycreds 
Get-SPOProvisioningTemplate -force -out "D:\home\site\wwwroot\project-template.xml"

Connect-SPOnline -url $destination -Credentials $mycreds 
Apply-SPOProvisioningTemplate -path "D:\home\site\wwwroot\project-template.xml"

Out-File -Encoding Ascii -FilePath $res -inputObject "Done $destination"

Note: PnP was updated about a week after the blogpost and several PnP commands have been renamed to avoid confusion with SharePoint admin commands. See PnP-PowerShell documentations.

Connect-SPOnline is now Connect-PnPOnline
Get-SPOProvisioningTemplate is Get-PnPProvisioningTemplate 
etc

A way to verify the powershell is to run it first on your local machine and make sure it works first, before uploading the same DLL modules and script to Azure Function.

Special thanks to @pskelly

The script will run unattended.  So you will need to provide your username/password stored in Function Settings and read from $env variables.

Run success!

What do we have here?!  PowerShell cmdlets that wrapped the PnP C# library and talks to SharePoint Online and everything running inside an Azure Function.

You have a service.

If you are counting lines - there is 10 lines of PowerShell - including the function wrapper.

 

Flow

Let's top off this demo with a Microsoft Flow - this was GA'ed just this week.

Create a list "Copy-Site" - with two fields "ToSite" and "FromSite"

Have the flow trigger off ItemAdded, and call our Azure Function via the Http Request.

Pass in source and destination.

 

Test Flow... workflow

Flow tells you what was sent to the HTTP Request, and what Response body it got back.  Use this for debugging your functions.

 

We always talk Costs because Azure Function is cheap

The 11cents total is over several days, and the breakdown shows it's almost entirely bandwidth costs - which doesn't have a free bucket.  My AzureFunctions don't exceed the allowed free tier, so I'm not paying for any compute time.

 

Where do we go from here?

What can you do with PowerShell?  Desired State Configuration?  Download lists to CSV?  Synchronize and fix metadata?  Locate orphaned or misplaced documents and fix that?  Improve and merge user profile properties?  Provision users and sites and what else do you fancy?

Does this give you wild ideas?

 

Azure Functions is Serverless

When we say Azure Functions is Serverless - it always seems we are saying we don't need servers (and thus don't need IT Pros).  This isn't strictly the truth - which is that the server is maintained by professionals, and developers could focus on writing code.

But after working on these PowerShell for the last two weeks - actually, I'm beginning to wonder whether there will be a day we don't need Developers.

An analyst says to a Bot - I want this System A to connect to this System B and put result in this System C.  The bot arranges three connections and suggest how the parameters will be connected.  Offers to perform a trial run.  Analyst checks it and approves it.  The code goes into production.

Think about this.  Everyone needs to evolve.  IT Pros - you have a huge opportunity here to use your PowerShell skills to provide huge business value.  Developers - you should use the best tools to enable your users and do more, this is really easy to get into.

 

Tell me what you think

I really want to know what you think - if you can spend a bit of time and leave a comment below.  Where are we going as developers, where are we going as IT Pros?  What kind of PowerShell-AzureFunction are you going to build?  Does PnP need a repository for PowerShell recipes?

It's so nice to see the pieces fit together well.

Building Sandbox Solutions without Code Assembly

Sandbox Solutions are still supported in SharePoint.  But the subtle distinction is that Sandbox Code Service is switched off.  So solutions with Sandbox Code will not activate.

Activation of solutions with sandboxed code has been disabled in this site collection.

But I argue that No-Code Sandbox Solutions is still the best way to build and package assets like JavaScript, images, CSS to be deployed to SharePoint - works in SP2010, SP2013, and SPO.

So the one trick is you have to build it excluding the DLL that it compiles.

TLDR:

OK the deeper details:

By default, the sandbox solution project include assembly in package.  So when you build the solution, it includes the DLL.

When you set the project to "not include assembly in project", then the result is now "SPO-sandbox friendly"

What if I don't have the project, I just have a WSP?

You really need to know the ins and outs of the sandbox solution - what's in it, and what does it do.  But the line of thought goes

  1. Open up the WSP file with a CAB utility.  Consider IZArc - you will need to extract the contents, make your changes, then package the folder structure back into a new CAB file.
  2. Delete the assembly reference in the solution manifest.xml
  3. Repackage the WSP file and pray. 
    There are a lot of reasons why this might not work - if your package has feature or event receivers or sandbox WebParts that actually uses the code which is now missing - things will fall over (hopefully, quickly). 
  4. Please test in a separate test environment, at least a different site collection on the tenant.

delete this

When re-packaging

  • delete the assembly xml section
  • don't include the DLL again in the new CAB file.

Good luck!

 

 

Access Denied when expanding ListView Grouping

A user with partial permissions to a list gets Access Denied when expanding ListView grouping.

This is a bizarre bug.

The setup

  • Take your basic SharePoint list, a few levels of grouping and lookups.
  • List items are separated in folders in this case, which then are security trimmed to groups
  • Not every member can see list items in each set
  • When you expand the ListView webpart, you get an Access Denied error.

A quick fix

Thinking this is something to do with Postback, I ticked this option:

Life's good again.  No idea why.  

Speculate away!  Let me know in the comments if this helps you in the future.

:-)