May 05, 2015

SharePoint Add-in: Accessing Webcam with Only Javascript

May 05, 2015/ John Liu

This blog post details how to access your Webcam via Javascript through the browser, and upload that content to a SharePoint library. Then, with an added bonus, set it to be your User Profile Picture.

I have build this User Profile Webcam solution as an Add-in for SharePoint (was App for SharePoint). This is a SharePoint Hosted app. All the code runs in the browser and access SharePoint via the SharePoint Online API.

Step 1. Access your webcam.

Modern Browsers: Immersive Internet Explorer, Firefox, Chrome and Safari all has ways to access your webcam via

getUserMedia

But this doesn't work on IE (Desktop). So for that we'll add a Polyfill (Flash).

There's a project on github that does this nicely, and I use large chunks of the code from their demo.

https://github.com/addyosmani/getUserMedia.js

<script type="text/javascript" src="../Scripts/html5.js"></script>
<script type="text/javascript" src="../Scripts/getUserMedia.min.js"></script>
<!-- Add your JavaScript to the following file -->
<script type="text/javascript" src="../Scripts/App.js"></script>


<div id="webcam"></div>
<canvas id="canvas" height="240" width="320"></canvas>

<br />
<button class="btn" style="width:140px;" id="takeSnapshot">Take a picture</button>

this.snapshotBtn = document.getElementById('takeSnapshot');
$(this.snapshotBtn).click(this.getSnapshot);

this.getSnapshot = function () {
    // If the current context is WebRTC/getUserMedia (something
    // passed back from the shim to avoid doing further feature
    // detection), we handle getting video/images for our canvas 
    // from our HTML5 <video> element.
    if (App.options.context === 'webrtc') {
        var video = document.getElementsByTagName('video')[0];
        //App.canvas.width = video.videoWidth;
        //App.canvas.height = video.videoHeight;
        App.canvas.getContext('2d').drawImage(video, 0, 0, App.canvas.width, App.canvas.height);

        // Otherwise, if the context is Flash, we ask the shim to
        // directly call window.webcam, where our shim is located
        // and ask it to capture for us.
    } else if (App.options.context === 'flash') {
        window.webcam.capture();
        App.changeFilter();
    }
    else {
        alert('No context was supplied to getSnapshot()');
        return false;
    }
}

The code basically takes either the webrtc object and copy the image to the App canvas. Or use the flash polyfill and Flash will write the image to the canvas element.

Step 2. Binary Format

This part is hairy. But you are a developer, and dealing with Binary Encoding in JavaScript is what we do.

/*
Now, get canvas and do decoding.
1. canvas can return image data in png, in dataURL format.
2. strip the heading string "data:image/png;base64,"
*/

var ctx = App.canvas.getContext('2d');
var imageData = ctx.getImageData(0, 0, App.canvas.width, App.canvas.height);
var dataURL = App.canvas.toDataURL('image/png');
var base64 = dataURL.replace(/^data:image\/png;base64,/, "");

This is base64 dataUrl format.

/*
Reading.
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Base64_encoding_and_decoding
https://developer.mozilla.org/en-US/docs/Web/API/WindowBase64/Base64_encoding_and_decoding#Solution_.232_.E2.80.93_rewriting_atob%28%29_and_btoa%28%29_using_TypedArrays_and_UTF-8

We need to convert DataURL to ByteArray
Mozilla has these two functions.
*/

function b64ToUint6(nChr) {
    return nChr > 64 && nChr < 91 ?
        nChr - 65
      : nChr > 96 && nChr < 123 ?
        nChr - 71
      : nChr > 47 && nChr < 58 ?
        nChr + 4
      : nChr === 43 ?
        62
      : nChr === 47 ?
        63
      :
        0;
}

function base64DecToArr(sBase64, nBlocksSize) {
    var
      sB64Enc = sBase64.replace(/[^A-Za-z0-9\+\/]/g, ""), nInLen = sB64Enc.length,
      nOutLen = nBlocksSize ? Math.ceil((nInLen * 3 + 1 >> 2) / nBlocksSize) * nBlocksSize : nInLen * 3 + 1 >> 2, taBytes = new Uint8Array(nOutLen);

    for (var nMod3, nMod4, nUint24 = 0, nOutIdx = 0, nInIdx = 0; nInIdx < nInLen; nInIdx++) {
        nMod4 = nInIdx & 3;
        nUint24 |= b64ToUint6(sB64Enc.charCodeAt(nInIdx)) << 6 * (3 - nMod4);
        if (nMod4 === 3 || nInLen - nInIdx === 1) {
            for (nMod3 = 0; nMod3 < 3 && nOutIdx < nOutLen; nMod3++, nOutIdx++) {
                taBytes[nOutIdx] = nUint24 >>> (16 >>> nMod3 & 24) & 255;
            }
            nUint24 = 0;
        }
    }
    return taBytes;
}

var byteArray = base64DecToArr(base64);

This gives us ByteArray. We're getting close.

I'd like to tell you that I spend a good 3 evenings working to this point. Try debugging that code and see if your eyes bleed!

/*

Finally, 

The ByteArray needs to be encoded as a string, and in the POST call, "binaryStringRequestBody" needs to be "true"

https://msdn.microsoft.com/en-us/library/office/dn769086.aspx
http://blogs.msdn.com/b/uksharepoint/archive/2013/04/20/uploading-files-using-the-rest-api-and-client-side-techniques.aspx
http://sharepoint.stackexchange.com/questions/54218/sharepoint-2013-rest-api-upload-image

*/

var binaryString = '';
var len = byteArray.byteLength;
for (var i = 0; i < len; i++) {
    binaryString += String.fromCharCode(byteArray[i])
}

SharePoint API wants data in BinaryString format. Which is basically each byte of the ByteArray encoded as a concatenated string.

Step 3. Upload to SharePoint

/*
Use RequestExecutor to post file back to sharepoint
*/

var appWebUrl = context.get_url();
var requestExecutor = new SP.RequestExecutor(appWebUrl);

var uploadPictureEndPoint = appWebUrl + "/_api/web/lists/getByTitle(@TargetLibrary)/RootFolder/Files/add(url=@TargetFileName,overwrite='true')?" +
    "&@TargetLibrary='" + "Pictures" + "'" +
    "&@TargetFileName='" + _spPageContextInfo.userLoginName + ".png" + "'";

/*
Using RequestExecutor, don't need REQUESTDIGEST
*/
//var digest = $("#__REQUESTDIGEST").val();

requestExecutor.executeAsync({
    url: uploadPictureEndPoint,
    method: "POST",
    headers: {
        "Accept": "application/json;odata=verbose"
    },
    contentType: "application/json;odata=verbose",
    binaryStringRequestBody: true,  // binaryStringRequestBody must be true
    body: binaryString,
    success: function (x, y, z) {
        alert("Success! Your file was uploaded to SharePoint.");
    },
    error: function (x, y, z) {
        alert("Oooooops... it looks like something went wrong uploading your file.");
    }
});

Step 4. Setting User Profile Picture.

var appWebUrl = context.get_url();
var requestExecutor = new SP.RequestExecutor(appWebUrl);
var setPictureEndpoint = appWebUrl + "/_api/sp.userprofiles.peoplemanager/setmyprofilepicture";

requestExecutor.executeAsync({
    url: setPictureEndpoint,
    method: "POST",
    headers: {
        "Accept": "application/json;odata=verbose"
    },
    contentType: "application/json;odata=verbose",
    binaryStringRequestBody: true,
    body: App.binary,
    success: function (data) {
        alert('Set My Profile Picture succeeded, it will take a few seconds for the change to be propagated.');
    },
    error: function (error) {
        alert("Oooooops... it looks like something went wrong updating your profile picture (no permission?).");
    }
});

For this API call to /setmyprofilepicture to succeed, your App must have additional permissions.

This will ask the user (or Tenant Administrator) to grant the correct permission.

If you don't grant this permission, the webcam can still save picture to library, but it won't be able to set picture as user profile picture.

Summary

Get Webcam via Browser
Process canvas data to imageUrl to byteArray to BinaryString
Upload to SharePoint Library
Set as User Profile Picture
Achieve Zoolander face!
You can download the App for free from the Office Store to see it in action.
Feel free as a developer to hit F12 and step through the code.
Leave a comment below and let me know what you think. For example, I think there's a need for an Outlook Add-in that uses the webcam. Especially, if it can run on an iPad.

April 22, 2015

InfoPath Javascript - fixing image control tooltip.

April 22, 2015/ John Liu

In a previous blog post I discussed using Picture Controls to host lots of images within InfoPath and manage the tooltip.

InfoPath FormServer renders picture buttons with a pop-up Picture Icon that will hide any alt-text that you've set on that picture.

The key is in the Core.js file, which you should NOT modify. As this is what could happen if you modify core javascript files.

function LinkedPicture_OnMouseOver(a, c) {
    ULSrLq:;
    var b = ViewDataNode_GetViewDataNodeFromHtml(a),
        d = PictureControl_GetPrimaryDataNodeValue(b);

    /* This modification hides the picture icon when the control is disabled */
    var e = PictureControl_GetPicture(a);
    if (e && e.alt == "Picture") {
        e.alt = "";  e.title = "";
    }
    if (typeof(a.disabled) != "undefined" && a.disabled) {
        return;
    }
    /* end modification */

    a = PictureControl_EnsureControl(a);
    d != "" && PictureControl_ShowPictureIcon(a, b, true);
    LeafControl_OnMouseOver(a, c);
}
LinkedPicture.OnMouseOver = LinkedPicture_OnMouseOver;

So instead, you should create a separate webpart page, put the Form control on that page, and add this script override for the LinkedPicture_OnMouseOver function.

The key is "if (a.disabled) return"

Then in your InfoPath form template, set the control to read-only will ensure the Picture icon doesn't appear on hover, allowing the title text on your image to show up on the browser.

April 11, 2015

App for SharePoint: Web Cam

April 11, 2015/ John Liu

This is a Sway post. It is a short demo of an upcoming discussion on:

App for SharePoint
Web Cam
HTML5
And SharePoint Libraries

April 11, 2015

[Meta] Updating johnliu.net from Squarespace v5 to v7

April 11, 2015/ John Liu

This is actually something really, really long overdue. v5 is from the year 2008. That is SEVEN years ago.

You will notice:

Responsive Design
Comments are replaced with Disqus - the original comments are imported, but Squarespace's own comment spam detection has not been great for me.
Squarespace v5 renders pages in X-COMPAT-IE8 which means it looked extra sucky.

You will find missing:

Some of my game blog pages - I've removed them because they are just the rambling of a young man.
/Storage is a bit funny - I "hope" I imported my old stuff properly.
My badges and tag cloud. I hope to add them back in time.
I noticed the RSS feeds got confused and prepended www.johnliu.net - I've removed the www but there may be duplicate records in the RSS reader.

What I am missing:

I am absolutely devastated that v7 doesn't work with Windows Live Writer.

March 26, 2015

Azure Logic Apps: Build SharePoint Workflows by clicking buttons: a picture guide

March 26, 2015/ John Liu

TOC: Azure Logic Apps

Build SharePoint Workflows by clicking buttons [This article]
- Introduction
- SharePoint Online
- Office 365
- Connect them all
Hybrid Workflows - SharePoint On-Premise
"Code", Template Language Expression
Observations
Social
XML

Introduction - What are Azure Logic Apps

Microsoft announced a series of Azure App Services today:

http://weblogs.asp.net/scottgu/announcing-the-new-azure-app-service

Specifically, I want to focus on Azure Logic Apps.

http://azure.microsoft.com/en-us/documentation/articles/app-service-logic-what-are-logic-apps/

In Microsoft's words.

Azure App Service is a fully managed Platform as a Service (PaaS) offering ... allow any technical user or developer to automate business process execution via an easy to use visual designer.

Best of all, Logic Apps can be combined with API apps and Connectors from our Marketplace to help solve even tricky integration scenarios with ease.

Microsoft has a tutorial on how to create Azure Logic Apps:

http://azure.microsoft.com/en-us/documentation/articles/app-service-logic-create-a-logic-app/

http://channel9.msdn.com/Shows/Azure-Friday/Azure-App-Service-Logic-Apps-with-Josh-Twist

You should go through these first. There are a number of new Azure templates that are wordy to describe, but a video will show how it all works together fairly quickly.

So I only want to focus on the SharePoint Online Connectors. They are easy to set up, but actually, tricky to find.

Create the Logic App

The Workflow:

Grab tweets from my twitter timeline and put them into my SharePoint Online List, then email me.

Once you watched the video and we start by creating our own Azure Logic App.

Once it's ready, head into Triggers and Actions - this is where the rules are defined.

On the right hand side you wouldn't have any API Apps in the resource group. We'll configure them in a minute. Click Visit the Marketplace

Configure the Office 365 Connector

There is a giant header at the top to add Office 365 Connector. We should go ahead and add that.

Note, in your happiness to add the Office 365 connector, you will, like me, completely fail to read "send and receive emails, calendar and contacts". No files, or SharePoint sites. Essentially, this connector is only for Exchange-related services.

Still useful for sending emails, so let's configure it.

Read this too: http://azure.microsoft.com/en-us/documentation/articles/app-service-logic-connector-office365/
You will be seeing the "Creating..." box a lot!
I had a look inside once it is created - there isn't much to see, or change.
You'll notice we haven't authenticate the API App - we'll do that later.

Go back to Triggers and Actions on the Logic App

Sometimes - the API Apps are listed by their template name and not the API App Name that you've assigned. This is actually very confusing, I'm sure it'll be fixed soon.
Adding either the Office 365 Connector or the SharePoint Online Connector will also add the HTTP and Recurrence API Apps automatically.

Configure the SharePoint Online Connector

http://azure.microsoft.com/en-us/documentation/articles/app-service-logic-connector-sharepoint/

So far so good. Let's do the next one. And I think you'll fall into another hole.

You look at this and you say, surely. That one is for SharePoint, since the earlier Office 365 connector isn't.

Yes, I say. That's exactly what I want. Those are the right triggers and actions!

And here is the next hole I fell in.

Chances are, you got all the way to the end and thought why do I need a service bus for Azure Logic App to talk to Office 365. Something don't smell right.

I have failed to read the text again - this one is for On-Premises SharePoint, and you will need to install a listener proxy that will talk to Azure Logic Apps via an Azure Service Bus.

We'll cover that later when we talk about Hybrids.

For now, go back to the Marketplace.

Use the search filter. There are 2 SharePoint connectors. The SharePoint Online Connector isn't shown in the default view.

Create -> Package Settings -> OK <- Create

TIP: Unpin

While you wait for the API App to be created, your Startboard is currently looking like a mess. Right click on the other connectors and unpin from Startboard.

Add the Twitter Connector

This is in the video above. So I'll jump through this one really quick.

More spinning. All done! OK now we have all our connectors. Let's look at them.

Triggers

I want to show the various different triggers from different connectors. But I won't actually use these in the later example.


Office 365 Trigger looks like this.	SharePoint Online Trigger looks like this	Twitter Trigger looks like this	The trigger I want to use is a simple Recurrence timer. It will run every hour.

Activities

Next, pull down some tweets.

Add Twitter Connector and Authorize

Twitter Authorize, and set up to grab my timeline.

I want to put them into SharePoint Online.

Click Authorize - you'll see a pop up

I want this to talk to my Office 365 Work Account - so pick the top one.
Note - sometimes, it doesn't seem to work, I would close the pop up and try Authorize again. I consider this another Preview Bug

You should see these actions to perform on the connector. Notice it connects to two lists specified during the setup, and there doesn't seem to be a way to change that afterwards.

Pick Insert Into tasks

We'll need to loop through the tweets and insert each one.