Silverlight + SharePoint + CAML - best practices

I was reviewing http://microsoftpdc.com/Sessions/PR07 regarding SharePoint client object model tonight, in consideration of how the client object model should be used with Silverlight to build Silverlight SharePoint solutions.

I really liked slide 20, but I think it needs a bit more expanding:

• Use .Where method to:
  • Filter down items retrieved in a collection
• Use .Include method to:
  • Explicitly select properties or child objects to retrieve
  • You own specifying what you want!
  • (use .IncludeWithDefaultProperties for default + custom properties)
• Use .Take method to:
  • Restrict overall number of items retrieved

The object model provides (through LINQ query), these three ways of filtering down the objects that you want to return from SharePoint. 

There's a little bit more that wasn't mentioned - the gem (or curse, if you hate everything related to CAML…) that is

var query = new CamlQuery();
ListItemCollection items = list.GetItems(query);

Yes, CAML is far from dead.  And LINQ is not the answer to everything.

The basic issue is that the LINQ syntax is converted into CAML to be passed to SharePoint to execute on the server.  In particular:

• .Include - dictates the View.Fields that are returned from the service call

WARNING

• .Take
• .Where - these two may seem intuitive enough, but actually they are hidden pitfalls here.  The reason is that these parts of the LINQ statement are not translated to CAML.  So you are actually performing these operations on the client side, using essentially LINQ on an unfiltered objects collection. 

Bad performance, and quite possibly bad logic!

If you are really after paging behaviour - SharePoint 2010 has this built into the object model.  A skip and a take is done via:

CamlQuery camlQuery = new CamlQuery();
camlQuery.ViewXml =
                @"<View>
                    <ViewFields>
                      <FieldRef Name='Title'/>
                      <FieldRef Name='Category'/>
                      <FieldRef Name='Estimate'/>
                    </ViewFields>
                    <RowLimit>10</RowLimit>
                  </View>";

camlQuery.ListItemCollectionPosition = itemPosition; 

Skip is done via paging - see CamlQuery.ListItemCollectionPosition property.
Take is done via CAML <RowLimit>, or alternatively, you can do this via CamlQuery.RowLimit property.

I'll throw in one more, in CAML you can additionally perform Order By - this is one additional area where the order will affect the records returned by the query significantly - especially if you are using paging on top.

Guidance

In Silverlight, you should never query ListItems without some safeguards around the Query's RowLimit, as well as possibly handle paging.  On the server side, SharePoint will impose a limit of (default) 2000 items per query.

See more references here

I once read that CAML is compared to the butcher knife, and LINQ is the pocket carving knife (sorry forgot which blog I read this on - let me know).  I agree whole-heartedly: both are tools to control the slice of data you want from SharePoint, but you must keep performance in mind!

MCTS exams - what are these certification trying to solve exactly?

Just walked out of a MCTS exam, the one in question is actually 70-630, after 30minutes.  The exam was allocated 2hours.  I'm full of mixed feelings.  Relief, but highly puzzled and unsatisfied by the whole exam.

The format is a series of questions - of the contents I can't comment, but of the quality of the questions…  Imagine something like this:

You need to do something in SharePoint, what do you need to do?

• Fiddle with Active Directory and do A
• Play with SQL Server and do B
• Configure SharePoint and do C
• Run Exchange Server and do D

No surprises which is the right answer.  Even if you don't know jack about SharePoint.  Repeat these questions for half an hour.

I try to justify why such exams (and by association, certifications) are necessary.

1. It proves you have experience with the technology, or at least, can find your way around in that technology, and use it to Actually Solve Problems
2. It proves that you have a good overall understand of the technology and product, and that you can be relied on to Implement Solutions with the technology
3. It makes the companies sponsoring you to do the test happy that they've got another certified professional, which they can then tell their clients

I then, try to justify, and fail badly, at understanding why the questions were give away…  Pay the money for the examination, most of it is a tax write-off anyway.  No previous training or experience necessary.  Pass the test.  Pat on the back.

1. Perhaps…  and this I can somewhat understand, it's just hard to write "good but wrong" answers.  You start with a question, with an answer.  Then you try to put in the typical two way-off answers, and one really close answer.
2. Perhaps the problem is with the multiple-choice format.  Itself, which lends to the traditional 4-answer format.

Suggestion

1. Make the exam harder.  Make people sweat, make them unsure whether they should submit the exam, or pause and go back and review one more time
2. Or get rid of the multiple choice format.  How about this:

Each question's answer is made up of a sentence with dropdown boxes where you can select keywords.  Imagine something like this:

You need to do something in SharePoint, what do you need to do?

[Fiddle/Play/Configure/Run] [Active Direcotry/SQL Server/SharePoint/Exchange Server] and do [A/B/C/D]

(imagine [xxx] is a drop down box).  You can even give partial answers for getting "really close" to the "right answer" but no cigar.

FiddlePlayConfigureRun Active DirecotrySQL ServerSharePointExchange Server and do ABCD as well as EFGH

Outlook (and blogs) are for code

After being annoyed with Outlook... well, forever... for always autocorrecting my code in my emails, it suddenly strikes me to find the auto correct settings to just stop this once and for all.

So apologies in advance from now until the end of time - if my "quotes" actually be doublequotes and my - (minus or hyphen) actually be a minus and not a dash.

Both are very important in being able to correctly paste code, I figure my human recipients could live with less than perfect typography ;-)

Turn off:

1. "Straight quotes" with "smart quotes"
2. "Hyphens (--) with dash (-)

In Outlook

in Windows Live Writer

This is something extremely simple, and I wonder I couldn't be the only person that has done this.  I can't understand why it took me 10 years... of pressing Undo right after pasting code into an email…  Goes to show you that most people never tweak the out of the box settings.

Windows 7 Troubleshooting Wizard is not a myth

I know this sounds crazy.   The network troubleshooting wizard actually works.

So I was on a designer’s machine and she was really having trouble with her wireless card – it’s reporting an IP Address conflict.

We figured that a simple:

ipconfig /release
ipconfig /renew

Should do the trick, but it was skipping the wireless adapter.  Confused – we stumbled across the wizard and clicked “Windows Network Diagnostics”.

To our amazement it asks us if we want it to fix the network for us. 

Behold!

Also explained why it wasn’t working before.

Certified Scrum Developer

John Sonmez made some comments that scrum had become all about money in his post:
http://simpleprogrammer.com/2010/03/31/scrum-for-the-money/
I highly disagree.

Before I go any further, continuing the thought of honesty, here’s my disclaimer:
I have participated in Certified Scrum Developer (.NET) series, and I’m extremely proud of the quality of the course, so take my views with a grain of salt. 
I am also, first and foremost, a .NET zealot – that means I value highly the technology stack from Microsoft as well as maximizing the ROI on the tools available to me. 
Though I will gladly confess that scrum is awesome at helping me get things done (ship products).

Confusion over costs

I honestly thinks that John Sonmez is mostly upset over the costs, which probably surprised him as greedy and blood thirsty.  But in the article he’s mixing up a great number of different costs involved – for very different things.

• Training to be CSM (~3 days)
  • $1195
  • Intended for: Individuals who will work on a Scrum team. The course has a strong emphasis on the role of the ScrumMaster.
• Training to be CSD (~5 days)
  • Start from CSM with additional electives, costs will vary, but I’m going to guess at least $2,500+
  • Existing trainers can offer electives to allow a CSM to extend to CSD
  • Existing CSM wanting to work with a particular toolset can add appropriate electives
• Training to be PSD (5 days / .NET)
  • $3995 – for 5 days hands on training
  • Entire training is packed and planned as a single continuous course.
  • There is high emphasis on the 3 pillars and how these form a synergy: Scrum, software development, and the tools (VS.NET 2010, TFS 2010)
  • Intended for: Individuals who will work on a Scrum team.  The course has a strong emphasis on the software developer team member, using Microsoft’s .NET tools
• Training to be PSD (Java)
  • N/A
• Training to be PSM
  • N/A

Confusion over Assessment

Most “training” includes one attempt at the certification.  If the trainee fails the assessment, then they can retake a second certification at following cost.

I think both SA and Scrum.org do not allow you to retake a third time if you failed the second retake.  You’ll need to repeat the training again.  So it’s not like Microsoft Certified Professional (MCP) where you can just brute force exams until you pass.

• Certification CSM
  • $150
• Certification CSD
  • $150
• Certification PSD
  • $100
• Certification PSM
  • $500
• Certified Scrum in depth
  • $2000
  • Required to be a CSM trainer or PSD trainer

Learning should be free?

I think there’s a big difference over what one considers to be fair price.  At the minimum we’re looking at $1195 for the 3 day CSM training.

Honestly speaking - one does not need to be certified to practice scrum. 

Most of the materials are freely available online.  But please consider that these materials are developed and paid for by other people who are certified and delivering training, and by organizations such as SA and Scrum.org that’s promoting Scrum.

Troubles without formal training? 

1. There are an incredible amount of Scrum Practitioners that aren’t practicing scrum at all.
2. You may end up with Scrum-but. 
3. You may encounter issues in your project or organization, and you don’t have the knowledge or the backup support to keep going with scrum. 
4. You most likely will end up making your own ‘agile’ thing.

So here we are again, back at square one.  Do you need the training or don’t you need training?  Are you happy with how scrum works in your organization.  Or do you think you are only saying that you practice scrum, when in reality you are doing nothing that even remotely looks like scrum?

For the individual – there is intrinsic value by being trained.  Becoming a member of the SA / Scrum.Org groups also give you additional backup and support discussion.

For the company – the value is having you certified.  They can justify your training costs for tax write-off.  They can increase your rates if they get you to work for their client.  They can tell their stakeholders – hey we got a super professional now that’s taking care of our projects and look at our awesome burn down charts – our velocity is flying through the roof!

Certification should be cheap?

This one I really don’t know.  I know you can’t have it for free – since then everyone will be a certified scrum master and that will definitely degrade the quality of the entire certification programme.

Then at the minimum, there is an ongoing charge for membership fee to SA / Scrum.org – think of this as tithe for being a valued member of the community.  You can also see it as a support contract.  Either way, you are supporting them to keep on preaching Scrum, so you don’t have to.

Finally, to become certified requires both training, then assessment.  This isn’t about walking into some test centre and brute-forcing your way to certified.  No, you need to be trained by an authorized trainer, and then your training is assessed via a centralized exam issued by SA / Scrum.org

So you are juggling between people who see the value of being trained, assessed and certified, or people who cough at seeing any price tag.  I think the line lies somewhere in between, and given the success of the CSM programme I’d say it is just about right.

The case for Professional Scrum Developer

In the case of PSD – the course is still at its first steps with a lot of materials being crammed into a very short space of time.  I think the $4000 price tag is fully justified at the amount of training, knowledge and hands-on that you will receive at the 5 day training – believe me… we all thought the course at 5 days was too short for the number of topics available that we wanted to get through.  (At the minimum we wanted 2 more sprints)

• Scrum fundamentals
  • Scrum teams
  • Define your done criteria
  • Scrum hands-on practice (5-6 short sprints)
• TFS 2010 templates for Scrum
  • User Stories, work items in TFS
  • Burn down charts in TFS
  • Build Server
• Using VS.NET 2010 to work in a scrum team
  • Refactoring
  • ASP.NET MVC project
• Using VS.NET 2010 Test Manager to conduct tests
  • Creating unit tests
  • Fixing bugs in iterations
• Thorough Scrum reviews and retrospectives

Yes – all the materials is probably somewhere on the Internet and hidden in people’s blogs.  But to produce this 5 day course work requires months of preparation.

No doubt as the months move forward and the course material settles down the costs may fluctuate to better reflect market prices.  But I think trainers are currently already absorbing a lot of the costs to keep the initial offering at least affordable.

Microsoft wants, I guess, five thousand PSD (.NET) in the market before VS.NET 2010 is launched (and by the way it’s already launched).  If they are lucky, they probably may get to 500.  A very respectable number, but far short of the number they wanted.

Sum it all up by more links

• http://courses.scrum.org/classes/oceania
• http://www.scrumalliance.org/training
• http://msdn.microsoft.com/en-us/vstudio/