Does Xbox's project Natal spell trouble for Nintendo Wii?

Just saw the videos for project Natal (http://www.xbox.com/en-US/live/projectnatal/) a new revolutionary video camera based controller for new and existing Xbox consoles. It is really great that to see that there is some competition in the new generation of console games. Even though Wii is still the true leader in this category it is refreshing to see that Microsoft will offer a comparable and even better gaming experience, while further developing its own video based technology.

As a father of a Pokémon fan, I still think that the real reason for the leadership of Wii remains in the engaging, long lasting and social games that Nintendo offers. These games have great stories behind them and a typical Pokémon game is played both on a small factor device, such as the DS(i), the Wii and most importantly on the playground with friends. A typical game/story takes months to complete. The social interaction between buddies playing the game (often using wireless) and the research a kid has to do to move to the next level using text guides, friends tips and the Internet is just so much more engaging than games with short span, such as sports games for example.

Another strong point of Nintendo is that they always have backward compatibility for their games for the previous generation. This protects the investment in games for many years. Microsoft on the other hand does not have a good record in this area with Windows and other products, so they should make it abundantly clear that after the planned retirement of Xbox, whatever comes next will have the capabilities to play older games and will be compatible with the Natal controller.

In addition to the usual sports games, judging by Stephen Spielberg's endorsement, I am guessing some of the first titles specifically designed for the new controller will be related to Clone Wars, which would be a really smart move for the game console trying to get some of Wii’s market share.

All in all this looks like a great product with tons of innovation, but I am still not certain I’ll be rushing to the store buying an Xbox… I just exhausted my gaming budget for a new Nintendo DSi, sorry…

 

Dovizhdane!

SharePoint developers get special treatment

In the last week or two in SharePoint land there were some interesting announcements and changes. The first on my list is the availability of Office Designer for free. While this product is not a spoon for everybody's mouth, it certainly makes it easier for a wider developer community to try out and use it and most importantly you can be productive in short time. This is great for many who are interested to create SharePoint applications, but it could be less fun for many administrators, since the lifecycle management of web applications with this version of the tool could be tricky. So watch out when you modify live sites. You can download Office Designer here.

Another good news is the simplification of the SharePoint server name for the next version of the product. We shall no longer have to use the long Microsoft Office SharePoint Server 2007, but just SharePoint Server 2010. The “Office” falls out of the name and becomes distinctive for the desktop product line, while the enterprise server product becomes just SharePoint Server or simply SharePoint. Not a big change for many, who already were using this convention. Some mild implication from this change result in losing  the somewhat beloved MOSS acronym, which was mixed in all kind of geeky combinations, such as mosslover, mossmozis, mossgonewild etc.  read more about this name change and other announcements on the Office product front here: http://www.microsoft.com/presspass/features/2009/Apr09/04-15Office2010.mspx

Last but not least I would like to mention some announcements about Visual Studio 2010. There are tons of new features that will continue to define this development environment as the best in the industry, but my favorite addition is the integrated support for SharePoint development. No more extensions that do the work half way. SharePoint developers become first class citizens with full support in Visual Studio. The fact that we can import WSP projects into Visual Studio makes me feel great. This means that we can finally take any project, even those developed using SharePoint UI or Office Designer, import them in Visual Studio and do things that developers do… put the files in source control system, make modifications and redeploy, rearrange features etc. Talking about features the new feature manager looks really promising, no need for directly manipulating XML files anymore. Other functionality currently available as community tool(http://www.codeplex.com/spm) is the integrated server browser . Many other server products had one (i.e. SQL Server), so why not SharePoint? To read more about what’s coming in Visual Studio 2010, have a look at the following announcements.

http://blogs.msdn.com/somasegar/archive/2009/02/19/sharepoint-tools-support-in-visual-studio.aspx

http://channel9.msdn.com/posts/VisualStudio/Sharepoint-Development-with-Visual-Studio-2010/

http://download.microsoft.com/download/C/0/9/C0965791-049B-4200-9008-F07A783026F6/VisualStudio2010_ProductOverview.pdf

Even though these changes came a little late for many, after all the voice of the SharePoint developer community has been heard. The platform will continue to grow and gain market share, but in addition to that this will continue the trend of replacing traditional ASP.NET/IIS development with SharePoint applications in many business scenarios.

 

Dovizhdane!

Interested to know the most popular SharePoint software development tool?

Take the survey posted by Todd Bleeker of Mindsharp here: Preferred SharePoint Development Tool. With no surprise for me with 44% at the time of writing this, the winner is WSPBuilder by Carsten Keutmann.

What is your favorite?

Starting New Year with a renewed MVP Award

Isn't it great to be a "January" MVP? You always get the news for the award first thing in the morning of January 1st. Perfectly in time for new year's resolution. I am pleased to announce that I was awarded the MVP Award for a second consecutive year. This is a great honor and validation of my work for the developer community.

However the most important aspect of the MVP award for me is that it reinforces my believe that static knowledge is useless. In the super connected world we live, obtaining knowledge and keeping it to yourself, either because of ill understood sense of job security or because of geekiness, or lack of communication skills, devalues this knowledge tremendously. With the fast pace of change nowadays, the window of opportunity to make use of your skill is rather limited. We create much more value for our companies, for the community and for ourselves, by sharing ideas, provide support to the new comers and expressing opinions about the future of the industry niche we specialize in.

This process provides tremendous opportunity for developers, who are not so fortunate and work for companies that either don't understand the value of continuing education, do not have the resources to do it, or simply are overworked. Several years ago I worked for several startups in a row and I recall that the lack of resources, the stressful work and the lack of social interactions with peers in the developer community really took a toll on my satisfaction on the job. Couple of years later, when the Florida code camps became big, I found that this is the perfect opportunity to share my knowledge with other developers in similar position. Even though now I am working for a company that takes really good care of it's developers, I find that the knowledge I exchanged in community in recent years is very valuable for me and for others.

Many ask me what is the MVP award. Many think it is some sort of skills certifications in the traditional certification terms, some simply reject it as a mark of the overachiever, and many savor it for what it is not. However in my opinion the MVP award is a "certification" of an open and outward attitude towards learning, sharing knowledge and building teams and communities. Some get in for assisting in the MSDN forums, other for writing books, third for preparing a presentation for a code camp or other community event, or may be sharing code on codeplex. Regardless of the form we all have one thing in common we keep our skills and knowledge in motion building new connections, ideas, friends and most importantly value.

Happy New Year!

Dovizhdane!

Caching of dynamic resources

It is a popular technique to store resources, such as CSS, XSLT, JavaScript and HTML files, in resources files embedded in an assembly. This approach makes solutions more self contained and makes localization easier using satellite assemblies. The processing of the resources is usually handled by a specific HTTP handler. Many times these resources are rather static and there is no need to reload them on each requests. For example a simple HTTP handler will look like this:

 

   1:  public class MyWebResourceHandler : System.Web.IHttpHandler

   2:  {

   3:        public bool IsReusable

   4:        {

   5:             get { return true; }

   6:        }

   7:        public void ProcessRequest(HttpContext context)

   8:        {

   9:             context.Response.ContentType = "contentType";

  10:             context.Response.Write("resourceContent"));

  11:             context.Response.End();

  12:        }

  13:  }

 

Recently I had to optimize the performance of such a HTTP Handler. It's purpose is to process a resource HTTP request, perform resource substitutions and other resource transformations and send the processed resource file back to the browser. Since this process was quite intense, it required regex and string operations, and the output rarely changed, it made sense to adjust the HTTP handler to analyze the cache information send by the browser and make the HTTP handler respond either with a complete response, sending the content of the resource, or with HTTP 304 - resource not changed.

To make this happen I added a check to determine whether the resources has been modified. This could be any condition you find appropriate. Based on this the handler either sends back the original content or responds with HTTP 304. To complete the response you have to specify the Content-Length HTTP header and set it to 0. To make sure that all caching information for this resource is correctly send to the client we have to specify the Cacheability, LastModified and Etag. All headers which control how the browser will request the cached resource at the next request.

 

   1:  public void ProcessRequest(HttpContext context)

   2:  {

   3:      if (IsFileModified(someParameters))

   4:      {

   5:          context.Response.ContentType = "ContentType";

   6:          context.Response.Write("resourceContent");

   7:      }

   8:      else

   9:      {

  10:          // File hasn't changed, so return HTTP 304 without retrieving the data 

  11:          context.Response.StatusCode = 304;

  12:          context.Response.StatusDescription = "Not Modified";

  13:   

  14:          // Explicitly set the Content-Length header so the client doesn't wait for

  15:          //  content but keeps the connection open for other requests 

  16:          context.Response.AddHeader("Content-Length", "0");                   

  17:      }

  18:   

  19:      // set cache info

  20:      context.Response.Cache.SetCacheability(HttpCacheability.Private);

  21:      context.Response.Cache.VaryByHeaders["If-Modified-Since"] = true;

  22:      context.Response.Cache.VaryByHeaders["If-None-Match"] = true;                  

  23:      context.Response.Cache.SetLastModified(creationDate);                

  24:      context.Response.Cache.SetETag(ETag);

  25:      context.Response.End();

  26:  }

 

Since resources are stored in an assembly, whenever the assembly changes we need to make sure that the resources get reloaded in the browser. Our file modification criteria will be the last modified date of the assembly, which contains the resources. To complete the handler we add two more methods IsFileModified and GetETag. IsFileModified checks if the ETag or the date modified have changed. GetETag generates a hash from the assembly's name and the last modified date.

The complete code of the handler can be found here: http://code.msdn.microsoft.com/ResourceCache

After applying these changes in our solution, we significantly reduced the traffic for downloading resources.

Now that we know how to deal with the cache I only need to find where the real cash is.

Dovizhdane!

 

PS: I would like to give credit to Dan Larson, who uses the same approach, but in a different context in his book Developing Service-Oriented AJAX Applications. Great book on AJAX in ASP.NET 3.5!

Encoding Optimizations for IE

Recently I had to tackle some performance issues in a set of AJAX web parts. We experienced unexpected slowdowns, which were increasing exponentially with the increase of the data being handled. After some investigation I found that the code is using the very popular base64 algorithm in JavaScript. This encoding is has the advantage that it has a counterpart in the ASP.NET server side and can be used to encode data send to and from the client when this is needed. It is a very popular algorithm and the JavaScript implementation can be found on many sites. Here is one of them: http://www.webtoolkit.info/javascript-base64.html.

It turns out that the string concatenations in this algorithm are killing IE and shooting the CPU utilization to 100%, locking up the UI. The situation progressively worsens as we go back to older browsers. In IE6 the performance is miserable. In IE7 it is better but still unacceptable. In IE8 BETA things looked much better, but still not in the range of performance in other browsers.

After poking around and getting some advice, Torben H. suggested that the string concatenation can be replaced by a simple buffer/array operation in JavaScript. This eliminates the recreation of the strings on each iteration of the encoding loop. In addition the performance of the modified algorithm increases in a linear progression when the amount of the encrypted characters grows. I also wrapped the algorithm in a separate JavaScript class, so it's use is a bit more intuitive.

You can see that on line 2, I create a buffer to hold the encoded string. On line 22 instead of using

output = output + this._keyStr.charAt(enc1) + this._keyStr.charAt(enc2) + this._keyStr.charAt(enc3) + this._keyStr.charAt(enc4);

I use buffer.push(...). Finally on line 25 the encoded string is returned as a join of the array. This is the complete code:

 

   1:  StringBase64Encoder.prototype.encode = function(to_encode) { 

   2:  var buffer = new Array(Math.round(to_encode.length / 3) + 1 * 4); 

   3:      var chr1, chr2, chr3; 

   4:      var enc1, enc2, enc3, enc4; 

   5:      var i = 0; 

   6:   

   7:      do { 

   8:          chr1 = to_encode.charCodeAt(i++); 

   9:          chr2 = to_encode.charCodeAt(i++); 

  10:          chr3 = to_encode.charCodeAt(i++); 

  11:   

  12:          enc1 = chr1 >> 2; 

  13:          enc2 = ((chr1 & 3) << 4) | (chr2 >> 4); 

  14:          enc3 = ((chr2 & 15) << 2) | (chr3 >> 6); 

  15:          enc4 = chr3 & 63; 

  16:   

  17:          if (isNaN(chr2)) { 

  18:              enc3 = enc4 = 64; 

  19:          } else if (isNaN(chr3)) { 

  20:              enc4 = 64; 

  21:          } 

  22:          buffer.push((this.key.charAt(enc1) + this.key.charAt(enc2)) + (this.key.charAt(enc3) + this.key.charAt(enc4))); 

  23:   

  24:      } while (i < to_encode.length); 

  25:      return buffer.join(""); 

  26:  } 

  27:   

  28:  StringBase64Encoder.prototype.decode = function(to_decode) { 

  29:      var chr1, chr2, chr3; 

  30:      var enc1, enc2, enc3, enc4; 

  31:      var i = 0; 

  32:      var buffer = new Array(Math.round(to_decode.length / 4) * 3); 

  33:   

  34:      // remove all characters that are not A-Z, a-z, 0-9, +, /, or = 

  35:      to_decode = to_decode.replace(/[^A-Za-z0-9\+\/\=]/g, ""); 

  36:   

  37:      do { 

  38:          enc1 = this.key.indexOf(to_decode.charAt(i++)); 

  39:          enc2 = this.key.indexOf(to_decode.charAt(i++)); 

  40:          enc3 = this.key.indexOf(to_decode.charAt(i++)); 

  41:          enc4 = this.key.indexOf(to_decode.charAt(i++)); 

  42:   

  43:          chr1 = (enc1 << 2) | (enc2 >> 4); 

  44:          chr2 = ((enc2 & 15) << 4) | (enc3 >> 2); 

  45:          chr3 = ((enc3 & 3) << 6) | enc4; 

  46:   

  47:          buffer.push(String.fromCharCode(chr1)); 

  48:   

  49:          if (enc3 != 64) { 

  50:              buffer.push(String.fromCharCode(chr2)); 

  51:          } 

  52:          if (enc4 != 64) { 

  53:              buffer.push(String.fromCharCode(chr3)); 

  54:          } 

  55:      } while (i < to_decode.length); 

  56:   

  57:      return buffer.join(""); 

  58:  } 

The results show a significant improvement in performance in both IE6 and IE7.

Performance in IE6

Algorithm\Chars	62917		125834		188751
	encode	decode	encode	decode	encode	decode
original base64 (seconds)	11.313	6.453	48.531	32.594	128.203	53.516
modified base64 (seconds)	0.547	1.203	1.437	3.906	4.703	8.078
XOR	0.719	0.703	2.125	2.141	4.015	4.016

Performance in IE7

Algorithm\Chars	62917		125834		188751
	encode	decode	encode	decode	encode	decode
original base64 (seconds)	2	1.672	7.359	5.25	21.172	9.36
modified base64 (seconds)	0.64	0.781	1.344	1.641	5.937	2.578
XOR	0.485	0.515	1.016	1.031	1.547	1.547

 

Just for fun I added an alternative encoding to the mix to see how it performs. This is the well know XOR algorithm, which is not as stable as the base64, because you can get unwanted characters in the encoded string, but it can be certainly used in AJAX applications. As you can see the performance is better for the XOR algorithm in IE7, but not really so much better in IE6. If your application is targeting older browsers there is no significant advantage in the performance of the XOR algorithm versus the base64. Here is the code I used for the XOR algorithm:

 

   1:  function StringXOREncoder(key) {

   2:      this.key = key;

   3:  }

   4:   

   5:  StringXOREncoder.prototype.encode = function(to_encode) {

   6:      var buffer = new Array(to_encode.length);

   7:      var i = 0;

   8:      do {

   9:          buffer.push(String.fromCharCode(this.key ^ to_encode.charCodeAt(i++)));

  10:      } while (i < to_encode.length);

  11:      return escape(buffer.join(""));

  12:  }

  13:   

  14:  StringXOREncoder.prototype.decode = function(to_decode) {

  15:      var rawinput = unescape(to_decode);

  16:      var buffer = new Array(rawinput.length);

  17:      var i = 0;

  18:      do {

  19:          buffer.push(String.fromCharCode(this.key ^ rawinput.charCodeAt(i++)));

  20:      } while (i < rawinput.length);

  21:      return buffer.join("");

  22:  }

 

Similar test in other browsers such as Firefox and Chrome did not have any issues with either the original or the modified algorithms. They handle JavaScript string operations much better. However most corporate environments are IE based and such optimizations may have dramatic impact on the performance of AJAX applications.

Happy encoding :)

 

Dovizhdane!

Saving WebPart Personalization Properties

If you tried to modify some properties of your web parts and save them from server code you probably noticed that there are some differences in the way this is handled in the SharePoint web parts namespace and in the ASP.NET web part namespace. Particularly if you are trying to save web part properties from code that is outside of the actual web part.

In SharePoint land the SPWebPartManager class has the SaveChanges(webpart) method, which allows us to save the properties of any web part on the page from any code location. However in the ASP.NET namespace this method is not available. Certainly one option is to switch to the SharePoint web parts namespace, however this is not recommended and prevents the web parts from being used in pure ASP.NET environment.

The only viable alternative I was able to find out is to expose the protected WebPart.SetPersonalizationDirty() in all my custom web parts:

   1:  public class CustomWebPart:WebPart

   2:  {

   3:  ...

   4:          public void SaveProperties()

   5:          {

   6:              base.SetPersonalizationDirty();

   7:          }

   8:  ...

   9:  }

Then I can make changes to my web parts' properties and save them from other page events, such a Page_Load etc.

   1:  WebPartManager wpm = WebPartManager.GetCurrentWebPartManager(Page);

   2:   

   3:  CustomWebPart wp = wpm.WebParts[0] as CustomWebPart;

   4:  wp.Title = "New name, new property";

   5:  wp.SaveProperties();

Seemingly easy, but for some reason took my colleague and me tons of times to figure out.

Dovizhdane!