Google recently released a URL API for charting.

What's a URL API, you ask?  Well you open an image to a specially formatted url, and google returns you a chart.  You can do some pretty amazing things with this API.

Check out these samples:
       

The last one was created with this url:
http://chart.apis.google.com/chart?cht=p3&chd=s:Uf9a&chs=200x100&chl=A|B|C|D

The urls are a little bit cryptic as you can tell, but this allows you to use one source for charting regardless of your language or platform.  A .NET wrapper is sure to spring up in the next couple days that will allow you to take advantage of this API with a more structured format.

Update:  Boy did I underestimate the community.  Chris Pietschmann already created an ASP.NET 2.0 Server Control to leverage the Google Chart API in .NET.

I'm wondering how they are able to handle all of this.  Not only is it a lot of traffic to handle (but they're obviously pros at that by now) but it takes a significant amount of CPU cycles to generate a graph like that on the fly.  How do they keep their response times so quick?
 

I’m currently assisting with another project that is experiencing some performance issues.  A lot of finger-pointing and blaming has been going on, and most of the parties involved just want to jump in and blame NHibernate.

I figured that we could optimize the way NHibernate is being used to help gain some perf. back in the application and hopefully restore its reputation at the client (and frankly, my company). 

Some info about the application:

It’s a WinForms application that communicates via Web Services to a service layer.  The service layer utilizes NHibernate over a domain model for persistence.  The domain types are serialized and used directly on the client.  (more on this later)

There are roughly 100 tables and almost as many mapped entities.  It is using NHibernate 1.0.2.

We did some basic benchmarking of the application so that we could quantify how effective various fixes were.

Startup time: 14 seconds.  Ouch!  I turned on the hibernate.show_sql setting in the config file and watched the SQL queries fly.  They were covering the screen!  I wrote a simple log4net IAppender that would count the number of log statements that contained a query and found out that 304 queries were being executed at startup.  Obviously something is wrong here, so I copied the queries to notepad and began to group them.

The domain of the application is shipping (like, in ships across the ocean).  So there is a concept of Terminals, Ports, Port Locations, and Geographic Locations.  If we load all of the terminals, we need to load the port for each one, the port location for each one of those, and the geographic location for each one of those.  It turns out that this many-to-one is not that bad.  You can get most of the data you need in one query using left joins.  The problem we faced was that GeographicLocations held an IList of all of its PortLocations.  And furthermore the PortLocation held an IList of all of its Ports. 

Picture trying to build up just a single terminal…

• select terminal details and build up a Terminal
• need a port, so build it up
• need a port location, so build that up
• need a geographic location, so build that up
• not done?  Now we need all of the port locations for the geographic location
• you guess it, now we need all of the ports for all of the portlocations you just loaded
• now we need to get all of the terminals for all of the ports for all of the port locations, ….

This is starting to sound like that children’s story The House That Jack Built.

This is EXACTLY the reason that ORM’s have lazy-loading.  If you haven’t used it before, here’s how it works:  You load up the terminal, but it doesn’t fire a query for the port.  It gives you a stand-in object instead.  The second you access the terminal.Port property, *boom* – a database query is executed, the port is loaded, and you are returned a port.  This allows us to select a minimal amount of data, and is generally favorable unless you know ahead of time that you’ll need the port, in which case it’s faster to just query for it all at once.  Needless to say, you need to at least think about this issue while developing.

Lesson #1:  Understand Lazy Loading Or Be Doomed

But remember that our domain entities are being returned from a web service.  There is no way for NHibernate to open a database connection on the client.  Thus, we cannot utilize lazy loading at all.

I decided to remove the IList of port locations from the GeographicLocation class.  As it turns out, there was no code using this method, so it was a simple change.  I removed the list, removed the relevant mapping xml, and rebuilt.

The queries were reduced to 94.  That 5–minute change improved the application load time to about 6 seconds.  This is not the end of the story on improving startup time, but this was a big win at a very low cost.

There are likely other areas like this where we can break collections, but I probably will be breaking client code, and I will have to resort to adding a service method and utilizing NHibernate queries to provide the objects in a collection, rather than object graph traversal.

Lesson #2:  Don’t go crazy with your colleciton associations.  Possibly favor querying over object traversal for problem areas.

As a side-note to lesson 2, read Eric Evans’ Domain Driven Design for a much better understanding of aggregates, aggregate roots, and how to partition your domain model.

Then I started looking at the logs.  These were turned off in development, so nodody ever saw this.  I noticed that there were a ton of messages like “CodeDOM failed for class …..   unknown character `” — this was the reflection optimizer trying to compile some code on the fly to assist with getting/setting property values.  The code was listed along with the error and you could clearly see that this happened only on types that had a generic parameter.  I searched around and found that this was a bug in 1.0.2 that didn’t understand how to read the string representation of generic types.  It was fixed in 1.2.0, so I decided to upgrade. 

The upgraded version reduced the startup time to 3 seconds, and I’m pretty confident that we won’t get much better than that.  Sure we can reduce the number of queries a bit more, but we’re getting into the realm of acceptable values here.

How did we go from 6 seconds to 3 seconds?  Well the reflection optimizer had previously been turned on and was failing… once for every entitiy.  This caused an exception and easily slowed down the startup time.  Additionally, the reflection optimizer wasn’t helping out getting and settting property values so they fell back on reflection, whcih can contribute to overal runtime performance degredation.  The reflection optimizer is FAST, so let it do it’s job!  (see how fast in this article by Jay Chapman)

Lesson #3: Turn on logging (even in Dev!)

Lesson #4:  Take advantage of the reflection optimizer!

Next we took a look at one of the trouble screens and discussed how to make it faster.  The screen needed a lot of different types of data.  It had details at the top of the page, most of which were dropdown selections loaded by another entity, and the grid needed data from 3 different entities, flattened out.

The issue here is that the UI needs SO many different entities, that a signification portion of the entities are being loaded when only a subset of the data is actually being displayed.  This is an excellent opportunity for applying Screen Bound DTO’s.

A screen-bound DTO is a custom type that contains a flattened view of only the data I need to fulfill the needs of the screen.  I can utilize Projections to create slim entities that only contain name/value pairs for example.  I can accomplish this all with HQL.

This means less queries being sent to SQL server, and less data going over the wire.

Lesson #5:  Consider Screen-bound DTO’s instead of consuming your entities on the UI

On the database side, we can probably deal with a more friendly transaction isolation strategy, such as ReadCommitted.  This will allow more concurrent reads than other isolation levels.  This is accomplished through configuration.

Lesson #6:  Consider using ReadCommitted as your isolation level

We should also configure the default_schema setting to be databasename.dbo because that will be used to fully qualify the objects in all database queries.  Without this, SQL Server will not cache the query plan for these queries.

Lesson #7:  Always make sure you’ve set hibernate.default_schema

There are still more optimizations that we might make, by analyzing opportunities for cachine on both server side and the client, tweaking the mappings, and loading large data sets on a background thread to keep the UI responsive.

I believe that we will be able to achieve very acceptable performance with a lot of analysis and a tweak here and there.  Maybe then we can restore some faith in NHibernate, as it is a truly powerful persistence framework and it would be a shame to yank it because it was implemented poorly.

An excellent resource is Billy McCafferty’s article on NHibernate Best Practices with ASP.NET.  Be sure and read it if you haven’t already (it’s full of useful advanced tips on NHibernate.

Are there any perf. considerations I’ve missed?

Enjoy:

http://asp.net/downloads/3.5-extensions/

Also, be sure and check out the brand-new open source project http://mvccontrib.org for all of the extra stuff that ASP.NET will not ship with.

Time to play!

Looks like we can expect the ASP.NET MVC CTP to be released in about a week.

Great stuff!  Check out some of the other major things that are coming out of Microsoft in the coming months at ScottGu’s blog.

A few weeks ago, Jeff Atwood sent me a free review copy of his new book, The ASP.NET 2.0 Anthology.  He co-authored the book with K. Scott Allen (from OdeToCode), Wyatt Barnett, Jon Galloway, and Phil Haack.

Normally I wouldn't get excited about a book like this, but looking at that list of authors I was more than intrigued to give it a try.  Granted, the book is targeted for the beginner to intermediate ASP.NET developer, so a lot of it was not very helpful to me, but I did find quite a few good nuggets that I will take away and utilize in the future.

The first few chapters give a general overview of the newer features of .NET 2.0 and some other basic concepts.  I think that this would be helpful to someone just coming to ASP.NET (maybe from another language) and want to get started. 

The book was published by SitePoint, under their "Anthology" style.  This means the book is organized into 101 different "tips" phrased as questions that someone might ask on a forum or similar.  Some examples from Chapter 1 are "How do I use source control?" and "How do I go about using open source code?" -- I was pleased to see these sections in the book.  You normally wouldn't expect this and I think it helped make the book more well-rounded.

The middle chapters were a bit more targeted to the intermediate ASP.NET level, which made it more interesting for me.  They covered things like custom configuration sections, custom validation, how to maintain control state when viewstate is turned off.  All throughout I think the authors gave solid advice on all these features.  There was a good section on Membership and Access Control, which is sure to be helpful to intermediate ASP.NET developers.

The last few chapters were the most interesting to me, which is funny because most of them weren't specific to ASP.NET.  A section on troubleshooting bothersome SQL queries is sure to help people out.  There were a good number of general web developer tips, such as considering SEO and clean HTML for good search engine rankings.  I appreciated the tip on how to utilize URL rewriting at the IIS level, and how to serve-up custom images to evil hotlinkers on myspace (thanks Jeff Jon!).  These types of things are great to know when pursuing a career as a web developer.  The last chapter introduced SubSonic, which I think was awesome.  Not many developers that I meet have heard of SubSonic and I find it to be a pretty useful tool with a low barrier of entry.  It definitely warranted some attention in this book.

While a lot of the content was your typical ASP.NET 2.0 fodder that I avoid (like SqlDataSource), I thought that the authors did a good job of noting when things were applicable, for example when to use the Web Site model vs. Web Application Projects.  The book didn't have enough beginner content and explanation of ASP.NET to be a beginner book (which is a good thing), but it did have quite a lot of excellent tips that makes this book a great companion to any intermediate developer.

A few weeks ago, Evan Hoff posted a screencast on building up a domain model in a test-first manner.   We talked about it and thought it might be interesting to show how to apply persistence to this already functioning and tested domain model.

So take a look and tell me what you think!  It weighs in at 40 minutes (I tried to keep it under 30 but hey).

I was one of the early adopters for the Zune.  My biggest attraction to it was the nice screen, which is great for watching movies on the busride to work.



Well now that the new Zune 2.0's are out, Microsoft released a much needed software update, both on the PC software side and the Zune firmware.  If you have a 1st generation Zune, then go to http://www.zune.net and grab the latest software update!  It's worth it!

Some new features:

• wireless sync
  
• support for podcasts (finally)  -- both in the PC software and the Zune itself
• updated UI
  

So I decided to start that NHibernate Mapping DSL project and open-source it.  Right now it doesn't do anything, but you're free to take a look at http://www.assembla.com/space/nhibernate-mapping

I wrote all of this code test-first, and it allowed me to tackle the interface and flow the way I thought might be useful.  I ended up with something that was mildly usable like this:

IMapping mapping = Mapping.For<Product>()
   .Identity("Id")
   .Property("Name")
   .Property("Price")
   .Map();

So this allows me to quickly accept the defaults of the mappings, so NHibernate will assume that I have ...

• the column names are the same as the property names
  
• the type of the column matches the type of the property and it can be picked up via reflection
• the identity generator of the primary key is Identity
• the identity/properties have setters
• the "Name" and "Price" columns are nullable

This isn't always true of course, so we need to be able to specify where we deviate from the defaults.  So how can we accomplish this using a fluent interface?  We could resort to adding more overloads to the Identity() method, so you might have....

IMapping mapping = Mapping.For<Product>()
   .Identity("Id", "product_id", Access.NoSetterCamelcaseUnderscore, Generator.Native)
   .Property("Name", "product_name", false)
   .Property("Price", false)
   .Map();

But we don't know how many settings the user might want to specify.  If we have 4 simple, independent settings, then we'd have to support 4! method overload options!  Yuck!

public IPropertySpecificationPredicate Property(string name);
public IPropertySpecificationPredicate Property(string name, string columnName);
public IPropertySpecificationPredicate Property(string name, string columnName, DbType type);
public IPropertySpecificationPredicate Property(string name, string columnName, DbType type, int length);
public IPropertySpecificationPredicate Property(string name, string columnName, bool nullable);
...

This is just the beginning.  What if you wanted to only specify the property name and the nullability?  We quickly end up in an anti-pattern I like to call overload explosion. 

We can do a bit better, right?  I started to gather my thoughts and came up with this syntax:

IMapping mapping = Mapping.For<Product>()
      .Identity("Id")
      .Column("product_id")
      .Generator(Generator.Native)
      .Map();

So now the Identity() method takes a single parameter, the name of the property.  Anything else is optional.  The return type is now something like an IIdentityBuilder interface, which accepts methods on altering the identity object that its building.  Here you can see that we only specified the settings that we wanted, nothing else.  We still had to have a way to "pop" the current object (and thus the entire mapping) so that we can indicate to the builder that we are done building the property and you can return the IMapping object next.  This feels awkward for now, but for now it's ok.

Let's extend this pattern.  Once I'm working with the IMappingBuilder interface, I can now see methods called Generator(), Access(), Column(), etc.  There's nothing there to prevent me from calling .Column("...").Column("...").Column("...").Column("...") which I guess isn't such a big deal.  All it's doing is setting the column property of this object we're building.

Now I'm noticing that there has to be a spot where the user gets finished with the mapping identity and now has some choices.  The obvious next operation would be to create a property, but how do I pop the current context and start working on a new one?

Maybe something like this?

IMapping mapping = Mapping.For<Product>()
      .Identity("Id")
      .Column("product_id")
      .Generator(Generator.Native)
      .AndProperty("Name")
      .Nullable(false)
      .Map();

I'm already started to notice that my identity and property elements are getting lost in the noise.  If I keep up this pattern not only do I have TONS of ISomethingPredicateBuilder objects, I also remember that it's not only properties that I can add at any time.  I could add a Bag, a Set, a List, or any number of other mapping elements.

Another route I might take is using anonymous to provide me with the flexibility of code within the interface.  Take a look:

IMapping mapping = Mapping.For<Product>()
      .Identity("Id", delegate(Identity i) {
                         i.column = "product_id";
                         i.generator = Generator.Native;
                      })      
      .AndProperty("Name", delegate(Property p) {
                         p.column = "product_name";
                         p.nullable = false;
                      })
      .Map();

Here I have a lot more flexibility over the different ways that I define various mapping elements.  The only part of this that is really cumbersome is the delegate syntax itself.

This is probably where Boo or Ruby would come in and save the day because of it's super-dynamicness-flexibility-extraordinaire, but I'd like to see how far I can push C# for now.  At least until C# 3.0 comes out, where I'll get object initializers, lamba expressions, and a few other neat tricks that really make this stuff more fun to sculpt.

What about you?  What types of syntax do you find most readable, usable?

I often talk to people who haven't worked with one of the cool javascript frameworks out there.  I am partial to prototype and scripaculous, but I also really like MochiKit.

If you aren't taking advantage of one of these, then perhaps this post is for you... I give you the ultra quickstart to diving into prototype and scriptaculous!

Things you shouldn't ever program javascript without

The best thing ever invented since rubber tires: a replacement for document.getElementById('control_id');

You know you're not supposed to use document.all or document.ctrl_id, right?  Well at least now you do. document.getElementById() is the cross-browser safe way of getting an element off of the page. The prototype equivalent is $('control_id'). This will automatically extend the element returned with some helpful utility methods (like extension methods in C# 3.0). -- more on this later.

Along the same lines is a useful method for selecting elements using CSS selectors. If you're a CSS junkie (like me!) you'll appreciate this.    

$$("div span.info");

This will return all of the DOM elements that would be matched by that CSS Selector. (for those who don't know, this would get all span tags with a class of "info" that are directly nested underneath a div.

Want to show or hide an element? Use Element.hide(ctrl), Element.show(ctrl), or Element.toggle(ctrl). You can attach these methods onto the element itself by saying Element.extend(ctrl). If you retrieved it via the $() method, then your object already has these methods.

Let's say you want to grab all of the span tags from a parent div tag.  Typically you'd do this:

var spans = parentDiv.getElementsByTagName("span");

but what is spans?  If you thought it was an array, you'd be wrong.  It's a NodeList, which is not nearly as powerful as a javascript array.  (With javascript arrays you can push(), pop(), shift(), and other cool things that you can't do with a NodeList.  Anyway, back to the spans collection, we can elevate this to an array by using the $A( spans ) method.  Now we can use all of the nice array goodness for that list.

Effects for that extra polish

How about fading an element?

new Effect.fade(elem);

This also takes options so you can control how fast it fades, for example. Check the excellent docs for all the gory details.

Want to make a slide out box?

new Effect.toggle(elem, "slide");

This will toggle the element using slide up and slide down. ("fade" and "appear" also work here for toggling).

How about some ajax?

Forget UpdatePanels. Most of the time we just need to fire off a request and get a list of values back from the server. Prototype has you covered there:

new Ajax.Request( "/path_to_server/ajax/AjaxHandler.ashx?op=getUsername&value=bob",
                  {
                      method: "get",
                      onSuccesss: function(result) {
                                     if(result.responseText) alert('available!); else alert('taken');                    
                                  },
                      onFailure: function(e) { alert('something went wrong ' + e); }
                  }

I don't know about you, but that's pretty painless to me.

Want something a bit richer?  How about an autocomplete textbox? I wrote this one to auto-complete cities as you type.

var url = '~/AjaxHandler.ashx?op=get_cities';

ac = new Ajax.Autocompleter(city_textbox_id, results_id, url, 
{
  paramName: 'filter',
  frequency: 0.2,
  minChars: 1,
  indicator: indicator_id,
  callback : function(element, filter) { 
                var state = $(state_dropdown_id);
                var state_code = state.options[state.selectedIndex].value;

                return filter + '&state=' + state_code;
  }
});

That's it. I'm not kidding. I handed it a textbox, a div to display the results, and a url to get the data. It takes care of all of the ajax requests, the frequency limiter to prevent us from firing a request with every single keystroke, and returning the results into a list. It even has full keyboard support. The url looks like ~/AjaxHandler.ashx?op=get_citiees&state=TX&filter=Hou and it will return the matches that it finds (based on an HttpHandler I wrote).

What are your favorite prototype / scriptaculous protiops?