In my search for the one-command automated build, I’ve often stumbled upon how to deal with the database while writing code on a multi-developer project.

Most of us use source control, however only a small fraction of us actually store the database scripts in source control.  This is a powerful tool that is often missed completely.

This is a development process I see frequently:

• get the latest version of the source from VSS
• exclusively check-out the code files you need
• make changes to the code and the database to enable your new features
• check in the code once it is all done
• do a sql compare to the dev/test databases to get a delta script
• run the compare script in the test environment and deploy the project to test.

There are a number flaws in this process, but it doesn’t stop people from continuing along this path for an entire project.  Rather than point out everything that scares me from that list, I’ll focus on the database portion.

What’s wrong with just making changes and then doing a SQL compare?  Well, for starters… that compare script is used once and thrown away.  It is never checked into source control.  If we need to restore the state of an application as of 4 months ago, we can do it with the code, but not the database.  The database only stays on the current version, and there’s no going back.  Another drawback is that the process is a manual one, and we want to get into automation.  Lastly, SQL Compare (and SQL Delta) are both licensed tools, and not all of us can purchase them.  At home I cannot justify the cost, so I am forced to come up with something else.

The Ruby on Rails community is lucky.  Baked into the Rails framework is something called migrations.

Let’s say you’re on a ruby project for managing houses for sale.  You get a new feature request where you want to know how many bathrooms a house has (strange request, huh?).  You make the changes to model (adding a bathrooms property) but now you need the database to support it.  So you type rails script/generate migration Add_House_Bathrooms.  This will generate a ruby code file in the /project/db/migrations/xxx_Add_House_Bathrooms.rb file.  (the xxx will be replaced with sequential ordering, so rails knows what order to execute the migrations) Inside this file are 2 methods:  up and down.  This is, how to I create the changes, and how do I remove the changes.  Inside these methods you can write code that adds and removes the column you want.

So then when you need to deploy, rails knows what version the database is on (by having a version table), and can apply the migrations since that version to bring the database up to speed with the code.  If something goes wrong, it can roll the changes back to the original version (by using the down methods).

You can even create migrations that load dummy test data, or default values for static lookup tables.

It usually only takes a little while for a cool idea to make its way into the .NET world, and migrations are no exception.  The Castle Project is a collection of many RoR-esque utilities packaged into one.  One of those utilities is called Migrator, which lives in the Generator project.

To generate a migration, you type:

generate migration Add_House_Bathrooms

With this, you can enable this style of migration in .NET.  You create migrations, these end up as C# code files in your db/migrations folder.  The same concept applies.

The migrator is fairly young, but it was extracted from CastleContrib into the main Castle trunk, so that is a good sign that it is here to stay and will have active support.

When I have more time to play with this, I’ll try and post a little demo.  For now, check out the castle project at http://www.castleproject.org

What does your team do to manage database changes and enable versioning?

InfoQ has published a small book on DDD called Domain Driven Design Quickly.  They're giving away the PDF for free!  The print version is only $22, so I'll probably buy it if I like the PDF version.

Check it out:
http://www.infoq.com/minibooks/domain-driven-design-quickly

I came across this site today, which boats 53 CSS Techniques you can't live without.

Normally I don't like to blog just links, but this one is definitely worth it.  It covers most of the major techniques, both simple and advanced for styling web pages.

My latest assignment at work was somewhat small, and the requirements were fairly straightforward so I got to work immediately.  My early requests to meet with my user were met with “let’s get a demo first.”  Given the availability of all parties involved, this took longer than I would have liked.

So here I am, 60 work-hours later, I have a fully functional, ready-to-deploy application, and I have yet to meet with my user.  Let’s sincerely hope he doesn’t decide that this isn’t what he wanted in the first place.

Then my office-mate recognized our new development process.  Ladies and gentlement, I give you Eliga! 

Eliga is the opposite of agile.  Here is our manifesto:

Forget users, they will only slow me down!  Don’t disturb me when I’m coding you peons!  I’ll hear your impressions when I’m finished!  I can’t be bothered with workflow changes!  They’re already coded and set in stone!

Well the internet sure is a tiny place. I was tagged by JP Boodhoo and Aaron Murrell.

Here it goes, 5 things that you probably didn't know about me before:

• I was born on a farm in Oregon.  I still remember my favorite cow (Jesse) from our farm.  One day he was gone.  Where did he go?
• I used to be insane on rollerblades.  My friends and I would jump pretty much anything.  It’s actually amazing that I didn’t break any bones jumping over staircases and stuff, especially since I am so accident prone.
• I got my first guitar when I was 7, however I didn’t really pick up playing until 12 or so.  I still play, but not as often as I would like to.
• I first got into computer programming watching my dad write a hangman game from scratch.  I picked up QBASIC when I was 13 or 14, and I would write little screen savers using the primitive graphics drawing routines it had.  My pinnacle QBASIC program was a hoax that my friend and I pulled on his younger brother.  We called it “DelBug” and it was supposed to remove bugs from programs, such as Doom 2, which just-so-happened to be causing him troubles.  It went through a series of fake menus and ended up pretending to format his hard drive.  It was great!  He almost wet himself when it started flashing and rebooting in a loop.
• And lastly, I just got married!  (Pictures coming soon).  Silvia and I had a beautiful ceremony and lots of close friends and family were present.

And I’ll tag:

• Venkat Subramaniam
• Shey Sewani
• Tim Gifford
• Ben Day
• and my sweetheart Silvia 

 

I received a black Zune for Christmas, despite it’s lackluster sales and reviews by some.

I resisted getting an iPod for the longest time, because I wouldn’t buy into the hype.

I compared a number of features to decide between a Zune and an 30gb iPod Video.

Screen Size – The Zune has a very large screen compared to the iPod.  For video you turn it sideways for a widescreen view, and it makes watching movies very do-able.

Interface – Call me crazy, but I’m not too fond of the iPod interface.  When I used the iPod Video at the Apple Store, I was immediately annoyed that it asked me every time if I wanted TV on or TV off when I clicked on a video.  The TV wasn’t even connected!  For a company so anal about usability, making the users make a redundant and unnecessary choice before showing videos is a no-no.  The interface for the Zune is easy to use and the buttons make sense.  The iPod isn’t bad, I just don’t care for it very much.

Wireless – I originally thought this was one of the coolest features of the Zune.  It uses standard WiFi to transfer data between 2 Zunes, however it doesn’t connect to my wireless network at home, which is a disappointment.  I don’t know anybody that has a Zune, so I haven’t yet tried out the Social aspect of it.  There are hacks that allow you to transfer a file unlimited times between Zunes wirelessly, but it involves registry hacks on the host computer, and renaming the file as an image.  That makes it pretty much useless because I have a usb thumb drive in my pocket that can do the same thing easier.

Podcasts – I’ve become addicted to a number of podcasts, some with video, so I wanted good podcast support.  I was disappointed to find that the Zune has almost NO notion of podcasts.  Sure I can point the Zune software (which sucks, more on that later) to a folder where I download podcasts, but they don’t live in a separate section on the Zune.  I hope Microsoft fixes this in a future firmware update.  iPod / iTunes wins on this hands-down.  I’m currently using uTorrent for BT feeds, and Doppler for video/audio feeds.  These go into my podcasts folder which gets picked up by the Zune software.  The video has to be re-encoded to the Zune format, which is annoying.  I would think that WMV would be playable out of the box.

Video – I originally dismissed Video as excessive in a portable player, but now it’s my favorite feature.  I have a lot of downloaded tv shows, and I plan on putting them on the Zune to watch while I work out.  I ride an exercise bike to strengthen my leg and I can easily watch a TV show in that time.  The problem I see now, however, is that my shows are in Divx or Xvid format, which I have to re-encode using Any Video Converter to AVI or MPEG, then it has to be encoded again in the Zune format.  I’m hoping this process gets easier, or that the Zune will support Divx in the future.  One thing that would be nice is if I could encode video before I’m ready to sync.  I find it annoying that I have to surrender my Zune for at least 30 minutes if I need to transfer a couple diggnation episodes.

Music – This is a no-brainer for most MP3 players.  Both the iPod and Zune have excellent usage in terms of usability.  I like the large album art that is displayed during playback on the Zune, though I hear the new iPods do this as well.

Size – The iPod wins in this category.  The Zune is about 1/2” thick, which is quite large.  The iPod is about 1/4” thick, so it’s twice as thin.  I decided that the other features were worth the trade-off in Size.

PC Software – The Zune software sucks.  Big time.  It reminds me a lot of iTunes, but it doesn’t really deliver.  I ignore the Marketplace, so for me it’s just a mechanism to transfer files over.  I’d like to use MediaMonkey, but it’s not supported yet.  I found a few mistagged files while listening to music on the Zune, so I flagged them.  They showed up in my Zune “inbox” the next time I sync’ed.  From there it shows me the file, but it doesn’t let me edit the tags!  How stupid is that?  The only option I have is to delete the file and look at Properties, which is useless to me. 

The Zune software also has zero support for subscribing to podcasts, let along distinguishing my podcasts between regular music aside from Genre.

Audiobooks – I haven’t yet tried listening to an Audiobook, however I plan on it.  I listed to the Da Vinci code about 8 months ago and I loved it.  I’m hoping that it will be easy to listen to Audiobooks.

All in all, you can’t go wrong with an iPod or a Zune.  Both are fantastic devices, and I am very happy with my Zune.  I hope for a few improvements via firmware soon, but it’s a new device and I think Microsoft will catch on soon.

Now, the question is… do I attempt the 80gb mod?

Silvia’s family is in town, and it is a lot of fun to have so many kids in the house.  They (9 people) drove down from upstate New York.  It took them about 23 hours driving time and they were very tired when they arrived.

On Christmas Eve, we all spent about 4 hours wrapping gifts and talking over coffee.  It was fun, but we were all exhausted.

We had presents for all of the children, and it was quite a sight!  We probably had 80 presents under the tree.  The morning arrived and the kids went crazy!  There was wrapping paper everywhere, but I think everyone enjoyed their gifts.

I received a Zune from Silvia, which I will write about next.

Silvia’s family will stay for our wedding, which happens this Saturday.

We left off last time with our first association mapped and tested.  Let’s dig in a bit deeper with NHibernate mapping. 

Today I decided to separate my inherited classes into their own mapping file.  This was as easy as creating the new files and <hibernate-mapping> root elements, copying the <joined-subclass> elements over. 

I created the Item.hbm.xml:

and Video.hbm.xml:

In the video mapping, you can see how it inherits from the Item class.  If you recall back to where I explained how inheritance works, you can see it in action here.   Each video will have it’s Name and UPC stored in the Items table, and the Format and MovieId in the Videos table.  The Movie.hbm.xml is below:

This should give you an idea on how to map your entities.

I also did a little refactoring on the naming of my classes.  I didn’t like the naming of VideoGame because I thought it was better represented as a GameTitle.  I also renamed Transaction to RentalTransaction.  This will help avoid confusion when talking about database transactions.  I realize that not everything will be a rental (obviously most video stores also sell movies and candy, etc) but this name will suffice for now.  Since I am backed up by tests, I can refactor often and with complete confidence that I haven’t broken any old code.

So let’s revisit our feature list

• Add new Customer / Account
• Add other members to an account
• Restrict Certain members from renting certain content
• Query for a customer by customer # (swipe card, etc), phone number, or last name
• Add new rental item (move game, console, vcr, etc)
• Rent an item to a customer
• Check Items Back in
• Get movies checked out for a customer
• Query for an item, see who has it

Some of the remaining things on the list involve querying, so we’ll create a class to assist in that.  Listed below is the overview of a class called Repository.  

Repository takes advantage of generics to avoid a lot of duplicate code.  This is a quick implementation of the Repository pattern, you’ll find more in-depth implementations in Ayende Rahien’s Rhino Commons and Dave Donaldson’s NHibernate Repository.  The main point here is that we can abstract 90% of NHibernates functionality in an easy to use class.  If we need to do something complex, we can either expose some more methods here, or we can expose the session to the rest of your project.  This is a design choice that is really up to you.  I generally choose to hide NHibernate from UI developers, especially junior developers.

I provided some static members for just retrieving data, however to save data and participate in transactions, I created instance methods.  This class will serve most all of our data access needs.  It is the single communication point with our persistence layer to the presentation layer.

One side-note.  I have not written any tests for this class (shame on me!).  I find it difficult to test it without duplicating what I have already done with the integration/persistence tests.  I imagine that I could use a mocking framework like NMock or RhinoMocks, however this would distract from the series and frankly I am not very skilled at mocking.  If someone would like to inject a few words about this in the comments, I’d love to hear them.

Ok, back to our list.  I’ll tackle Restrict Certain Members from renting certain content.  This basically means that minors shouldn’t be able to rent rated R movies. 

The ratings for our items apply not to the items themselves, but to the Movie and GameTitle classes.  So to validate that the customer can rent one of these items, we must inspect the rating.

A naïve approach would be like this:

The bold portion is what I added to check for the rating.  This code should never see the light of day.  Each new type of item needs another cryptic if statement here.  A better solution is to use the Template Method design pattern.  We’ll leave it up to the item to setup a construct for verifying that rentals are allowed.

By default, we won’t restrict rentals.  In the item class, we’ll provide a virtual method that returns true always.  Inherited classes can make their own decisions by overriding the method.

In our Video class, we need to base our decision off of the Movie’s rating.

In our Game class, we need to base the decision on the GameTitle’s rating.

For another Item we might create, say Console, we can choose to accept the default behavior or write our own.  This will ensure that certain customers do not rent items that they are not allowed to rent.

I’ve added a few tests and ensure they are passing.

Now I’ll switch gears and look at the next item on the list, which is Can Query for Account by # (swipe card) or customer phone number.  This requirement will allow us to bring up the customer’s account when they are at the counter.  We would normally extend this requirement to add many more search options, however to be brief, we’ll just implement the two of them.

For this we can make excellent use of the Repository class we wrote earlier.  This test will involve the database, so we will place it in the Persistence test project.

 I want to keep the database related code out of the DomainModel class.  I created a class that can hold any of our queries for us.  This way we keep a clean separation between the model and persistence.  Remember, we cannot have a dll reference to the persistence project in the domain model project.  (This would lead to circular references – not to mention a bad design idea).

Here is the AccountFinder class:

Here I am allowing you to specify the session to use in the constructor. This way we can easily supply a session with a transaction (which will come in handy for the test).  In the first method we simply defer the call to Session.Get().  The second method takes advantage of HQL, or Hibernate Query Language.  This allows us to write queries easily, using familiar syntax, only we speak in terms of our objects.  You don’t see any database columns here.  HQL is very powerful, but can take some getting used to, because the results of your queries go directly into objects.  I’ve found that the best reference so far for HQL is Hibernate in Action book.

I created 2 tests to verify the above functionality:

And they are both passing!  We’ve come to a good stopping point here.  I’d like to wrap up our feature list next time, then start on a basic UI.

 

I’ve gotten a lot of positive feedback so far with this series, and I always welcome more!  If you haven’t downloaded the code to follow along I encourage you to do so.

 

Here is the latest source: Videocracy - Part 8

The very handy Firebug javascript debugger for Firefox has just released a major update.  Click here to get it.

If you aren’t using Firebug, you’re missing out.  It is hands-down the best javascript debugger out there.

It can intercept and log all XMLHttpRequests, you can set breakpoints, inspect and modify the DOM, execute javascript on the fly with the console, and a lot more.  New features allow you to tweak and visualize CSS metrics (with rulers), and monitor network activity.

Seriously, what are you waiting for?  Go get it.

After a very long spell of inactivity, we’re back for Part 7 of the series!

Last time we left off with a decent domain model, a database, and the beginnings of NHibernate.

Since the last article, the NHibernate team has produced the ever-so-fantastic v1.2.0 beta 1.  This release supports numerous new exciting features.  Topping the list of cool new features is:  native support for .NET Generics!  Native SQL 2005 is also supported.  Another feature was just enabled that will really help bridge the gap for those folks who will clutch to stored procedures forever:  Stored Procedure / Custom Query Support.

I’ll start today by slapping in this version of NHibernate and do a little bit of housekeeping.  With this new version of NHibernate, I no longer have a need for NHibernate.Generics (and thus I will remove it).  Some people might still use it, as it has the nice side-effect of automatically managing bi-directional relationships (which we’ll visit later), however I’m going to show this manually in this series.

I’ve changed my EntityList<T> / EntityRef<T> / EntitySet<T> to normal generic collections, and NHibernate will use them automatically.  These classes do provide you with an extra nifty feature (explained later), but it’s important to understand what it does under the hood, so I will implement the functionality the manual way, and later I’ll reference these classes to show how you might still want to use them.

So I’ve remove the old dll of NHibernate and I referenced in the new one.  I also updated my nhibernate schema file in Visual Studio so that I get the updated intellisense.  I compile, nothing is broken, and we’re ready to test!  The beauty of having our unit tests is that we have much more confidence that we know exactly what to fix, and we also know the instant that we are done fixing any breaking changes.  Our application returns to a known state of behavior very quickly.

Since I’ve made persistence level changes (the new NHibernate dll) as well as domain model changes (replacing NHibernate.Generics.Entity*<T> with standard .NET generic lists) I need to run both test assemblies.

 Here are the immediate results of the tests:

TestCase 'Flux88.Videocracy.Persistence.Tests.PersonTester.TestCanListPeople' failed: NHibernate.InvalidProxyTypeException : Type 'Flux88.Videocracy.DomainModel.Person' cannot be specified as proxy: method get_Id should be virtual
	c:\net\nhibernate-1.2.0.Beta1\nhibernate\src\NHibernate\Proxy\ProxyTypeValidator.cs(28,0): at NHibernate.Proxy.ProxyTypeValidator.Error(Type type, String reason)
	c:\net\nhibernate-1.2.0.Beta1\nhibernate\src\NHibernate\Proxy\ProxyTypeValidator.cs(76,0): at NHibernate.Proxy.ProxyTypeValidator.CheckMethodIsVirtual(Type type, MethodInfo method)
	c:\net\nhibernate-1.2.0.Beta1\nhibernate\src\NHibernate\Proxy\ProxyTypeValidator.cs(63,0): at NHibernate.Proxy.ProxyTypeValidator.CheckEveryPublicMemberIsVirtual(Type type)
	c:\net\nhibernate-1.2.0.Beta1\nhibernate\src\NHibernate\Proxy\ProxyTypeValidator.cs(22,0): at NHibernate.Proxy.ProxyTypeValidator.ValidateType(Type type)
	c:\net\nhibernate-1.2.0.Beta1\nhibernate\src\NHibernate\Cfg\Configuration.cs(915,0): at NHibernate.Cfg.Configuration.ValidateProxyInterface(PersistentClass persistentClass)
	c:\net\nhibernate-1.2.0.Beta1\nhibernate\src\NHibernate\Cfg\Configuration.cs(891,0): at NHibernate.Cfg.Configuration.Validate()
	c:\net\nhibernate-1.2.0.Beta1\nhibernate\src\NHibernate\Cfg\Configuration.cs(1035,0): at NHibernate.Cfg.Configuration.BuildSessionFactory()
	c:\sandbox\Videocracy\Flux88.Videocracy.Persistence\SessionSource.cs(55,0): at Flux88.Videocracy.Persistence.SessionSource.Initialize()
	c:\sandbox\Videocracy\Flux88.Videocracy.Persistence\SessionSource.cs(81,0): at Flux88.Videocracy.Persistence.SessionSource.GetSession()
	c:\sandbox\Videocracy\Flux88.Videocracy.Persistence.Tests\PersonTester.cs(27,0): at Flux88.Videocracy.Persistence.Tests.PersonTester.TestCanListPeople()

We have a few failing tests, however they all relate to the same problem.  NHibernate 1.2.0 beta 1 now sets everything to lazy-load by default.  This means that our objects must be able to be inherited so that NHibernate can inject proxy subclasses of our objects (the consumer doesn’t know or care that this happens).  To do this all of the methods and properties need to be virtual.   This is a large change, and it’s not something I entirely agree with, but it’s very easy to turn off.   (It pays to read the release notes of these 3rd party products!)  We’ll discuss how to effectively lazy-load a bit further on.

In our mapping file, we can specify default-lazy=”false” to override this behavior.  At a later time we may investigate and find that the class will benefit from lazy loading and we’ll revert it, but until then I’ll just make all the classes default-lazy=”false”.

Here’s the change:

Once that change is in place, all 5 of my persistence tests are passing.   I have 6 failing tests in my domain model, mostly due to my careless replacement of the EntityRef stuff with standard .NET collections.  The tests expose my oversights and once I fix them I’m back on track.

Ok, now where were we?  (It’s been a while, so I open NUnit and see that we have Save/List/Load for a Person object, but nothing else). We introduced some simple mapping concepts, and this time we’ll dig in a bit deeper.

How does NHibernate deal with associations?  Lets start with Account.  Account has a collection of Customer objects that represent its members.  In standard database terms, Account is a one-to-many with Customer.  How do we map this kind of relationship with NHibernate?

There are a few choices when looking at the documentation for NHibernate.  To understand them you must understand the semantics of each kind of list.  Ask yourself these questions:

• is it one-to-one?  many-to-one?  many-to-many?
• does order matter?
• is it bidirectional?

From the NHibernate FAQ, we find this informative nugget, that explains the collections available to us (and their .NET counterparts):

• The <list> maps directly to an IList.
• The <map> maps directly to an IDictionary.
• The <bag> maps to an IList. A <bag> does not completely comply with the IList interface because the Add() method is not guaranteed to return the correct index. An object can be added to a <bag> without initializing the IList. Make sure to either hide the IList from the consumers of your API or make it well documented.
• The <set> maps to an Iesi.Collections.ISet. That interface is part of the Iesi.Collections assembly distributed with NHibernate.

Our Account -> Customer relationship is one-to-many.  I could choose to allow one customer to have many accounts, but I don’t see any real use for that yet.  Order does not matter either.  I could have 4 members on an account, and it doesn’t really matter what order they are given to my in a list. 

We also have to consider what each end of the association looks like.  Right now I’m looking at it from the perspective of Account, where Account has many Customers (members).  On the customer object, the customer only has one Account, so this isn’t a bidirectional association.  (We’ll see an example of this later).  Techically our collection maps to an ISet, but I’d rather not depend on the Iesi.Collections.dll, so I’m going to map it to a list (the only drawback is that the list will allow the same member twice, so I’ll just get around this in our model manually).

Here’s how we’ll deal with that:

 The rest of the class is self-explanatory, so I won’t include it here.  You’ll  be able to download the project at the end of today’s article, however.

Next up is the mapping.  Let’s take another look at the table structure for reference:

This is slightly different than what was in the last archive.  Instead of an account having a single primary customer, any customer can be the ‘Owner’ of his/her account.  I chose to keep the association simple for this demo.  I also wanted to allow multiple ‘Owners’ for a single account.  I’ve updated the tests to reflect this change.

Here you see that many customers belong to exactly one account.    This is a one-to-many relationship from Account to Customer.

If we stop and think about the domain for a minute, we can determine that the Customer will be identified first (by his/her ID or Account #) and that needs to be able to pull up the account.  I’d also like to keep this relationship bidirectional.  That means that I need to map the relationship on both classes so that we can do this:

customer.Account ….  

or this:

for each Customer c in _account.Members { … }

Here is the relevant mapping for Customer:

This tells NHibernate that this is the *many* end of a one-to-many association.  We tell it the property name, type, foreign key column name, and our cascade strategy.  This basically means, whenever we save or update our Customer, cascade the save/update to the Account as well.  You can also specify to cascade deletes as well if you need to.

The Account mapping looks like this:

This section tells NHibernate to map this association as a bag.  Lazy=true means that the collection will only actually be retrieved from the database once it is accessed.  NHibernate does this by silently replacing our IList with a proxy that knows about the session and how to hydrate the collection.  This is 100% transparent to our consumers, so make sure you are using this in appropriate places.  Lazy=false will force NHibernate to pre-load the entire association (and it’s associations(and it’s associations(…. you get the point…))). 

To test that this association worked, I wrote a simple test:

This test is passing and we have our first association mapped!

I think this is a good stopping point.   We have migrated our version of NHibernate and our tests forced us to fix any outstanding issues before continuing.  We also identified our first association, mapped it, and tested it.

From here on out I will step up the pace and start glossing over the details a bit and focus on getting working product done using this domain model.  As I begin writing some UI components I will come across some features I have not developed, which may require more mapping changes and tests.  In order to finish this series sometime this year I’ll be speeding up a bit.  I will still continue to post the source so that you may follow along.  If you have comments or questions, feel free to comment.

Here is the latest source:

File Attachment: Videocracy_07.zip (44 KB)

I was very excited to hear the creation of the first Houston Agility User Group, Agile Houston.

If you are interested in Agile practices and methodologies and live in the Houston area, I encourage you to join the group and attend the first meeting!

Currently I expect the first meeting to occur sometime in January, so as soon as I find out a definite date, I’ll post it here.