James Kovacs' Weblog

DevTeach is heading back to Toronto in a few weeks (March 8-12, 2010)and you’ll get a bigger dose of awesome than ever before. We’ve got a fantastic line-up of top-notch, internationally renowned speakers. 6 tracks covering Agile, Web, Windows, Silverlight, Architecture, and SharePoint. A metric ton of sessions. (I’m both the Agile and Web Track Chairs and am really excited about the speakers and sessions for each.)

Microsoft Canada is a platinum sponsor and every attendee receives a full copy of Visual Studio Professional with MSDN Premium. (N.B. Conference registration costs less than this subscription alone!)

And if you can’t get enough of that Sugar Crisp James Kovacs,  I’ll be there in full force with two sessions and a one-day post-con on agile development.

Convention-over-Configuration in an Agile World

As developers, we spend an inordinate amount of time writing “glue code”. We write code to transform database rows to domain objects… domain objects to view-models or DTOs… We write code to configure inversion of control containers and wire dependencies together. We write code to style our UIs and respond to UI events. Wouldn’t it be nice if this could happen automagically for us? This session will look at using convention-based approaches using Fluent NHibernate and Castle Windsor to reduce the amount of repetitive code and accelerate application development.

Convention-over-Configuration in a Web World

As developers, we spend an inordinate amount of time writing “glue code”. We write code to transform database rows to domain objects… domain objects to view-models or DTOs… We write code to configure inversion of control containers and wire dependencies together. We write code to style our UIs and respond to UI events. Wouldn’t it be nice if this could happen automagically for us? This session will look at using convention-based approaches using AutoMapper and jQuery to reduce the amount of repetitive code and accelerate application development.

Agile Development with IoC and ORM (Post-Con)

As developers we now have powerful tools in our toolbox, such inversion of control containers and object-relational mappers. But how can we use these tools to rapidly build maintainable and flexible applications? In this pre-con, we will look at advanced techniques such as convention-over-configuration in IoC containers and automapping ORMs to quickly build applications that can evolve over time. We will use test-driven development (TDD) to design and evolve a complete working application with supporting infrastructure during this one-day workshop.

Hope to see you in Toronto!

A few months back, I announced that I was doing a series of articles for MSDN Magazine on improving a “classic” ASP.NET application with modern tooling and frameworks. As an application, I chose ScrewTurn Wiki 3.0 to use as my example throughout. The first article, Extreme ASP.NET Makeover – Getting Your House in Order, went live a few days ago. The article is purposefully a different format for MSDN Magazine than “traditional” articles in that it incorporates short screencasts where appropriate rather than just code snippets and pictures. (Code snippets and pictures are included too, though!) I tried to make the screencasts an integral part of the narrative where actually showing something was easier than text, pictures, or code. I would love to hear your feedback on the format and content.

Nitpickers Corner: In the series, I use MSBuild as the build tool. Yes, I wrote my own PowerShell-based build tool, psake. Yes, I use NAnt on many of my projects for clients. (They’re already using NAnt and PowerShell is a new skillset for them.) So why MSBuild for the series? Because it is installed by default with .NET 2.0 and above. Not my first choice, but a pragmatic choice for a series focused on improving what you have.

My first dnrTV episode went live today. I am talking with Carl Franklin about dependency inversion, dependency injection, and inversion of control. I demonstrate how to build a very simple IoC container. My intent is to show developers that it isn’t any thing crazy scary. I talk about how IoC facilitates decoupling dependencies and creating more easily testable software. Check it out!

dnrTV #126: James Kovacs’ roll-your-own IoC container

Feedback is always welcome.

Carl and I plan to do another show focused on IoC containers in the next few weeks. Specifically we’ll be talking about what a full-fledged container offers over and above the roll-your-own. If you have other IoC questions you would like answered on the next show, drop me an email.

During my geekSpeak screencast last week, one of the attendees asked:

Any recommendations for refactoring existing code to insert interfaces? (e.g., what’s the best dependency to break first, the database?)

Excellent question! Most of us do not have the luxury of working on greenfield projects, but instead work on brownfield projects – existing applications that could use some tender loving care. Brownfield projects are often inflicted with legacy code. What do I mean by legacy code? I agree with Michael Feathers’ definition:

Legacy code is simple code without tests.

Michael elaborates further saying:

Code without tests is bad code. It doesn’t matter how well written it is; it doesn’t matter how pretty or object-oriented or well-encapsulated it is. With tests, we can change the behavior of our code quickly and verifiably. Without them, we really don’t know if our code is getting better or worse.

If you haven’t read Michael’s book, Working Effectively with Legacy Code, you really must. It is on my short list of must-read development books. Michael provides excellent strategies and patterns for safely implementing tests around untested code, which is crucial in order to make non-breaking changes to an existing application AKA refactoring. If you can refactor your code, you can improve your code.

Now back to the question at hand… Which dependency to break first? Let me turn the question around. Which dependency is causing you the most pain?

In my experience, it is typically the database as round-tripping to a database on each test to fetch or save data dramatically slows the tests down. If you have slow tests, you’ll be unlikely to run them as often and then the tests start to lose their value as a safety net. (N.B. You still want integration tests that access the database. You just don’t want each and every unit test to do so.) As well it requires a lot of effort to keep consistent data for tests, often using test data setup scripts or rolling back transactions at the end of tests.

Other areas that often cause pain are integration points – web services, DCOM/Enterprise Services, external text files, … Anywhere your application is relying on an external application. Integration points are problems for tests because if you’re relying on them, your tests will fail when the external applications are unavailable due to crashes, service outages, integration point upgrades, external network failures, behaviour changes, … Imagine that your e-commerce website integrates with 6 external systems (credit card processor, credit check, inventory, sales, address verification, and shipping). Your development environment integrates with DEV/QA versions of each of these services. Each service has 95% uptime, which translates into 1.5 days of downtime a month for maintenance, upgrades, and unexpected outages. The chance of all systems being available is the product of their availabilities or 95%*95%*95%*95%*95%*95%=73.5% uptime for all 6 integration points. If your tests directly use these test systems, your test suite will fail over 25% of the time for reasons beyond your control. Now is that because you introduced a breaking change or because one of your integration points is temporarily unavailable or misbehaving? Life gets worse when you integrate with more systems or when the availability of those systems is lower. Imagine you have to integrate with 12 integration points with 95% availability – your test suite will only pass 54% of the time. Or if your 6 test systems only have 90% availability, your test suite only passes 53% of the time. In each case, it’s a coin toss whether you know for certain whether the change you just made broke the application. When you start getting a lot of false negatives (failing tests when the problem is in an integration point), you stop trusting your tests and you’re essentially flying without a parachute.
By decoupling yourself from your integration points by using interfaces for the majority of your tests, you know that the code base is healthy and you can separately test the interactions with your integration points.

To figure out which dependency to break first, take a critical look at your codebase. What is causing you the most pain? Pain can manifest itself as long debug sessions, waiting for external services to be available, high bug counts, … Solve the pain by decoupling yourself from that dependency. Then look at what is now causing you the most pain and solve that. Lather, rinse, repeat. It might take a few weeks or months, but eventually you’ll have a codebase that is a pleasure to work with. If you don’t believe me, read this email that I received from Karl (originally posted here):

Hi James,

I’m writing you because I just wanted to thank you!

It was about two months ago and I attended TechEd/Orlando. I have to say that it was the first time for me and it really was an honor to be the one and only chosen from approximately 300 software developers working in my company to go to Orlando. I was very impressed about the good quality of the sessions, especially on the architecture track, but your tiny little discussion on Tuesday evening really opened my mind. At that time I had been working as a software developer for 7 years with about 5 years experience in software design and project management and 5 years of .NET experience. I was fighting with a 400,000 LOC .NET legacy monster that’s used by public safety agencies around the world in security related areas. I have a team of 12 developers and we were trying really hard to keep this beast up and running and extend it with new features. I think you can imagine that most of the time we were using the trial and error approach to get new features in (just hack it in and prepare for long debugging sessions hunting weird bugs in parts of the application you never expected to be affected by the new feature…). The main problem was – guess what – the dependencies. There were about 50 classes (all singleton “Managers”), and every single manager was tied to at least 10 other managers – no interfaces at all – and that’s just one of several layers of the app… We knew that dependencies were our problem, but we had no clue how to solve it – there was this chicken/egg problem – I want to decouple my system, which needs a lot of refactoring. To ensure that I don’t break anything I’d need unit tests but I can’t use them because my system is so highly coupled We have tried TypeMock, but my feeling was that this went in the wrong direction. Needless to say that this attempt failed.

During the discussion after your session you gave me some quite useful hints:

1. Read Applying Domain Driven Design and Patterns by Jimmy Nilsson
2. Read Working Effectively with Legacy Code by Michael Feathers
3. Use ReSharper (especially for Refactoring)
4. Use a Mock-Framework (preferably RhinoMocks)
5. Use a Dependency Injection Framework like Spring.NET

I bought Jimmy Nilsson’s book in the conference store and read it cover to cover on my flight back to Vienna. Then I bought the second book and read it within one week. I started to use ReSharper more extensively to apply some of the patterns from Michael Feathers’ book to get some unit tests in place. I extracted a lot of interfaces, brought Spring.NET into action and used RhinoMocks and the VS2005 built in UnitTest-Framework to write some useful unit tests. I also used the built in code coverage functionality in combination with the unit tests. In addition we already started Design for a messaging based service application that we want to develop in a very TDDish style.

As you can see there’s a lot going on since I attended your session. It was really this discussion about agile principles that gave me sort of a boost.

So again – thanks for opening my mind and keep on doing this great work!

Regards,
Karl

In my opinion, Karl and his team are the real heroes here. You can be a hero too by taming your software dependencies!

Today at lunch* I’ll be joining Glen Gordon and Lynn Langit on geekSpeak to talk about Taming Your Software Dependencies. Specifically I’ll be talking about moving from tightly-coupled to loosely-coupled architectures using dependency inversion, dependency injection, and inversion of control containers. geekSpeak is an interactive LiveMeeting driven by audience questions with no PowerPoint and lots of code. Come and geek out with me on geekSpeak! Register here.

* Today at lunch == Wednesday, March 26, 2008 from 12-1pm PST or 1-2pm MST or 2-3pm CST or 3-4pm EST or …

My latest article just hit the web in the March 2008 issue of MSDN Magazine. Loosen Up: Tame Your Software Dependencies for More Flexible Apps takes you on a journey from a highly-coupled architecture, which we’re all familiar with, to gradually more loosely-coupled ones. First stop is the problems inherent in highly-coupled applications. To start solving those problems, I look to dependency inversion and service location. Next stop is poor man’s dependency injection and then a simple, hand-rolled inversion of control (IoC) container. From there, I look at the features provided by full-fledged IoC containers and use Castle Windsor as an example, along with some Binsor thrown in for configuration goodness. My goal was to help developers understand the benefits of dependency injection and IoC containers by showing them the problems solved at each stage of the journey.

A big thanks to Howard Dierking, MSDN Magazine editor extraordinaire, for his encouragement and not having an issue with using Windsor for the advanced examples. Thanks to Oren Eini for help debugging a Binsor configuration problem in one of the examples. And an especially big thanks to my spouse and two young sons for their patience while I was writing.

Thanks in advance for reading the article. I welcome your feedback and questions.

Here are the code samples from some recent presentations…

Alberta Architect Forum 2007 – Creating Flexible Software

Austin .NET User Group – The NHibernate Mafia and the Persistent Ignorant Domain Model

Edmonton Code Camp 2007 – Building Applications Using ALT.NET

Thanks to everyone for attending the sessions, asking questions, and providing feedback.

The ALT.NET Open Space Conference in Austin has come to a close. It was a fantastic experience. I had an opportunity to connect with many people whose books I have read and tools I use. Thanks to everyone who came for sharing their time and knowledge. Given that I’m used to wearing the presenter hat, it was a refreshing change to attend a conference where the purpose was to discuss and question our core tenets. Which brings me to…

What is ALT.NET? This continues to be a hotly debated topic – before, during, and after the conference. My own take… ALT.NET is about considering alternatives, it is not about being alternative. It is about taking the best frameworks and tools from Microsoft, third-parties, and the open source community to build an application that is maintainable. It is not about being anti-Microsoft. Scott Hanselman suggested that Pragmatic.NET might have been a better name. I would tend to agree. At heart, everyone there was a pragmatic developer. (If you have not read The Pragmatic Programmer: From Journeyman to Master by Andrew Hunt and David Thomas, you really should.) It’s about thinking outside the Microsoft MSDN canon when building software.

What I find truly amusing is that, as far as I can tell, Microsoft has not attempted to create a canon. The collective knowledge has been created by regular people like us and people are fallible. Ideas that seem good at first may not stand the test of time. (DataSets/DataAdapters are a great example of an idea that in hindsight isn’t such a great one.) Microsoft is very open to better ways of developing software, as evidenced by the fact that many Microsoft employees attended. What many perceive as hostility to the community is more often disconnection from the community. But the honest truth is that they want to engage with us. I had great conversations with Simon Guest (Architecture Strategy Team), Howard Dierking (Editor in Chief of MSDN Magazine), Scott Guthrie (DevDiv), Peter Provost (P&P Team) Scott Hanselman (newest member of the Borg), and many other ‘softies about the values of the ALT.NET community and how Microsoft technologies support or hinder the development of maintainable software. (My intended purpose isn’t name dropping, but to show you, dear reader, that it’s the movers-and-shakers in Microsoft who are interested in providing a better development experience. These are the people who shape the platform and its documentation.)

Some people, who didn’t attend, have claimed that the conference was cliquey; that it was a venue for like-minded developers to say, “I do TDD.” and another developer to say, “Hey, cool. I do TDD too. You’re awesome.” On the contrary, the purpose was to foster better understanding of new techniques, such as BDD, specifications, domain-specific languages, FIT, etc. The conversations all started with a mutual understanding of what we were trying to achieve – more maintainable software supported by appropriate test artifacts.

For example, there was a lot of discussion around specification languages (like NBehave) were valuable compared with a written story on an index card. A written story is approachable by a non-technical business stakeholder and can elicit a discussion that might not otherwise happen. A specification written in C# (or Ruby) sitting in a source control system is not approachable by the non-technical business stakeholder. The answer is it depends on the individuals involved in the project. Each technique causes friction and your choice depends on which type of friction results in less project risk.

Another topic that has been widely reported is Scott Guthrie’s first public demonstration of the new ASP.NET MVC Framework. There were some pleasant surprises, including:

• Microsoft’s commitment to testability. Controllers interact with the ASP.NET pipeline through IHttpRequest/Response rather than HttpRequest/Response. That little “I” allows us to mock out the dependency on the HTTP pipeline and run controller tests without all the ASP.NET hosting infrastructure. (ScottGu is looking to port those interfaces back into ASP.NET WebForms too.)
• Microsoft will provide easy integration with a variety of other frameworks. For instance, if you want to mock a dependency, you can use Rhino Mocks, NMock, or TypeMock. If you want to use an inversion of control (IoC) container, Microsoft will allow you to bolt in Castle Windsor, Spring.NET, StructureMap, or ObjectBuilder. The team is even planning on providing templates for these open source frameworks. Microsoft is working with the community rather than trying to compete against it.
• ScottGu’s team is not trying to wedge MVC into the existing WebForms model. If you use MVC, you are opting out of viewstate, postbacks, and the dreaded ASP.NET Page lifecycle. This is a brave move to make. WebForms will still work as it always has, but if you choose MVC, you will use a new set of controls specifically designed for the MVC model.
• The bi-directional routing engine looks awesome. It allows for very flexible and human-readable URLs. The use of lambdas to create URLs was very cool.
• The default view engine is ASP.NET. So you get all the design-time, CSS, and Intellisense support, but without the complex ASP.NET Page lifecycle, as noted above. You can easily create typed view data to communicate between your controller and view. (In MonoRail, the controller and view communicate via an untyped property bag. So you don’t get Intellisense and refactoring support.)

Overall, the design of ASP.NET MVC looks a lot like Castle MonoRail and has aspects of Ruby on Rails and Django sprinkled in. I’m looking forward to watching this technology develop. Jeremy Miller echoes my sentiments when he says, “I’m more excited about an upcoming Microsoft technology than anything else since moving from VB6 to .Net.” If you know Jeremy, he’s a hard developer to impress. So that says a lot!

There was so much more. I’ll try to put my thoughts around other discussions in the coming days as I mentally digest everything that was discussed. I’ll leave you with some personal highlights:

• I enjoyed finally meeting Jamie Cansdale (TestDriven.NET) and hearing that he and Microsoft had come to an amicable agreement over TestDriven.NET.
• It was great to meet Jay Flowers and thank him for all his great work on CIFactory. I even convinced him to implement an option in CIFactory to flip from using the command-line, CCNet.exe, to the Windows Service, CCNetService.exe.
• Rock climbing Saturday night with Scott Hanselman, Russell Ball, and others was a blast. I didn’t feel like “a monkey with fly paper gloves”, but it was still a lot of fun. Maybe I’ll take it up as a hobby… 
• Talking to Martin Fowler over dinner about the challenges in parallel computing. He said, “Concurrency is a lot like a white fluffy bunny…” at which point he leapt up and down wildly yelling, “with fangs and a mean streak a mile wide!!!” (Fans of Monty Python’s The Holy Grail will understand the reference. If you don’t, go rent the movie right now.)

Since I’ll be in Austin for the ALT.NET Open Space Conference early next month, my friend, Jeffrey Palermo, asked me to give a talk at the Austin .NET User Group while I was in town. So without further ado…

How does NHibernate enable a persistence ignorant domain model? Why would you want a persistence ignorant domain model? Why does the NHibernate Mafia feel so strongly about persistence ignorant domain models? And who are the NHibernate Mafia anyway? This session will answer all these questions and more.

Date/time: Monday, October 8, 2007 starting at 5:30pm
Location: Microsoft MTC

Why do I do what I do? To make a difference in the lives of other developers. I was delighted to receive the following email recently.

Hi James,

I’m writing you because I just wanted to thank you!

It was about two months ago and I attended TechEd/Orlando. I have to say that it was the first time for me and it really was an honor to be the one and only chosen from approximately 300 software developers working in my company to go to Orlando. I was very impressed about the good quality of the sessions, especially on the architecture track, but your tiny little discussion on Tuesday evening really opened my mind. At that time I had been working as a software developer for 7 years with about 5 years experience in software design and project management and 5 years of .NET experience. I was fighting with a 400,000 LOC .NET legacy monster that’s used by public safety agencies around the world in security related areas. I have a team of 12 developers and we were trying really hard to keep this beast up and running and extend it with new features. I think you can imagine that most of the time we were using the trial and error approach to get new features in (just hack it in and prepare for long debugging sessions hunting weird bugs in parts of the application you never expected to be affected by the new feature…). The main problem was – guess what – the dependencies. There were about 50 classes (all singleton “Managers”), and every single manager was tied to at least 10 other managers – no interfaces at all – and that’s just one of several layers of the app… We knew that dependencies were our problem, but we had no clue how to solve it – there was this chicken/egg problem – I want to decouple my system, which needs a lot of refactoring. To ensure that I don’t break anything I’d need unit tests but I can’t use them because my system is so highly coupled We have tried TypeMock, but my feeling was that this went in the wrong direction. Needless to say that this attempt failed.

During the discussion after your session you gave me some quite useful hints:

1. Read Applying Domain Driven Design and Patterns by Jimmy Nilsson
2. Read Working Effectively with Legacy Code by Michael Feathers
3. Use ReSharper (especially for Refactoring)
4. Use a Mock-Framework (preferably RhinoMocks)
5. Use a Dependency Injection Framework like Spring.NET

I bought Jimmy Nilsson’s book in the conference store and read it cover to cover on my flight back to Vienna. Then I bought the second book and read it within one week. I started to use ReSharper more extensively to apply some of the patterns from Michael Feathers’ book to get some unit tests in place. I extracted a lot of interfaces, brought Spring.NET into action and used RhinoMocks and the VS2005 built in UnitTest-Framework to write some useful unit tests. I also used the built in code coverage functionality in combination with the unit tests. In addition we already started Design for a messaging based service application that we want to develop in a very TDDish style.

As you can see there’s a lot going on since I attended your session. It was really this discussion about agile principles that gave me sort of a boost.

So again – thanks for opening my mind and keep on doing this great work!

Regards,
Karl

All I can say is wow! Karl, you did the really hard work of tackling and taming a 400 KLOC legacy monster. Kudos to you and your team.

I would like to comment on a few points…

TypeMock vs. RhinoMocks

Karl’s team tried using TypeMock to, presumably, introduce unit tests into their code. Unfortunately TypeMock too easily takes you down the wrong path. TypeMock uses the Profiling API and allows you to mock anything in the .NET Framework. Want to mock out System.String? Go right ahead. Unfortunately mocking out framework types does not get you any closer to your real goal, which is to lower the coupling and break dependencies in your system. There has been much discussion of whether TypeMock is too powerful. I wouldn’t say that it’s too powerful, but more that it encourages you to do things that aren’t necessarily a good idea – like mocking types in the .NET Framework.

RhinoMocks uses Dynamic Proxy 2 to perform its magic. In order to generate a proxy, you must mock out interfaces or abstract classes (with virtual methods). This forces you to confront your coupling and dependencies head-on. You could do the same thing with TypeMock, but it takes discipline. RhinoMocks forces you to improve your design in order to make it testable.

If you’re wondering how to retrofit unit tests into an existing code base safely, I would highly recommend checking out Michael Feather’s Working Effectively with Legacy Code mentioned above. Oren Eini (aka Ayende Rahien), the creator of RhinoMocks, wrote RhinoMocks after being inspired by this book.

Singletons and the Single Dev

As Karl mentioned, they originally had a lot of singleton managers. The code probably looked something like this:

CustomerManager.Instance.Save(customer);

The singleton is the most-overused Gang of Four (GoF) pattern in existence, probably because it is so easy to implement. It also leads you down the dangerous path of essentially global variables, which OOP tried to get rid of. Rather than a global string variable, we now have a static instance to which everyone couples – the “Instance”. (Hey, don’t feel bad. I once overused singletons too! Live and learn.)

Why is this bad? Because any class that uses the CustomerManager is now tightly coupled to CustomerManager. And as Karl found out, when you’ve got 50 managers, they quickly become coupled to every other manager. If you try to test any small part of your system, you end up pulling in a large fraction of the 50 managers and likely they bring along most of the rest of the system. You end up with a tangled morass of virtually untestable code. Your codebase ends up being relatively fragile because a change in one manager often ripples to every other part of the system. So making small localized changes is difficult.

The solution is to move toward interfaces and dependency injection. I’ll be writing more about this in the future, but for now, go back and read this for a refresher. Once you have dependency injection in place with unit tests and mocks, you can introduce an Inversion of Control (IoC) framework and take things to the next level. Which brings us to…

Inversion of Control (IoC) Containers

Karl and his team chose to use Spring.NET, which is a good IoC framework. There are many options for Inversion of Control containers, including Spring.NET, Castle Windsor, and StructureMap. I have personally been working with Castle Windsor and am pleased with the functionality and integration that it offers. I would encourage readers to investigate all three and make a decision for themselves. I’ve got some blogposts and screencasts coming in the next few months on the importance of IoC and how to get started with it.

VstsUnit vs. NUnit

Karl mentions using VstsUnit (aka mstest.exe) for unit testing. Even though I wrote the VstsUnit Plugin for ReSharper, I would personally recommend using NUnit or MbUnit as both are more TDD-friendly than VstsUnit. Many improvements are being made to VstsUnit in the VS 2008 release, which will make it more TDD-friendly, but for now, stick with NUnit or MbUnit.

Code Coverage

For code coverage, I would recommend using NCover and Grant Drake’s NCoverExplorer combined with NUnit or MbUnit. NCoverExplorer provides nice UI navigation of NCover results. If you want easy integration between all these tools, I would recommend checking out Jamie Cansdale’s excellent TestDriven.NET add-in for Visual Studio.

ReSharper

What can I say? I love JetBrain’s ReSharper. If you haven’t seen it before, it’s an add-in for Visual Studio geared toward refactoring and TDD. Doing TDD without ReSharper installed is like coding Windows Forms in Notepad. It can be done, but it’s painful. To give you another analogy… Vanilla Visual Studio is good for production line assembly of code. ReSharper turns you into an artist sculpting code to be aesthetically beautiful. JetBrain’s tagline is “Develop with pleasure!” and ReSharper definitely lives up to that. If you haven’t checked it out, I would definitely recommend downloading it and checking it out.

Conclusion

Karl, I am so glad that my session inspired you. I am delighted that you took the initiative to tame a 400 KLOC monster into something that’s a pleasure to work with. Thanks for taking the time to write and making my day. That’s why I do what I do.