Joel Hainley : San Francisco Bay Area Software Consultant

Shad, a fellow developer and good friend, and I were hiking a great new loop I found in Mt. Diablo State Park the other day and after solving the rest of the world’s problems we eventually came around to talking about things that were driving us nuts in our current projects. We both had recently found ourselves with code that was initially a quick and dirty library with a “Button 1″ interface. ( You know the type, programmer’s make them. They have a single command button on them and you click them and then they do stuff..and never notify you when they are done, and don’t resize and a host of other things that a normal UI should have ). These “Button 1″ quick and dirty apps have a nasty habit of turning themselves into indispensible tools for production/operations stuff.

The big problem we were both having was that while the apps were decent enough for their initial use we were being asked to extend them and both of us wanted to do some refactoring of the application before we released them into the wild. The best approach for this sort of thing is to develop unit tests so that, once refactored, we could be sure that we hadn’t broken anything important.

The biggest problem with dropping unit testing into an application after the fact is that sometimes you run into problems where you have to refactor just to get the unit tests access to all of the routines that you want to test. This isn’t an ideal situation, as you quickly end up wondering if you are breaking things just trying to get some unit tests built that are supposed to ensure you aren’t breaking anything. So you start wondering if instead of refactoring if you simply make functions that wrap your private functions and expose the behaviour might be the answer, pretty soon you have stuff everywhere and you wonder which is your testing code and which is your unit testing code to make all of this possible. Madness.

As we dropped off of the fire trail back into a heavily wooded area with a nice creek, it came up in conversation that perhaps partial classes were the way to go. If you haven’t used c# partial classes, you will. You can write class code in multiple files and yet at compile time it is assembled into a single logical unit. Several days later I had some free time and was able to verify that this would indeed work, I could drop all of my unit testing code into a single file that would provide all of the unit testing logic and wouldn’t muddy up the existing class. Furthermore, with a couple conditional complilation statements I could get rid of all of the unit testing code from the final release of the new library.

This is obviously not the approach you want to take if you can avoid it. It is arguable that you do not want your classes invaded in such a way for the purposes of unit testing. This is something you can work around if you use unit testing from the beginning, but what if you inherit a project and it doesn’t have any unit test capabilities built into it? What if the way that the classes are structured make unit testing pieces of the application difficult? Do you just plow forward and hope for the best? I hope not, and hopefully this stop gap measure to get unit tests built into the classes so that you can test existing functionality is the way to go. With these “nunit partials” in place you should be able to then start refactoring the rest of the application to be a little bit better structured for unit tests without having to risk dropping anything.

I met with a company recently about doing some work for them. During the meeting one of the UI developers let me know that they were using xhtml in strict mode.  Now I have to admit it has been a while since I looked at xhtml and I’ve never actually done anything with the strict variant before. So I got a little curious and decided to spend some time and see what it was all about.

I pulled up the xhtml specification the next day and read through it. The name “xhtml strict” kinda hints at what you would expect to find and it does a pretty good job of living up to its name. It’s a very precise way of working with xhtml, it doesn’t allow you to “get away” with sloppiness, it requires that all attributes must be quoted, all tags must be well formed, and closed properly, attributes must be lowercase, you can’t have “name” attributes they are all id attributes and a bunch of other things like that. My initial impression was that this doesn’t seem that arduous, it’s simply a way of enforcing a convention upon a document. .

In order to see how it stands up in the “real world” I decided that I would update hamtesting.com to be xhtml 1.1 strict compliant just to see what sorts of things a developer could run into while working in this environment. Below are some of my thoughts about the process.

My initial efforts at getting all of the templates into shape went pretty quickly in about 45 minutes I was able to get all of the main site templates updated to be compliant. Things like updating the br’s and hr’s and adding /’s to input items on forms were all of the things that I expected. Also expected was that all attribute values need to be lowercased. So if you have an onclick defined for something in your page it can’t be onClick or ONCLICK or anything else, it needs to be onclick.

The first thing that kinda gave me pause for a bit was that it is really picky about which elements can be just below the root element in a document. This ends up not being an issue and actually leads to a lot more consistence in the document structure. It was a little bit of a pain at first but it quickly got into shape. The specification does spent a certain amount of time talking about which elements can contain other elements and such things, so it’s probably good to have that floating around in your head. Or use a validator plugin ( see below )

The second thing that was a little difficult was that I had used the name attribute in a lot of places for some of my javascript interactions. So first I made everything xhtml compliant by switching things from name attributes to id attributes, with this done the site didn’t work terribly well. So I updated one javascript routine in my main library and then whammo everything was working well again.

With this all done I spent a bit of time and knocked the css document into shape and got it validated as well. So now when you go to www.hamtesting.com you will see badges from w3c for the site being both xhtml compliant and css compliant. Kinda fun.

Probably the biggest lesson learned was that while the w3c’s online validator services are fine for a page or two they can be a pain after a while, especially if you are trying to validate pages that are in an authenticated area of your site. Do yourself a favor and go get a html/sgml/xhtml validator plugin for firefox so that you can just pull up the errors as you run across them.

In the end, this wasn’t as difficult as I thought it was going to be. My impressions are that xhtml is kinda nice, and using the strict variant definitely enforces a certain amount of discipline into your efforts. It’s a good exercise if you have some spare cycles and want to dig in a little.

I ran into this over the weekend and since it took a bit of time to find I thought it might be useful for others. I was setting up an ATG development environment and had installed a version of MySQL that was known not to work with ATG..(oops..I’ll cover this in another posting). I uninstalled MySQL using the following steps :

1. cd /usr/local
2. sudo rm mysql
3. sudo rm -rf mysql-<some version number here>

With this done I downloaded a different version of MySQL and tried to install it, however I received the following error.

“You cannot install MySQL blah-version-blah for Mac OS X on this volume. A newer version of this software already exists on this volume”

After digging around about this for a while I was finally able to come up with the problem. You need to go into the /Library/Receipts directory and delete the Mysql related items in there was well. An example :

• cd /Library/Receipts/
• rm -rf Mysql<blah blah blah>

With this done. I was able to continue installing the new version and get back to stuff. Hopefully this helps someone else.

Recently I was working on a project and needed to proxy a request to a website that was returning a binary file.  I ran into some problems with my first pass at a solution. I believe the problem was a consequence of proxying binary data without getting at the underlying streams. I’m not sure how often I’ll need to do this, but I thought it might be useful to post here for others to know about in case they ran into the same problem I did. So here’s the code that worked for me, hopefully it’ll work for you too.

---

HttpWebRequest req = (HttpWebRequest)WebRequest.Create(”http://to.some.binary.data”);
req.Timeout = “blah”;
req.Method = “GET”;

HttpWebResponse resp = (HttpWebResponse)req.GetResponse();
Stream reader = resp.GetResponseStream();
// you might want to set your responses content type here
System.IO.Stream r = Response.OutputStream;

int Length = 256;
Byte [] buffer = new Byte[Length];

int bytesRead = reader.Read(buffer, 0, Length);
while(bytesRead > 0){
r.Write(buffer, 0, bytesRead);
bytesRead = reader.Read(buffer, 0, Length);
}

---

Hope that helps.

In part 1 of this article I argued that exceptions sent “over the wire” to flex/silverlight clients from a webservice weren’t necessarily the right way to put things together. I argued that that exposing your clients to raw exceptions happening in your server was not elegant in the same way that throwing raw exceptions to a user in a windows desktop app was not elegant. In talking about the last article with some people I realized that I needed to make a point. There are times when an exception is appropriate, and I’m not advocating avoiding exceptions altogether. My point was that I felt people were relying on exceptions too heavily when a more appropriate method might be possible.

Let’s take a look at input validation to see what we mean about relying too heavily on exceptions. Validation is a bear, you want to try and have your validation rules in one spot only. This way you can avoid the chore of duplicating validation logic on the client and the server. This seems to be one of the main reasons people are looking at bubbling exceptions over the wire because if they can simply put the validation logic in the server side, then they can throw an exception back to the client and deal with it there. It solves the the problem of duplication of validation logic and allows developers to deal with validation errors in the same way that they do within desktop application environments.

The question now becomes can we avoid exceptions and still provide for a rich validation environment? I believe that we can and I’m going to spend the rest of this article describing an approach. It may not be the best approach, it may not be the only approach, but it was worked well for me on the last few projects and enough people I’ve talked to about the approach seem to feel that it’s powerful enough to consider for their future efforts along these lines.

In order to provide a framework upon which to hang our theory let’s consider a simplified method of a web service named “CreateAccount” that takes as it’s only parameter an Account object.

When we want to create a new account we will fill out the information about the account as described and then pass it to the CreateAccount method of the webservice. However, instead of just having a void for the return value and throw exceptions if there are errors you simply use a response object such that the method signature would look something like the following :

CreateAccountResponse  CreateAccount( Account acct)

This is certainly not a new concept. Response objects litter many namespaces, but in this particular situation they allow us to avoid having a lot of exceptions flying out of our webservice and doing all sorts of ridiculous plumbing hacks to get it all to operate nicely.  Now let’s take a quick look at the CreateAccountResponse class and see what it looks like, then we can talk about how a response object allows us to avoid duplicate validation and exceptions. Below is the UML for the CreateAcountResponse object

We see that this class mmics the Account class pretty closely with the addition of a couple of routines that would be needed to actually communicate results. The Success flag would indicate whether the account creation was successful and the error message would only be filled out if there was a particular error that you wanted to communicate to the user such as “You do not have the rights to create accounts” or something along those lines. The other fields would be filled out if there were problems with the validation of the individual fields of the account object. This would allow us to have all of the validation/error messages specified on the server side in a single location and still be able to get the information out to your users. Such that if you require the state field to be only 2 characters long and someone puts in 4 you could set the success flag to false and then add error text to the State field of the response object that could then be displayed to the end user.

WIth this approach if there was an exception generated in the webservice, we could trap it server side and provide the appropriate message to the CreateAccountResponse object and not have to require the client to exception handling just to know what is happening.

Now one of the complaints  that I have heard about this approach is “but now you have to create all of these response objects”. Well yeah, but one way or another you’re going to have to write some code to deal with errors and wouldn’t it be nice if you could deal with the errors and the error messages in a single place? Also you could simply create a generic results object that would suffice for most of the calls that you want to make. Then again consider that in some of the more mature environments such as .NET WebServices  you can easily create webservices that provide this sort of functionality and the WSDL’s and whatnot are all created for you.So it’s not nearly as much work to do this as it could be.

What about the overhead of all of these custom classes? Is it really any more overhead than the throwing of exceptions? Exception throwing, and handling are expensive operations. I find it a little be cleaner to my thinking because now you have a clear and concise definition of what you need to program against when you are dealing with your WebService’s API.

I’m not saying this is necessarily the best approach for all occassions, and I’m not really down on exceptions in general, or even for web services. However I think that there are many times when the extra effort you have to go through just to get robust exception handling done will lead you into a lot of needless efforts. Especially if you are just trying to provide validation/failure information back to the client applications.Keep in mind that what I’ve presented here isn’t a blueprint, it’s more like a pattern of how you might consider structuring your webservices in certain situations.

Your mileage may vary.