Poor man’s usability test

July 3rd, 2008

Going public with donjon was a great decision. We got several levels of feedback, and not all of them are conscious or direct. Let me go over some of these:

Exceptions

First of all, we use MonoRail Exception Chaining extension and get e-mail with all exception details, plus the page, form, query string, session and flash state, so it’s relatively easy figuring out how to reproduce the situation that caused the exception. This approach wont send us only unhandled exceptions, instead we use RegisterExceptionAndNotifyExtensions to make sure we get it. The following is a typical donjon’s action in a controller

[return: JSONReturnBinder(Properties = "Id,UserName,FullName,Email")]
[AccessibleThrough(Verb.Post)]
public User[] AddMembers(string key, string[] users)
{
    try
    {
        ...
    }
    catch(Exception e)
    {
        RegisterExceptionAndNotifyExtensions(e);
        Logger.Error("AddMembers failed", e);
        throw;
    }
}

RegisterExceptionAndNotifyExtensions will make sure the chain is going to be executed. The Logger is our ultimate place to detect problems, finally we throw so the ajax call get a status 500. Some places I dont even throw, just set StatusCode to 500 and RenderText(a nice error message)

I also find a best practice to always start writing and testing an action/page with the error handler. So my actions usually starts as this:

public void Action()
{
    try
    {
        throw new Exception("something bad");
    }
    catch(Exception e)
    {
        RegisterExceptionAndNotifyExtensions(e);
    }
}

I find a good practice to assume that at some point something might go wrong, and the user is properly notified. It’s just too much to assume that the action will always be successful.

Settings and defaults

Each user on donjon has a preferences setting. Inspecting the database I found that almost no one ever set a preference. Disturbing to me, as I think the Issue list as it is is just too bloated with information, so the first thing I do is configuring it to show less columns. Nobody did that, possibly because nobody knew how to do it.

Ok, lesson learned and code fixed. donjon will set up an user with an initial preference set to less columns.

Right side Navigation

Aaron complained about this one. It was inspired on some tool I’ve used, so I thought breaking the convention would be a good thing. Bad news to me: “don’t break user expectation” and “don’t try to create new conventions unless they are revolutionary” are some huge bullet points in the book “Prioritizing Web Usability”.

Just to unarguably prove that my decision was dumb, I found the article Navigation: Left is Best

Another lesson learned.

Others

The issue list page was designed and redesigned at least 10 times. All of them functional. The present version is the one that I’m satisfied, but I’m sure there are room for improvements.

I gotta tell you that at first it’s not easy to not to think as a programmer and instead wear the user hat when designing a page. Your programmer’s mind want to make things easier for you (code-wise), your user’s mind wants to make things much more complex. Yeah, sometimes to make the experience simpler you have to write much more code. My guess is that it’s an eternal struggle.

So expect some big improvements for beta 2 :-)

7 Responses to “Poor man’s usability test”

Victor Kornov Says:

From your code it looks like you could do exceptions logging using generic approach. Something simple like AOP :)

Ayende Rahien Says:

Is this really the code you are using?

catch(Exception e)
{
RegisterExceptionAndNotifyExtensions(e);
Logger.Error(“AddMembers failed”, e);
throw;
}

That screams “Use AOP”.

hammett Says:

Some reasons why I dont use AOP for these

- tends to remove predictability
- introducing proxies on controllers used to introduce problems
- performance is sub-optimal
- not all exceptions are treated the same way

Victor Kornov Says:

- not all exceptions are treated the same way

for that i would do:

T Action(params) {
return Safe(() =>
your code here
);
}

T Safe(Func yourCode) {
try { return yourCode(); }
catch { usual logging; }
}

+ an overload for Safe(yourCode, yourExceptionHandlingCode)

Michael Morton Says:

I don’t necessarily agree with Aaron about the nagivation. Also, after reading the linked article, I think the author picked a horrible title considering the conclusions from the research studies.

Key Quotes:
1. “Previous research by Kalbach and Bosenick (2003) reported no significant differences when users navigated using either right or left navigation panels.”

2. “In addition, navigation times were faster when users made all selections from the same navigation panel; users also preferred this option.”

3. “Performance and preference measures also showed that the right hand navigation option was a viable design option.”

4. “Kingsburg and Andre’s research demonstrated that navigational selection limited to either the left or right panels resulted in the best human performance and strongest user preference”

What I take away from the article in question, without reading the papers from the two studies, is that there is little difference between a “left” navigation and a “right” navigation if the navigation is structured well.

They only mentioned a performance advantage to a left based navigation twice (and from the sounds of it, it was probably a very small advantage) and mention that this could be explained by the reading pattern of the participants. Considering the limited number and scope of participants in each study (16 people that read *only* left to right) … I don’t know how they can even consider the results statistically sound.

hammett Says:

Michael, I’ve updated support.castle-donjon.com. I appreciate feedback.

Neal Blomfield Says:

I’ve just logged a bug in the new castle support website and I gotta say I really liked the look and feel of donjon. Awesome job!

Leave a Reply