WillowSense

Go with the flow

2010-07-21T22:16:00.005+02:00

The SPIN meeting in Cape Town this month again impressed with a high quality presentation. Phil Barrett from Flow highlighted the importance of user interaction design by elaborating on the role of the user experience designer in the software development process. He inspired us to think about better user interfaces, ensured us that Agile has room for UI design, and certainly made some of us wish we could have him in our team. I could not attempt even a summary of all the good things that were said. Instead I would like to share with you three key ideas that I took home from Phil's talk (obviously, these are heavily laden with my own egocentric perspectives J).

The first is that better user experience is money in your pocket. It is not about being nice, doing the right thing or getting a warm and fuzzy feeling – it is about selling software or converting website visitors into website customers. It is also about beating your competitor, and keeping your customers from looking around for better solutions. In short it might be a good idea to spend some money on user interaction – even if you think your software works well.

The second idea is that user interaction design can easily reach a local maximum. When you start coding you have in mind a particular solution – and as you code you become more and more tied into the many small design decisions you make along the way. Small increments are relatively inexpensive and each one improves the system ever so slightly. However, at some point change becomes difficult and your user interface could end up with a feel that is, in a very practical sense, at an optimal solution. At this juncture the identification of a better solution does not help because changing to this brave new world is simply not cost effective. You are stuck at a local maximum - it is the "payback" for all those short-cuts you took when the coding started. Avoid this situation by thinking through your user experience design carefully – and aim for that higher maximum from the start (this is very much like the concept of entropy I discussed in a previous blog)

The third concept revolves around the role of the user experience designer. The way I see it follows purely from the pig and chicken perspective:

If the application does not work, roll your eyes (if you must) toward the developer and the tester
If the application does the wrong thing, place the blame on the doorstep of the business analyst
If the application does the thing wrong, the user experience designer is your culprit

If you are not equipped or entitled to hire a user experience designer, Phil suggests that you could interview the users yourself. Focus on having an open honest conversation – ask politely, then listen carefully at the reply. Wait for four seconds before offering your comment. Good advice, I'd say!

Control that view to the model

2010-06-01T17:54:00.010+02:00

Remember that good old model, view, controller idea – simple isn't it? We need only a model, a view and a controller. Maybe we want more than one view – that would be nice. But then, what is a model – and when does one model become two models? OK, but surely the controller is simple – it must be a singleton, or not? The thing about MVC is that once you get past the M, the V and the C, you may have more questions than answers. Soon we all end up with our own definition of vanilla MVC. Then we deviate from this unattainable ideal. We have many excuses because we face real world problems, imperfect operating platforms, clunky programming languages, silver bullet UI frameworks and so on.

Maybe these realities are challenges, not excuses. Maybe throwing away the architecture is not an option and new ideals are what we need. But, before we go to that braver place, let's first find out where are we now.

My own vanilla perspective considers the MVC triad as three separate concerns. Each concern has a basic set of responsibilities. It goes without saying that I think this perspective is in fact crystal clear and entirely practical ;-)

The Model encapsulates the state, and exposes the functionality of the application. It notifies clients of significant changes to the state.
The View renders the model to the end-user. It receives notifications from and performs queries on the model.
The Controller defines the application behaviour. It receives a gesture event from the view, transforms this to an action and applies the action to the model. It also selects the parts of view to show as a response to an action.

From these responsibilities, it is clear that the model is the foundation, and that it should be isolated from the view and controller. This separation is a key benefit of MVC. The idea being that one can use the same model to write diverse applications that run on different technologies. Practise show that this benefit is unlike to be a key factor in most applications. However, it does help to think along these lines when putting together your application architecture.

Instead of one technology independent model, I lean towards the idea that there is some part of the model that is technology specific. I separate this part into a sub-concern called the application model. Those really non-technological model classes; that is the domain model.

This domain model exposes the functionality in terms of entities, values objects, repositories and such things. It is also responsible for validation and transactional cohesion of the information kept on long term storage. The application model is responsible for the application state - i.e. the information created during the application run cycle. This information could include session management, caching, job management, selection management and so on.

The application model for a web application might focus on remote user sessions, shared caching, cookie or server based user state (for shopping baskets and so on), while a rich client has client sessions, local cache and local user state in its application model. When writing for particular technology, the line between the two model types easily becomes blurred. The disadvantage of this blurring becomes very clear when the application is ported to a different technology or to a new framework. However, less radical changes could also be prohibited when mixing application and domain model classes. It could become difficult to fix non-functional problems (such as performance and responsiveness issues) without affecting key functional areas of the application.

The model could be a grey area, but clearly the view and the controller are technology specific. Different technologies call for different approaches on how views are to be rendered and how user gestures are captured and transformed into actions that are applied to the model.

It is good design practice to delineate a namespace for each concern. These namespaces form a layered design. Each namespace provides services only to those above it. At the bottom is the domain model – it does not consume services from any other namespace. Right above the domain you'll find the application model, and the view is the next layer on the stack. The top layer is the controller. The controller can use services from any other layer – and it provides no services to the other layers.

With these namespaces in place, the process of design starts with a decision where to place a new class, and to make sure this class does not violate the constraints imposed by its designated layer.

When the allocation of a class to a layer is not obvious, consider choosing the top layer and move down only when the need arises. For instance, if it is not used by any layer other than the controller, it belongs in the controller. You can always move it down later when things become clearer – but it is more difficult to move it up to a higher level in the stack (when things become more muddy).

Consider yourself reminded of the MVC basics – look, think ... improve!

Invalid ideas

2010-03-14T15:59:00.013+02:00

Faced with a clean slate, many application architects take yet another look at the model-view-controller (MVC) pattern to veer off the evils of a bad foundation. Even those architects that choose ready to use frameworks tend to measure these against concepts coupled with MVC. Of MVC itself many variants and interpretations exist, but I think the general idea idea is to divorce the "model" from the "view" (UI). The "controller" part is the artifact that is needed to manage the possibly complex interaction between the two. MVC regards the UI is as the presenter of information and as the initiator of many actions.

In many real world applications, the idea of "present" and "initiate" is well understood and makes up a small part of the code. A lot of the code lies in the "capture" and "store" of the information. In these applications, the complexity tend to revolve around the validation rules that facilitates data capture. Vanilla MVC does not address this issue in specific detail. Sometimes it is incorrectly assumed that validation happens on the "view" side of the MVC triad.

Recent developments on the domain driven design front also tend to suffer from this weakness. There seems to be a lack of a ubiquitous approach to validation. As you elaborate that domain model of yours you write unit test, you would conclude that many tests are about validation rules of some kind. Hence, the quality of an implemented domain model must be directly related to its ability to prevent invalid data from entering the system.

"Garbage-in, garbage-out" is a mantra repeated reverently when facing a scenario where some poor sod used an application incorrectly. Bah! "Garbage-in" was the fault of the developer. Instead of blaming the sod, we should promote him to the test team!

Values are supplied not only via user interfaces, but also by web services, data imports, queue processors and who knows what other ways. Consequently, field validation cannot be an issue that is simply shunted to the user interface level. Using MVC parlance: validation cannot be in the "view"; it has to be in the "model".

Key to your design must be the idea that a validation rule is written once and only once. Wherever a value governed by this rule enters the domain, the rule shall be applied. Non- conformance to the rule leads ultimately to the rejection of the change.

Why not simply throw an exception on the setter when an invalid value is given? The biggest problem with this approach is that the UI code (quite possibly your most important client) becomes bloated with exception handling blocks. It uses these blocks to try the translation of the cryptic exception thrown by the domain library into a meaningful error message. Mostly it fails - the message becomes irrelevant and the user is not helped. In fact he is agitated by such messages.

The exception throwing plan has more problems. Some validation rules apply to a combination of values (e.g. a rule that ensures the birth date and ID number should coincide), or require other objects to be validated (e.g. the ID number must be unique). On which setter do you throw these exception?

An alternative is to allow an instance of a class to be invalid at any given point in time. Such an object would have a set of validation rules that have been breached. Each rule is associated with the appropriate message text. This text is presented to the user. Keep in mind the user interface is responsible to present the error messages to the user. An exception occurs when the client tries to commit an invalid object (to storage or for further processing).

Many details remain to be sussed out. The message in this blog is in the shell of a nut: Do not throw up, invalid domain objects are cool!

Domains in context

2009-12-08T19:47:00.007+02:00

A few days ago I attended a two day overview of Domain Driven Design (DDD) presented by Aslam Khan. On the surface, DDD looks like the latest wave rolling in from the timeless ocean of IT fads. Dig a little deeper and you'll find that there is nothing new to DDD. It is not a new wave, and just maybe it won't go away. To be fair, there is definitely a new buzz in the air. DDD is whispered about in many corners, not all of them dark ones. There seems to be high hopes: SOA, web-services, behaviour driven design, test driven design, Agile, Scrum – all of these good things could benefit from DDD.

My interest in domains started long before I read the DDD book by Eric Evans. I have been the driver behind a few frameworks for domain driven development efforts, and with a team that contained a few good men (and one or two woman), I have tasted the sweet successes this approach can bring. The road was not paved in gold, often I felt like a lone ranger taking it upon myself to lay bare the obvious traps of data-bound controls and other evils. Lucky for me (and for Tonto) the world is changing.

Reading the Evans book was a real pleasure. Not only did I find that it agrees with the undeniable truth, but it also had wrapped in it a few patterns I really needed. We grabbed from it some pieces to create yet another framework. Of course this one (like each of its predecessors) promised to be the best framework of all. And the promise was delivered - the time was right, the problem needed it, and the book came in very handy.

It is fair to say the patterns in Evan's book are less vague than the Gang of Four. The book may also present the uninitiated with too many options. Place your focus on putting together the machinery that allows the development teams to roll out those precious domain classes. Keep in mind these classes must be loosely coupled and highly cohesive modules. But do not forget: on top of these classes some key user interfaces are needed; and on their sides high performance processes hammer them into shape. Do not even get me started on the mapping to the relational databases at the bottom of your architecture. Quite an interesting design challenge I assure you.

Maybe now you can understand why I was wondering how Aslam is going to chunk in years of experience and lots of ideas into a mere two day workshop. Selling DDD is difficult and implementing it for the first time is harder. Pleasant was my surprise when I noticed that he will attempt neither. Take the view that DDD is sold, and regard the implementation issues as infrastructural detail best left to others. Do this and your frame of mind is where Aslam wants it to be.

"Ah, but what else is there?", you may ask. The answer is obvious – it is the thing that drew all of us domain driven junkies closer to the kindle when we started on our journey. It is the absolute belief that the domain model is the key asset amongst the components of the enterprise software. It is where the knowledge lies, where the growth and refactoring happens, and also where the fun never sets.

Aslam's two day take on DDD focussed on this most important of stuffs. It is all about how to get the right model. He pulls together current topics and discusses those patterns from leading DDD books that help you come up with a good domain model from the start. Have a look at this reference card if you are interested in more detail of what he covered.

Consider yourself reminded: without the right model, the benefits of DDD will always be out of your reach. Thank you Aslam!

Dual core acting in Scala

2009-10-24T10:58:00.026+02:00

Having a dual core processor is a great thing for the operating system; but how can you make use of this cool technology as a (mere mortal) programmer? Potentially a complex issue but if you use Scala you'll be pleasantly surprised. It is the concept of Actors and message passing that saves the day. Let me explain using an example from mathematics.

A problem that can theoretically benefit from multiple cores is the one that calculates the integral of a function. Using the simple quadrature method this problem is reduced to calculating the area of all the rectangles that follow the curve, and adding these up to produce an estimate of the integral. More rectangles provide a more accurate evaluation. The image (from wikipedia) should help you get the picture.
Here is that algorithm coded in Scala:

type R = Double
def integrate(f : R=>R , a : R, b : R, intervalCount : Int ) = {  
val intervals = (1 to intervalCount)
val width = (b-a) / intervalCount  
val halfW = width/2.0  
(0.0 /: intervals) {
  (s,i) => s + width * f(a + i * width - halfW)
}}

Note that I created a type alias for Double called R. An interesting piece of code here is the last line of the function. The /: operator is the foldleft operator. This foldleft starts with a value of 0.0 and calculates the sum of width * f(a + i * width - halfW) for each i in intervals.

To calculate the integral of the Sine function from 0 to 90 degrees using 100 rectangles , the Scala function call looks like this:

integrate(Math.sin,0,Math.toRadians(90),100)

Note that I pass the Java sine function as an argument. The answer is 1.0000102809119051 Increasing the number of rectangles to 10000 gives a better estimate: 1.0000000010280816.

Here's how I can use my dual core CPU to calculate the integral faster: divide the calculation into two smaller calculations and get each core to calculate the result of a smaller part; then add up the two results and voila I am done. The two parts are defined by using the midpoint between a and b and allowing one core work on the rectangles less than this midpoint while the other core chows the rectangles that are greater than the midpoint . Thus, if I want 10 000 rectangles, each core would do 5000 calculations.

Here's the divide and conquer algorithm in Scala:

val m = (b - a) / 2.0
val caller = scala.actors.Actor.self
val iH = intervals / 2
scala.actors.Actor.actor{caller!integrate(f,a,m,iH)}
scala.actors.Actor.actor{caller!integrate(f,m,b,iH)}
val p1 = scala.actors.Actor.receive {case a : R => a}
val p2 = scala.actors.Actor.receive {case a : R => a}
p1 + p2

The first thing to note is that there is an implicit actor called self. The ! operator combined with the actor method sends a message to the actors queue and processes it asynchronously. When these processes complete they send the result back to the caller. The caller (i.e. self) receives these messages and assigned the values p1 and p2 to the messages.

The full program is given below. I optimised the final code a bit to use a tail recursive function instead of a range.

I am sure you get the picture. Scala actors are real stars!

package scalablogs





object Application {

  type R = Double

  val intervals = 10000000



  def integrate(f : R=>R ,  a : R,  b : R, intervalCount : Int )  = {

    val width = (b-a) / intervalCount

    val halfW = width/2.0

    def calc(i : Int, t : R) : R = {

      val area = width * f(a + (i+1) * width - halfW)

      if ( i < 2)

         t + area

      else

         calc(i-1,t+area)

    }

    val r = calc(intervalCount-1,0)

    r

  }



  def integrateNormal(f : R=>R ,  a : R,  b : R)  = {

    integrate(f,a,b,intervals)

  }



  def integrateConcurrent(f : R=>R ,  a : R,  b : R ) = {

    val m = (b - a) / 2.0

    val caller = scala.actors.Actor.self

    val iH = intervals / 2

    scala.actors.Actor.actor{caller!integrate(f,a,m,iH)}
   scala.actors.Actor.actor{caller!integrate(f,m,b,iH)}

    val p1 = scala.actors.Actor.receive {case a : R => a}

    val p2 = scala.actors.Actor.receive {case a : R => a}

    p1 + p2

  }



  def main(args : Array[String]) = {

    val start = System.nanoTime()

    val r = integrateNormal(Math.sin,0,Math.toRadians(90))

    val elapsed = (System.nanoTime - start)/1000000000.0

    println("Normal answer is " + r + " (" + elapsed + " second(s))")

    val start2 = System.nanoTime()

    val r2 = integrateConcurrent(Math.sin,0,Math.toRadians(90))

    val elapsed2 = (System.nanoTime - start2)/1000000000.0

    println("Concurrent answer is " + r2+ " (" + elapsed2 + " second(s))")

  }



}

Test driven you-hee

2009-10-17T17:02:00.013+02:00

In a previous blog I described test driven development (or TDD) and hinted that it has many advantages. However, when it comes to graphical user interface (UI) development, programmers expect less benefit from this development technique. The expected benefit outweighs the potential gain and many give the method no more thought. This is concerning because without test-driven development, the value of continuous integration and ultimately the benefit of the Agile method remains unattainable for the team.

The trouble stems from the fact that the user interface consists of widgets - the forms, buttons, drop-down boxes and so on. These widgets display information to the user and provides him with ways to interact with the application. In order to test this interface, someone that understands the expected behaviour use the widgets verify that the behaviour meets his expectation. This is a difficult and sometimes subjective assessment. Also, the UI test can only be written after the widgets has been implemented; and even then the cost of writing such tests might be too high.

Does this mean TDD must forever be discarded by UI developers everywhere? Of course not. In fact UI developers should embrace the method with more vigour than server side developers. User interfaces are the visible parts of the application and a change anywhere has a real chance of ending up on the user interface somewhere. In other words: it is a high maintenance area of the application. As such it should be covered with unit tests and refactored often.

The well trodden road to improve the maintainability of an application is to make sure you have a set of maintainable modules. Each module is a cohesive unit and it is loosely coupled with the other modules. The trick is to keep the user interface modules lean and avoid coupling between these UI modules.

Consider the one module application that is built entirely with forms. All the persistence, the validation rules, the processes are written in classes that extend the basic form class. This idea was promoted by the older data binding technologies. Only the simplest applications should be built this way. And even simple apllications suffer when new requirements imposes radical changes. Test driven development will not be a first choice for a developer that chooses this 'one module' application architecture.

Lets look at another plan: a two module application. The one module runs on the server, and the other module is the user interface. At least for the lucky winner of the server side development, test driven development is practical, he can start playing right away. But the user interface developer sits with the problem of validation rules, caching, some process issues and other not so little matters. He must now also retrofit his 'wacky' user interaction ideas on the 'logical' sever side API. You guessed it, no test driven development on the user interface side yet.

The answer is obviously to have more than two modules. Consider carefully what is meant by the term 'user interface', and divide the identified concerns into a number of modules. While doing this keep the size of the modules that involve widgets to a minimum. Apart from these 'widget modules' all the development work can now follow the test-driven method. The way to end up with these modules is to design them into the architecture from the start and then evolve them as the application grows.

To migrate to this Utopia, you'll need to make two hard choices:

* This first choice is to rid yourself from the data-table-driven world and move into the domain-driven way of application development. At least you'll end up with one more module: the domain module. There are many established patterns you can choose from; and test driven development is ideal for domain modules.

* The next hard choice is to choose a good user interface architecture. In many cases your development environment would lean towards a specific flavour. If it leans toward the bad old flat databinding model; it is time to look around for better tools. Patterns from the model-view-controller and model-view-presenter worlds are well established alternatives. It is important to know the limitations of the particular UI architecture and to understand how to use it well.

Generally there are always ways to isolate the model aspects, the view aspects and the control aspects of the application. The widgets are in the views; and apart from the view modules TDD works for everything else.

Keep that chin up, even for graphical user interface development test driven design is within the realms of possibility!

Say halla to Scala

2009-09-21T19:30:00.008+02:00

A few years ago I took a functional programming course at university; I immediately liked the idea. Since then I became an eager proponent of functional programming. I am now even more convinced that functional programming will continue to increase its reach into mainstream development. Advancements in C# (LINQ) and the bigger spotlight that F# enjoys is proof that Microsoft is also waking up and smelling the curry.

It's been a six months out of the Microsoft camp for me. A few weeks ago I started sniffing around in the wild for a Java VM offering that'll do for a functional thinker like myself. The language Scala was reverently promoted (albeit from a distance) by some of my esteemed colleagues, and I just had to take a closer look. What I found was a pleasant surprise: a first class language that is a viable alternative to Java. Scala is a fully featured object oriented language for the JVM that has been folded nicely with a very usable functional notation.

Allow me to shamelessly punt the language by listing some key language ideas here (mostly to whet your appetite):

Interoperability. Scala programs work with Java packages. There is no new standard object library to learn. Scala creates normal JARs, and you can import any old Java JAR and use it without modification.
Implicit typing. When the compiler can derive types (which is often enough), there is no need for you to state the obvious.
Better class design. In Scala you can declare a "Trait". This is a language construct that allows you to add implemented methods to an interface. Very reminiscent of multiple inheritance.
Local functions. Declare a function anywhere - even within the method of a class.
Functions are values. Pass a function around as a value and you end up with a very neat solution to event handlers. And that is just the start of the functional magic.
Operator overloading. Operators form an important part of the functional paradigm and Scala has a neat solution. Essentially any method that takes one argument is an operator; and yes, you can use operator symbols (such as +,-,++ and many more) as method names.
Matching. The idea of functional expression matching is nicely extended to regular expressions and compiled XSLT expression, Ready for use where you need it.
Domain specific language development. Put together a parser and interpreter for your own DSL using a powerful grammar framework.
Concurrent programming. Use a message oriented programming paradigm built into the language structure to write complex concurrent programs.

You must agree, for any language this is an impressive shopping list. Surely, many more goodies wait to be uncovered.

On a cautionary note, keep in mind I am just beginning to use the language (at home), some of the claims remain to be verified by yours truly. On the other hand, the Scala project seems very active, and the language is gaining ground.

If you feel the urge to learn something different, Scala might be just the thing for you!

The profile of your features

2009-08-08T13:55:00.002+02:00

You ask: "Is this interface user friendly?" and I'll ask "For whom?". An application that works well for one user may not be usable for another. A 3D CAD system able to render high resolution graphics is very user friendly for engineers but it is a mundane tool for 3D artists. Not all applications support the same range of users: some application domains support user profiles with vastly different skills while others have a more homogeneous user set. There are applications that should support profiles that change for a particular user over time. For example, a sales application could behave differently for beginners than it would for sales people that have expert domain knowledge. Most applications should acknowledge the difference between the needs for new application users and those imposed on it by advanced users.

A good user interface is one that leverages the strength of its users to help them achieve their goals. If you are in the business of developing tools for software developers you are in luck: you belong to the profile of your users. The high usability levels of IDEs (as attested by their wide adoption) is proof that knowing your user is actually a good thing.

You have probably heard it being said somewhere that all software developers are notoriously bad at user interface design. I despise this generalisation; but many developers actually agree with this assessment. The cause of this leniency towards us creating such abominations perhaps lies in a personality trait that we share. Software developers tend to enjoy keeping large and complex mental models in their heads. This is very useful when you are working in a small part of a very complex system. Surprisingly to us, not everyone is like this.

Most people dislike the idea of spending time to refine and perfect abstract models. They prefer to concentrate on the details that can seen in right front of them. This is not saying that they are less intelligent; just that they have a different personality type. For these people the user interface is the system, and the system is the user interface. To them, the following makes perfect sense: If you want to make the system easier, just fix the interface; and if you want to make the interface easier, just fix the system. What they get is what they see.

First impressions last, and the first thing the user sees when the application starts is the primary user interface. This interface strives to match a simplified conceptual model of the application by hiding away many features of the application. As the UI designer, you have many tools that can be used to hide features. Here are some of the most obvious ones listed in decreasing order of exposure:

A toolbar button provides access to features that must be highly visible.
A menu item requires a trivial user gesture to expose features.
A submenu requires a more involved gesture.
Dialogs exposes features that are invoked by an explicit user command.
Secondary dialogs are exposed from commands available on primary dialogs.
An advanced mode can be enabled to expose more features for advanced users.
Some applications provides scriptable functions to expose features available to expert users.

As part of your primary user interface design, you need to decide what is on your primary user interface and how the rest of the features will be hidden. Carefully consider each user profile in your decision: some user profiles could need alternative primary interfaces. Also consider how application adapts as the user moves on the beginner-expert profile continuum.

Next time a user complains about an interface, try not to explain how it "actually" works. Think out of the box: come up with a visual cue that resolves the complaint on the user interface. Odds are that you will end up with a more usable application!

Annotated Java

2009-07-12T09:46:00.025+02:00

Boilerplate code is a piece of code that is not deemed reusable enough to be abstracted into a function or a class, but it is useful enough to be copied and pasted. After pasting it, minor changes are applied to make it work in its new location. Over time, the boilerplate based code typically becomes a significant part of the source, hindering the cost-effective refactoring when the need arises. In some cases boiler plates can be removed by declarations in configuration files.
An annotation is a Java language feature that facilitates declarative programming at the source code level. Use this feature correctly and might end up with less boilerplate code, and less configuration files.
When creating your own annotation, you need to decide on the target and the retention policy of your annotation. The target specifies where the annotation can be declared; this could be on a type, a method, a field, a parameter, a constructor, a local variable or another annotation. The retention policy specifies when the annotation will be used. Annotations can be used on source level (ignored by the compiler), compiler level (ignored by the VM) or run time level.
An annotation can also be inherited or documented. When it is inherited, the values of the annotation on a super class is available on the sub class. The values of a documented annotation will be printed in the javadoc generated for the annotation class.

As an example, let's invent a simple scenario: You are implementing an API to you application data. The API is aware of the authenticated user and a set of security levels. The user is authorised to perform only certain methods on your API. The idea is to annotate a method with a security level. A user that does not have the assigned security level should not be able to invoke that method.
Assume we have an enum called SecurityLevel with three values BASIC, ADMINISTRATOR, SUPER. For our purposes we simply keep the security level as a property of the User, and the logged on user as a property of a Session singleton. The code of AnnotationDemo.java looks like this:

package com.willowsense;

import java.lang.annotation.*;
import java.lang.reflect.*;

enum SecurityLevel {
  BASIC, ADMINISTRATOR, SUPER
}
class User {
  final String login;
  final SecurityLevel level;
  public User(String login, SecurityLevel level) {
    this.login = login;
    this.level = level;
  }
  public SecurityLevel getLevel() {
    return level; }

  public String getLogin() {
    return login;
  }
}
class Session {
  static private final Session instance = new Session();
  static public Session getInstance() {
    return instance; }
  private User loggedInUser;

  public void startSession(User user) {
    loggedInUser = user;
  }

  public void endSession() {
    loggedInUser = null;
  }

  public User getLoggedInUser() {
    return loggedInUser;
  }
}

The annotation needs a property of type SecurityLevel. For now we'll use this annotation during run time. The developer declares the annotation on a method, and we want the annotated values be shown on our javadocs. Let's call the annotation @Authorise. The code for the annotation and an example class that uses it follows:

@Documented
@Target(ElementType.METHOD)
@Retention(RetentionPolicy.RUNTIME)
@interface Authorise {
  SecurityLevel level();
}

class FinancialRepository {
  @Authorise(level = SecurityLevel.SUPER)
  public void deleteAccounts() {
    AnnotationDemo.checkPermission();
    // do the real work here
  }
}

As you can see, there is some power in the declarative programming style. The author of the FinancialRepository class does not know or care about the details of how authorisation is implemented. However, he still needs to call the checkPermission() method. The checkPermission() method uses the stack trace and reflection to identify the calling method. It checks the annotation declared for that method against the logged in user's credentials. Have a look at the implementation in the code below. This code also contains a main method that shows how an authorisation error can be produced.

public class AnnotationDemo {
  public static void checkPermission() {
    StackTraceElement e[]=Thread.currentThread()
                          .getStackTrace();
    int i;
    for (i = 0; i < e.length; i++){
       if (e[i].getClassName()
             .equals(AnnotationDemo.class.getName())
          && e[i].getMethodName()
             .equals("checkPermission"))
          break;
       }
    Class<?> c;
    try {
      c = Class.forName(e[i+1].getClassName());
    } catch (ClassNotFoundException e1) {
      return;
    }
    String methodName = e[i+1].getMethodName();
    for(Method m : c.getMethods()) {
      if (m.getName().equals(methodName)) {
        final Authorise auth =
              m.getAnnotation(Authorise.class);
        if (auth == null)return;
        if (auth.level() != Session.getInstance()
                   .getLoggedInUser().getLevel())
          throw new RuntimeException(
              "Unauthorised access by "
              + Session.getInstance()
                .getLoggedInUser().getLogin() +
              " to " +  c.getName() + "." + methodName);
      }
    }
  }
  static public void main(String[] args) {
    try {
      User u = new User("Rogue",SecurityLevel.BASIC);
      Session.getInstance().startSession(u);
      FinancialRepository r = new FinancialRepository();
      r.deleteAccounts();
    } catch (Exception ex) {
      System.out.println(ex);
    }
  };
}

This sample code demonstrates how to implement an annotation at run time. Look out for places where annotations like these can be used effectively to reduce code and simplify your design.

Get traction on user interaction

2009-06-13T10:41:00.020+02:00

Like many software developers, I regard user interaction design as a subjective aspect of my work. Right and wrong seem to have little to do with user interfaces and user actions. This grayness that envelopes UI decisions does not always fit well with a brain that thrives in the black and white binary world. Nonetheless, the user interface is the only thing that other people see of our work. Also, it is very rewarding to help develop an interface that lives up to expectations and receives accolades from its users.

A far too typical mistake we make is to think of the user as a lesser being than ourselves. This is obviously not true. Users know how to do their work, and they understand the goals they want to achieve with our software. They can easily grasp the breadth of options we lay before them and when given enough leeway they are certainly able to learn how to use any application.

Once we accept our users as reasonable people, it becomes clear that there might be a list of universal expectations. This list can then be considered for any application. Yes, it is intuitive, but each item in the list that follows requires explicit deliberation when designing the user interaction for your application.

Don't make me hunt. For each view, anticipate the actions the user is most likely to take. Provide quick access to these actions from the view.
Don't make me guess. Consider what important status information is needed by the user, and make sure he in informed when that information changes. Status can be communicated using subtle elements, such as icon states and colours.
Don't confuse me. Keep the interface consistent, not only the look and feel, but also the behaviour. Avoid being too uniform: make sure that things that behave differently also look differently.
Don't waste my time. Save the user's time by combining actions that always follow each other into composite ones. Write tooltips and help text clearly and concisely. Avoid "Are you sure?" dialogs by introducing an undo facility.
Don't trap me. Give the user the freedom to explore the application safely. Ensure that it is easy return to well known views, and that also actions are revertable. A revertable action is one that can be undone by another action: e.g. the Enable XXX is revertable if there is a Disable XXX action.
Don't make me wait. When an action will take some time, move it to a thread to allow the user to continue with other things. Remember to provide appropriate feedback (to avoid guessing).
Don't make me think. Expose the conceptual model of the application clearly using appealing metaphors together with images and text cues. Choose the metaphors well and use them consistently.

User interaction design decisions are very likely to become part of the design considerations of the entire system. It is sometimes impossible to address all the user interaction issues on an older system, or one where the "server-side code" is fixed or inflexible. However, when doing greenfield development, user interaction must be discussed in detail as early as possible.

In summary, things may be less gray if you consider these seven simple guidelines. Keep on rolling out those terrific user interfaces!

Test drive your own design

2009-05-08T10:43:00.016+02:00

Agile development imposes change on the organisation, the team culture and also on the individual developers. A top-down change is unlikely to reap any benefit if the developers do not change their behaviour at the same time. Developers like me and you must grab the bull by the horns and equip ourselves with Agile methods. Without a deeper understanding of these techniques, it is difficult to motivate change or to steer the change in the right direction.
Test Driven Development (TDD) is one of the cornerstone Agile techniques that is relatively easy to learn from the bottom-up. A developer can add TDD to his repertoire independently, without help from the team or the organisation.
Contrary to first impressions, TDD is not about testing, it is about designing (read more about Agile design in a previous blog entry). As it is with many development processes, the input to TDD is a specification and the output is software. It differs from other processes in that the automated unit tests is the essence of the produced software.
The benefits that TDD brings become reachable when two assumptions are in place. The first is that unit tests are created, and the second assumption is that the TDD design approach is fully adopted. Here are some benefits you can look forward to:

Units tests allow you to verify that the software remains functionally correct after changes are made.
Running these tests help identify some non-functional issues soon after it was caused. For example, an unexpected performance glitch would be picked up when the tests take longer than usual to run.
The design is likely to be better because the code is developed for at least two interfaces (the tests is one and the calling module is the other). The developer designs code that is easy to use because he is the first user of his code.
The test set has a high level of coverage when the work is done. This increases the level freedom developers have to refactor code. Refactoring improves the design over time.
Unit tests illustrate many use cases to developers and as such the tests become an active form of a specification document. Developers look up the code in a test to learn how to use a feature, and also read through test code to gain a deeper understanding of the design.

As a development process, TDD is very simple - it consists of only four basic steps:

Create a RED test. Your first task is to create a unit test that fails. Writing this test confirms your understanding of the specification and it highlights exception conditions. You start forming an opinion about the classes and new methods you'll need. During this TDD step, you should implement simple stubs that deliberately fails your test. Keep the scope of the test very small - cover no more than one case of the specification. This step is done when the new test fails for the right reason, and when all old tests succeed.
Quick, get the test GREEN. Implement those stubs you created to get the test to pass. Use the simplest approach possible. Be sure to focus only on getting the test to pass - do not predict alternatives or allow for other cases. If you do, these cases will not have tests associated with them. This step is complete if and only if all unit tests succeed.
Refactor to improve. The main idea at this step is to revisit the implementation and make it more elegant. Now that you have a running test harness, you can confidently change the code towards a better design. But, do not get carried away, the principles of KISS and YAGNI should always be kept in mind. If you feel the urge to add another use case from the specification, it means you are ready for the next step. The first time you go through step (3), there may be little or nothing to do here. As the system grows and more cases are supported, this step becomes more interesting.
Refine and repeat. Take a new use case of the same feature and restart the process from step (1) using that case as specification. If all cases are covered, start with the next feature. If all features are covered, go get a beer: your application is complete.

Clearly, the TDD process is incremental and each increment must be kept simple by choosing a small scope. Each cycle should take two hours to implement at most and ideally no more than a few minutes.

TDD is fine when you start a new system, or when you add a new feature, but what about bugs? Even if you are not doing greenfield development, every bug presents a grand opportunity to exercise those TDD muscles. The process for apply TDD for a bug fix should be along these lines:

Create a RED test. Start by writing a functional test that reveals the bug. On an old system, creating the first test is likely to be more difficult than it sounds, but after this work, a basic framework will be in place to write more tests easily. While hunting down the bug you may create some tests that succeed. Leave those tests in place and keep on digging. This step is done when you have a failing test.
Quick, get the test GREEN. Now that one failing test is in place it is time to fix it. During this fix-it-quick period, you may develop a theory or two. Write tests for these, either debunking or confirming each theory. At some point you'll find the real cause and all your test will pass after you fix it!
Refactor to improve. You now understand the problem a bit better, and you might want to refactor your solution. Remember to avoid the refactoring of areas in the code that are difficult to test. Refactoring is good, but it is no excuse for messing up working code.
Refine and repeat. It is now time to think about alternative cases that may fail. Write tests for these and when one fails, go to the first step. If all cases are covered, its time for beer: you deserve it.

The basic TDD cycle is : RED -> GREEN -> Refactor -> Repeat. Understanding it is of little use. It is experimentation and practice that add the real value.

In short: Get TDD into your toolbox!

Align your compass

2009-04-10T10:20:00.019+02:00

Not often enough, I take a moment to look at my personal compass. From it I can adjust my heading and remain sure that I am doing the right thing. This is very much needed, especially when the wind is strong, or there is no land in sight (figuratively speaking). Recently, I came across an article written by Keith Self-Ballard that highlights the rules of success for a budding graphic artist. I feel that Keith presented a universal sense of rightness, and this inspired me to briefly discuss these three rules from a software developer's perspective:

Rule 1: Manage your ego. If you're like me, your ego could be the very thing that started you on this career. With pride I'd show off the cool stuff I created, and I'd make sure every one knows how quickly I get my work done. Bugs were my worst enemy. Wild horses could not keep me from proving that I had absolutely nothing to do with any bug. This ego thing was good. It motivated me to improve my skills and forced me to keep my eyes peeled for ugly gotchas that bite during testing. Unfortunately, the ego puppy grows up to be an ugly monster. Left unchecked, the big ego stifles your ability to grow and prevents you from reaching your potential. The developer with a small ego is always open to receive advice from others. He accepts responsibility for weaknesses in the software, and he is willing to adapt his own work (and his own ideas) for the greater good. It is definitely worth your while to become an ego-less developer. Don't worry: you'd still be writing good code, but it would be for much better reasons.

Rule 2: Communicate professionally. Bad communication is it the root of many problems; and it can litter your career with many failures. Always look for ways to improve your ability to communicate. A practical way forward, is to adapt your communication style to fit the audience.

With peers, share knowledge. Dedicate yourself to share and learn from your peers during informal interactions and during formal reviews. When you find something valuable, such as a new development technique, a new feature in your tool set or a new development in your company, share this knowledge with others. Remember that you have a limited time with each individual, because people move on to other jobs. Thus, when the opportunity arises, strive to learn as much as you can from each of your peers.
With supervisors, understand and recommend. As a developer, we face many types of supervisors. These come in many guises; architects, team leaders, development managers, and so on. With these people we share some responsibility, and they influence what we do and (sometimes) how we do it. Our primary aim must be to understand what is expected from us. Sadly, the world is not perfect - the reality is that situations arise where we strongly disagree with the supervisor. Instead of pointing out the (possibly obvious) problems with her idea, rather take some time to formulate an alternative course of action. Recommend this alternative to your supervisor as a well structured argument. Keep the interaction honest and respect your supervisor's view and also her decision.
With non-developers, change your hat. When interacting with a person from another department, or with your customer, the aim is information exchange. This information is mostly related to expectations and priorities. You can contribute in these interactions by being acutely aware of the other person's point of view. For example, do not ask the marketing manager: "What columns should I add to the costumer table?". Rather, ask: "What information is needed to help you identify potential customers for the new product?". Also, when dishing out information, do not baffle others with technical jargon (this often happens unintentionally). In short: to improve the transfer of information, wear the hat of the other person.

Rule 3: Diversify. Sometimes, we think of ourselves as UI developers, or as database developers, or maybe as C# developers. We use these pigeon holes to identify with some development tasks more keenly than with others. Generally, there are tasks that fail to attract our interest or are deemed to be no challenge to our keen intellect. Firstly, we need to acknowledge that these tasks must in fact be done by someone. Then, when such a monkey lands on your back (believe me, it will), approach it with the same level of commitment you'd tackle an absolutely worthy task. Doing tasks outside your area of core strength is an ideal opportunity to diversify. You might surprise yourself, and there is no doubt that a more diverse developer is more valuable to a company. So, diversify to improve your career prospects.

That's it for this blog. In summary, to improve your chances for success mind these three tips: trim your ego, communicate well and diversify your tasks. Simple, isn't it?

Agile design is not fragile

2009-03-22T09:43:00.021+02:00

A non-practitioner may be of the opinion that a software system should be better off after a change. But this is very seldom the case. The effort needed to implement a new feature is always less when hacked in than the effort required to redesign the affected part of the software.
The mark of a good software design is arguably the ability of the designed system to easily adapt to change. Yet, if every change weakens the design slightly, even the perfect system will gradually become inflexible over time. I cannot help but think of it as a law: any software design will naturally decay. This decay takes time; it could take months, years or even decades but decay happens.
In a paper written by Martin Fowler, he uses the term entropy to describe that property of a system that causes the decay. A system with low entropy has a slower rate of decay, and one with a high entropy has a faster rate of decay. A system developed with an up-front design starts with low entropy, and one that is slapped together haphazardly starts with a high entropy.
If up-front design is so great, why are we not doing it? Here are some issues that make this approach difficult in our industry:

The first is that the design can never contain enough detail to cover all the programming issues. This causes entropy to set in at the initial construction of the system.

The second problem is that the designer spends so much time designing that he gets no more time for coding. Gradually, he becomes unable to keep up with the changes in tools and technologies, and he could even loose respect for those who code all day long.

The third problem is that requirements change quickly. Every design is based on key requirements. A challenge of design is to decide which kind of changes you'd be able to accommodate easily. Entropy rises dramatically when requirements that introduce unforeseen changes are implemented.

There are surely situations that require up-front design, and there may be some success stories out there. But mostly, the realities of our work environments make this approach impractical. We can continue with (or start with) a haphazard approach, but I do not have the space on this blog to list all the problems associated with that! An alternative solution to face the design challenge is to start building software using practices and principles that aim to manage entropy.

Here are the key practices that relate to the system design:

Test Driven Development (TDD) means your software is designed for verification. The practice of test driven development is usually associated with unit tests, but must also be extended to the automated testing of integrated aspects, such as user interfaces. This means you do not have to pick the brain of the designers to make sure a change does not break something.

Continuous Integration (CI) refers to the practice of starting the system build and test process soon after changes are made to the source code. With this in place, a problem is identified minutes after a failure was introduced, making it possible to rectify the situation on the run.

Refactoring is the practise of making improvements on the design while coding new features. Slogans like "refactor mercilessly" can become reality only where TDD and CI is alive and well. The refactoring guru in your company is the guy that lowers the entropy in the system. He is worth his weight in gold!

The following principles have a high affinity with the design activity:

Keep it simple stupid (KISS) means that you should not complicate things because of reasons that are not 100% clear. Sometimes we implement things simply because it has worked in the past, or we want to try out a new cool technology. If there is a simpler plan that will do the job, do yourself a favour and follow it.

You aren't going to need it (YAGNI) is really saying that you should not waste effort on something that you "know" will be used tomorrow. Sounds easy, but it is not. As software engineers we tend to think that a system is better because it solves more problems than it requires. The reality is that the new feature is not yet well understood, and you are probably going to get the implementation wrong. It is not worth risking the entropy or the deadline on something like that.

It seems to me that Agile, as the proponent of these principles and practices, is a move in the right direction. It places more (not less) emphasis on design. Agile methods imply that everybody designs, and acknowledge that design happens all the time.

In short: Agile does not stifle design, it sets it free!

Spec out the competition

2009-03-14T08:59:00.028+02:00

Oh, how I wish the question "Have you ever developed software without a specification?" is a good way to start this blog. Alas, much more apt is this question: "When did you last develop software with a specification?".

How is this blatant disregard of functional specification so readily accepted by software developers? Most of them would certainly agree that specifications are very useful, and the righteous would argue that such a thing is absolutely crucial for success. The reason for this disregard seems to be that we are not worthy, our applications are too small, our company is too chaotic and so on. Like the problem of finding water on Mars, the problem of producing a specification is often side-handed as a thing that should definitely be done, especially at NASA.

I find myself ecstatically in agreement with Joel; he wrote a four part article sequence on the topic of functional specification. He inspired me to write up some of my own musings on this matter (unavoidably muddled up with the truths eloquently highlighted by the big man).

What is a functional specification? It is a body of knowledge that describes an application from the perspective of the end user. The conceptual model and the user interface aspects of the system are important aspects of this perspective. If you are technically inclined, keep in mind that the functional specification avoids describing how the application is (or will be) implemented.

It must aim to be complete, unambiguous and accurate. In addition, it should be easy to read. A tall order, I know. But, be not dismayed, a specification written by a mere mortal will prove to be worth much more than the time spent writing it down.

What happens when there is no specification? When there is no functional specification the implementation work takes longer; the application ends up with features that are less in number and less in utility; and resulting code is difficult to maintain. Here are some factors that contribute to this banal outcome:

The real problem is identified too late. Mostly things look easy on the surface, and it is in the detail the devil lurks. In some case the objective of the entire exercise can be missed. As highlighted by a (not so imaginary) friend: "We excitedly set out to solve the problem and quickly wrote an application. Now we have two problems."
Nobody knows what is really going on (and everybody hopes the programmer does). This essentially creates communication problems. The sponsor is unsure about what he is getting; the quality assurance team cannot test effectively, and the technical writer wastes his time by creating manuals that describes everything that is already blatantly obvious to anyone that saw the user interface.
The lack of information translates to lower productivity (and possibly higher irritation level) of the programmer. The constant distraction of people who want to know yesterday's news makes it difficult to focus on the creation of tomorrow's news. As the application grows in stature this overhead becomes more unbearable.

What then should be in the specification? Clearly, the elusive specification should contain information that solves the problems mentioned before. A common misconception is that a specification helps future programmers to understand the system. Although some programmer may be inclined to read the document, mostly they help themselves from the code (design documents has the potential to be absolutely fabulous - but that is a different story).

However, the specification does help the constantly distracted programmer. Instead of a hasty, cryptic comment that most likely does little justice to the application, he can now response with "Would you kindly read the functional specification?" . The specification should rise to this expectation, and be the reference document for the application. So, here are some ides on its content:

An overview that describes the utility of the application to its users. Here, some scenarios of use can be discussed to help the reader to understand the problem the user(s) expect to solve or the activity the user performs with the application as part of a broader process.
Details about the user interfaces, the validation rules, process invariants and so on. It cannot have too much detail.
A list of non-features that explains what the application does not do. These could come from discussion held at (formal and informal) meetings, where someone expected a certain aspect to be addressed by the application, but is does not. Another source might be features planned for the future, and behaviours that are viewed as unexpected by someone.
A list of open issues are very useful during the initial stages of the development. It reminds the reader of some aspects that are still under discussion.

When is the specification written? Here is the really big surprise. Nobody must wait for the specification. Those days are so over, wake up and smell with noses! Today things are supposed to be much more agile. Likewise, the specification must become agile.

The basic idea is to start the project with a functional specification that is on a high level.
Then, as the project continues, update the specification continuously.
During quality assurance, verify the specification.
And update the functional specification before extending the application beyond its initial feature set.

Who writes this thing? The problem seems to be that many organisations does not seem to employ the right guy to do this job. However, they did employ you, and my suggestion is that you should do it. It does not matter where you work in the development process - project manager, scrum master, programmer, quality assurance or even chief technology officer. Simply because no-one does it does not mean it should not be done.

Take ownership of the specification and you'll be guaranteed that it will be done. Yes, your time is limited. But, remember only a few minutes per day will be minutes well spent. For your company and for your career. Go for it!

General intelligence for games

2009-01-06T20:00:00.002+02:00

A game playing agent for Chess can play a chess game, but a general agent can play any game (or at least learn it). The idea of General Game Playing (or GGP) is to create a software agent that plays an arbitrary game. The description of the game is given during runtime. Because the rules of the game is not known a priori, the knowledge the programmer might derive from the game rules cannot be coded into the agent. The programmer's impossible mission is to build general intelligence into the agent.

The GGP competition defined by the AAAI limits itself to games that can be described by a state machine. The game has a set of states S, a set of players, P (P has size n), a set of actions A. L = S x A identifies the actions allowed in a given state. The mapping to determine the next state, M uses the current state and the action of each player:

The game starts at the state s (in S) and ends when it reaches one of the end states, t. At this point an outcome for each player is assigned from a mapping G: S x P -> [1,100]

The number of elements in any of the sets must be finite. However, this does not mean that the games have to be simple. Chess has 10^30 elements in S, but it can still be described using this simple scheme. However, to a game with this many states, a simple enumeration of states is not possible.

This is where the Game Definition Language (GDL) comes in. GDL uses a variant of first order logic. It is essentially a subset of PROLOG, and in fact PROLOG can be used to implement a game engine of sorts.

For GGP there are some constraints placed on the described game:

The game must be playable. This means there must be a legal move for every player at every state.
The games with one player must be strongly winnable. There must be some sequence of moves that maximises G.
The games with more than one player must be weakly winnable. This means there must be a path that considers joint moves from any state such that each player's goal is maximal (in G)

In a single player game, it might be possible to search the game space for a goal state, but in a multi player game, the moves of the other players cannot be determined. Typically, an exhaustive search likely to be out of the question. The problem with heuristic searches is that a heuristic cannot be defined for a general game without knowing the rules. In addition, games dot not have names -- in order to recognise a previously played game, the agent must first assess whether this is in fact the same game. Is is also not possible to learn the strategy of another opponent because the opponents' identity is not shared.

In order to implement the challenge a mediator called Gamemaster is available as a web service. Using a specifically designed protocol the Gamemaster controls the game and gives each player a chance to move. If he does not respond in time, a random move is chosen for that player. At the end of the game, each player receives his reward (calculated from G).

Anyone up for a small competition?