Monday, 12 January 2009

Data/Object Anti-Symmetry

I’ve been reading Robert Martin’s book Clean Code. This is an important book because, almost uniquely, it tries to improve the quality of software from the bottom up. It doesn’t tell you how to improve your system design or management processes; it tells you, for example, how to lay out your code.

It’s not perfect, though. Some of the examples are unconvincing – Dave Cleal has already written about one poor example. I want to focus on another.

Chapter 6 Objects and Data Structures has a section entitled Data/Object Anti-Symmetry that argues that a procedural style of programming, where the data structure is separate from the functions that act on the structure, is sometimes more appropriate than an object-oriented style, where data is hidden behind interfaces. Taken at face value this is undoubtedly true, and Uncle Bob goes on to discuss the very common and frequently justifiable case of Data Transfer Objects. The specific anti-symmetry claim (pg 101) is:
Objects expose behavior and hide data. This makes it easy to add new kinds of objects without changing existing behaviors. It also makes it hard to add new behaviors to existing objects. Data structures expose data and have no significant behavior. This makes it easy to add new behaviors to existing data structures but makes it hard to add new data structures to existing functions.
However the example used to support the anti-symmetry, based on manipulation of different shapes, seems to me a perfect example of a situation where you’d nearly always favour the object-oriented approach. In the procedural solution (pg 95) the function to compute the area of a shape looks like this:
public double area(Object shape) throws NoSuchShapeException {
if (shape instanceof Square) {
Square s = (Square)shape;
return s.side * s.side;
}
else if (shape instanceof Rectangle) {
Rectangle r = (Rectangle)shape;
return r.height * r.width;
}
else if (shape instanceof Circle) {
Circle r = (Circle)shape;
return PI * c.radius * c.radius;
}
throw new NoSuchShapeException();
}
Note that this function contains, effectively, a switch statement that selects between all the available shapes. Every other function that operates on the shapes will contain a switch statement with an identical form. This is a bad code smell that Martin himself criticises on page 37. In that critique he advocates – wait for it – having a single switch statement in a factory method that creates objects of the appropriate classes and then using polymorphism to access the required behaviour. If you applied that transformation in this example you’d end up with… the object-oriented solution!

If I decide I’m happy with the duplicated switch statements and stick with the procedural solution then to add, say, a perimeter() function I’ll actually write more lines of code than in the object-oriented solution because the polymorphic dispatching replaces the switch statement. In what sense, then, does the procedural approach make it “easy” to add new functions?

One important difference between the two solutions is in the scope of the change. When I add the perimeter() function to the procedural solution my change is all in one place, whereas with the OO solution the change is spread across multiple shape classes. Martin acknowledges this when he says (pg 97) “OO code makes it hard to add new functions because all the classes must change.” So perhaps for Martin “hard” means “touches lots of software entities” and “easy” means “touches only one software entity”. If that’s the case I have a little more sympathy with his position, but not much. Of all the evils in code, duplication is perhaps the worst, a point made several times in Clean Code. I’m prepared to pay the price of having to touch multiple classes in order to eliminate those evil switch statements.

In fact there’s no need to pay that price. If you want a procedural style – because you foresee that adding new functions is more likely than adding new structures – then the best way to achieve it is by using the visitor pattern, as Martin himself points out in a footnote. So why didn’t he show us a solution based on visitors? It’s a puzzle.

In truth I hesitate to criticise Clean Code at all because its heart is in exactly the right place and I hope everyone who writes code reads it. I criticise only in the hope that the second edition is even better.

Tuesday, 6 January 2009

Am I a Master?

Several people have observed, with a snigger, that I am the only person registered for the forthcoming Software Craftsmanship conference who is listed as a Master. Ade Oshineye has gone further by writing “Anyone who seriously claimed [to be a master] would suddenly find themselves having to explain why they were better than everyone around them. Someone could attempt it but they’d need a lot of ego and a diminished capacity for self-doubt and self-awareness.” (original here)

Gosh. The main reason I listed myself as a master is that Jason Gorman was kind enough to describe me as one in his promotional material for the event. Seriously, though, should I be prepared to call myself a Master of Software Development? It’s a tricky question because traditional craftsmanship is not an accurate parallel to what I do at work. As Oshineye points out, there’s no agreed way of determining what mastery of software means. Like many of the analogies applied to software development, a consideration of “craftsmanship” can improve our understanding but it is misleading to assume craftsmanship – or any other analogy – will provide a complete and useful model. Software development is like… software development. Perhaps the most useful insight that comes from comparing software development with craft is the realization that apprenticeship might be the most appropriate way of learning a set of poorly understood and rapidly evolving skills.

If I really am a Master I should be able to point to my masterpiece. But, as is the way with most software development, all the successful systems I’ve been involved with have been team efforts. Even my books have been co-authored. So if I had to go before my peers to argue my case as a Master, what would I say?

I think the most important considerations in this assessment are whether your peers recognize that you have:
  • advanced the body of knowledge of the field
  • made efforts to pass knowledge on to others
  • carefully and consistently met high standards in your own work.
Having defined the criteria of a Master for myself I feel confident that I can meet them (how convenient!). I won’t risk further accusations of self-aggrandisement by listing my claims here, but you can read them on my web site if you want.

As for the third category, I’ll leave it to the people I’ve worked with over the years to decide.

Tuesday, 30 December 2008

Modelling with a Sense of Purpose

Steve Cook and I, in our 1994 book Designing Object Systems, were the first people to set out clearly the different purposes of object models, especially the distinction between a model of a situation (sometimes called a model of the world) and a model of a software system. It seems ridiculous now, but back in the early 1990s many people believed that models of situations in the world could just be treated as software designs if you squinted a bit.

I say it seems ridiculous, and I thought the whole issue was settled back in 1997 when Martin Fowler explicitly endorsed our approach in UML Distilled, but I was disturbed to hear from Keith Braithwaite recently that he still encounters many developers who have been taught UML without grasping this essential point.

There has been a backlash against modelling driven by the wastefulness of many large projects during the last fifteen years. These projects valued models over code and paid the price. But there is still a huge value in having a language at a higher level of abstraction than code that developers can use to organize and communicate their thoughts. The UML - for all its faults - can be this language, but not unless we teach people how to use it properly. Understanding the different purposes of models is absolutely key to this. How can we make sure this subject is properly covered whenever and wherever UML is taught?

Sunday, 28 December 2008

Is Craftsmanship All About Code?

Much of what has been written about software craftsmanship has focused exclusively on code quality. Is it possible to be a software Master Craftsman if you don’t write code?

It is undoubtedly true that we need urgently to improve the quality of code, and we ought to extend our value set to include this aspiration (see for example this blog for some ideas on that). It is also true that code, in its various forms, is the long-lasting deliverable of most software projects. But the cleanest code in the world has no value if it fails to solve the business problem at which it is targeted.

Even for those of us whose primary output is code, mere mastery of its construction is not sufficient. My contention is that there are at least two other facets to an all-round Master of Software: the social skills discussed in my previous article and the ability to harness the power of code – our own and that of others – to solve problems. And while I’d be deeply suspicious of anyone claiming to be a Master of Software who had not at some point demonstrated exemplary coding skills, I think we have at accept there may be Masters who don’t code.

Wednesday, 10 December 2008

Inside the Mind of a Craftsman

I know some really skilled software developers who are not good craftsman. I want to understand why.

My definition of a good craftsman is someone I want to work with on a daily basis. It helps if they can write great software but that isn’t the primary attribute I’m looking for. Here are some characteristics of good software craftsmen:
  • They do what they promise, and they don’t promise what they aren’t sure they can do. Some people take on way too many tasks and then fail to deliver properly on any of them. One reason someone takes on too much is because they are scared to say no, but a more worrying reason – common among poor craftsmen – is that they are very bad at judging how long things will take.
  • They are well organized. Some people are naturally good at keeping things straight, and these people inspire confidence. Well-organized people can take on parallel tasks without compromising the outcome. Poorly organized people are sometimes good craftsmen but only if they have recognised their lack of organization and dealt with it by taking on only one task at a time.
  • They can tell me why they are doing things in a particular way. Donald Schön maintained in his landmark 1983 book The Reflective Practitioner that the best professionals know more than they can put into words. Well, he may be right, but software developers who can’t explain why they are doing something, or at least take the time and trouble to consider the reason, scare me.
  • They are very particular about the detail. There may be exceptions, but it’s hard to imagine someone being too anal-retentive to be suited to a career in software. Machines are very unforgiving to minor lapses in accuracy and a good craftsman treats them with the respect they deserve. The real skill, though, is in understanding which things require detailed attention and which don’t.
  • They are disciplined. If there’s a defined process they always follow it; if there’s not they make one up and follow that. Many of the processes we follow as software developers are not written down but that doesn’t make them less important. A good craftsman respects the processes because he or she knows they are there for good reason. Conversely, a good craftsman won’t put up with poor processes but will tirelessly seek to improve them.
  • They aren’t afraid to say they don’t know, and are always willing to learn. Admitting you don’t know something requires maturity, and maturity is a characteristic of good craftsmen.
  • They work at a pace I can stand. During my career I’ve met a handful of people who I found impossible to work with for long because their minds operate at a much faster pace than mine. These have generally been interesting, brilliant and likeable folk, but never have they been team players: other people just can’t keep up. These people often have difficulty answering questions about why they are doing something because they have already moved on to the next task and are now devoting their not inconsiderable intellect to solving that. For me, people who progress too slowly are nearly as problematic, even if they are skilled at getting a good end result. I’m not very patient, but I am working on it.
  • They can appreciate the implications of their work at different levels of abstraction. Good craftsmen always have in mind the end goal, the big picture, even when working on the tiniest of details. The implication of this, of course, is that good craftsmen have a deep understanding of abstraction as a tool.
  • They invest time in their tools. All good craftsmen know that they need tools they understand and can trust. So good craftsmen never cut corners on tools – they use they best they can get, and take the time and effort to learn how to use them properly.
My main point here is that it’s possible to have complete mastery of the technology, to know and use all the latest design techniques, and still be a poor craftsman.

Can you learn to be a good craftsman? I’d argue that a good number of the characteristics on my list can’t be learnt, at least not once you’re an adult. The obvious conclusion, then, is that some people can never be good craftsman.

So what can we say about people who have these characteristics? First, it’s likely they have been developing software a fair while – universities don’t turn out craftsmen. Second, they have reasonable inter-personal communication skills. Lastly, and most importantly, they care about their work. You never met a craftsman who didn’t care.

Tuesday, 9 December 2008

Power Management on the Sun SPOT

It seems to me that a driving principle for all software tools should be: "Let the developer concentrate on the problem, and let the tool take care of the incidentals." I call this principle let-the-tool-take-the-strain (L4TS). The application of this principle to power management on the Sun SPOT, a small wireless sensor network device programmed in Java, was far from straightforward. In this article I want to discuss the principle, how it was applied, and why the result isn't universally liked.

It is the L4TS principle that led to the development of high-level programming languages, and to the gradual acceptance of automatic memory management in languages such as Java. The same principle has applied in the development of operating systems. Long ago programmers wanting to store data on disks had to worry about sector allocation, now we take it for granted that the operating system will manage disk space for us.

I was determined to apply the L4TS principle ruthlessly in the design of the SPOT libraries and system software because our brief was to make the SPOT easy to program. One area where I felt the principle should be applied was power management. The SPOT is a pretty powerful computing device, and it can use up its battery in a few hours if it runs continuously. Since we wanted SPOTs to be able to go for weeks or months between charges it was clear that some fairly sophisticated power management would be needed.

Hardware support for power management
The SPOT uses the Atmel RM9200 processor package, which is based around the ARM9 processor. It's possible to put this package into a low power mode - called shallow sleep - where the processor clock is stopped; it restarts when an interrupt occurs. The problem is that the power consumed in this mode is still too great for the battery to last for months. A typical SPOT application that requires long battery life is actually only active for very short periods. To support applications like this the SPOT has some clever hardware tricks. It uses a separate low-power processor, the power controller, as an alarm clock. Code running on the main ARM9 processor can request a wake-up call and then ask the power controller to turn off power to most of the SPOT (but not the RAM). Consumption then drops to a level that the battery can sustain for months. This mode is called deep sleep.

Automatic or manual?
Should we try to automate power management or should we put it under the direct control of the application developer? To automate it implies that the system software - the Java VM on the SPOT, since the VM is implemented directly on the hardware with no intervening operating system - must determine when there is no work to do and select the best possible power-saving mode. Since the only processes running on the SPOT are Java threads the VM is in an excellent position to determine when there is no work to do: when there are no threads ready to run. But it's a lot trickier for it to determine which power-saving mode to use, shallow sleep or deep sleep. To do that it must understand which I/O devices are active, because a SPOT in deep sleep cannot respond to most external stimuli (for example the radio is powered-down in deep sleep).

By contrast letting the application manage power is simple. The application knows what it is doing and whether it is interested in receiving interrupts while sleeping. So it can simply request the appropriate sleep mode. There are three drawbacks to this:
  1. The application writer needs to learn about the power management mechanisms and write code to use them.
  2. The result might not be optimal. The application writer might miss opportunities to use deep sleep.
  3. It doesn't work if there are two or more separate applications running on the SPOT. Early releases of the SPOT SDK supported only one active application at a time, but more recent releases support multiple applications. There is a way around this: the system software could arbitrate sleep requests from the applications and only enter a mode that is compatible with them all.
It was clear to me that there were many advantages to automatic power management, and that's what is in the current SPOT SDK, but it did result in significant complexity and, somewhat unfortunately, has been a source of confusion among application developers.

Automatic power management implementation
Here's how it works. Whenever the VM thread scheduler (written in Java) determines that no thread is ready to run in the near future it invokes the sleep manager. The sleep manager then checks whether any hardware devices are active and if not forces the SPOT into deep sleep, otherwise it just uses shallow sleep.

Every device driver on the SPOT - they are all written in Java - must register with the system. To determine whether any devices are active the sleep manager iterates over all the registered drivers asking each in turn whether it is okay to enter deep sleep. If all the drivers agree then the sleep manager uses deep sleep. This call also gives the driver the opportunity to copy any transient device state, such as the contents of device registers, into Java variables. Since RAM remains powered in deep sleep the state of Java objects is preserved. The sleep manager iterates over all the drivers again as the SPOT leaves deep sleep so that the devices can be reconfigured.

This implementation provides automatic selection of the best sleep mode based on the state of the I/O devices. So although the application writer doesn't need to worry about explicitly requesting power saving he or she does need to make sure that devices not in use are turned off otherwise they may inhibit deep sleep.

The biggest issue is with the radio receiver because applications rarely access it directly; instead applications open virtual connections and read and write data from and to them. These connections transmit and receive data over the radio. In line with the standard approach, the radio driver inhibits deep sleep whenever the radio receiver is on, so the application developer needs to manage connections carefully by attaching the appropriate policy ("receiver always on", "prefer off" or "prefer on") to each connection.

You don't like it?
At this point you may be asking "Why would anyone have a problem with such an elegant approach?" One problem stems from the implicit nature of the approach: the application doesn't request deep sleep, it merely has to ensure that everything is in the right state for it to occur. Some application writers feel a loss of control, feel their lives would be simpler if they could just tell the device what to do rather than having to coax it. There are two ways to address this concern. First, it's important that the defaults are chosen in a way that, for most applications, optimal power management occurs with no extra effort on the part of the developer. To a large extent I think we achieved this, but as the complexity of the SPOT system code increases with each release it's vital that SPOTs remain configured out-of-the-box in a way that ensures a simple application of the form
public void startApp() throws MIDletStateChangeException {
Utils.sleep(10000);
notifyDestroyed();
}
will always cause the SPOT to deep sleep.

The other way to address application writers' feelings of a loss of control is to provide tools and instrumentation that make it easy to tell that the SPOT is doing what you want and expect. We provided APIs that allow applications to check that deep sleep was happening but that didn't really help. The problem is that having used the API to check, the application needs to communicate the result to the developer and the obvious ways of doing this - displaying a message over the USB cable to the host PC or sending a radio message - themselves will inhibit deep sleep! So we have to fall back on telling developers to watch the power LED, which flashes in a distinctive pattern as the SPOT enters and leaves deep sleep. Hardly a foolproof reassurance. This isn't a problem we anticipated but we should have done.