The "Programmers Hate Programmers" Hypothesis

I'd like to offer my fellow computer programmers the following hypothesis, which I've been kicking around for a few years: Programmers hate programmers. To explain: This hypothesis contends that the best explanation for the torments that programmers all-too-often lavish on one another with the software they create (especially libraries and APIs) is that programmers hate other programmers and therefore they design their creations to screw with each other. This hypothesis is obviously wrong, because, among other things, we could certainly screw with each other far more effectively than we do, if we were really trying. Nonetheless, at first glance, it seems to explain a lot of our experiences, and that's what I think is useful about it.

So, by way of example, this hypothesis seems to explain why...

• ...someone's code presents you with a "file not found" error, but doesn't tell you what file wasn't found, or where it was looking for the file.
• ...someone's code carefully checks a host of conditions to determine whether it can proceed with your request, and, finding some condition not met, fails, but won't let you in on the secret of why it failed. (In extreme cases, it won't even tell you that it failed.) It knew why, of course, because it went to the trouble of meticulously checking; it just didn't believe that you had need-to-know.
• ...someone's code does its error checking/reporting in such a way that the errors it reports have nothing to do with the actual problem and therefore—far from assisting you with debugging—they actually send you down the wrong path entirely.
• ...someone's API is so abstract that even after you learn it, you still don't know what it does, or, alternately, having learned what it does, you can't seem to make it do it.
• ...someone's system saves you from having to waste ten or twenty minutes writing code by forcing you to spend a week figuring-out how to write their config files, metadata, etc. (Hmm... what's the one thing my fellow programmers do best?.... Well, we can't have them doing that.)
• ...someone's API makes routine, simple operations lengthy and tedious, or makes implementing common, seemingly trivial, features, into major coding efforts, even though they had to know that everyone who used their API would be doing those sorts of things regularly. (Perhaps programmers who grow up with abusive APIs are destined to abuse their own APIs.)
• ...someone attempts to solve problems stemming from complexity by adding new layers of complexity.
• ...some APIs look as if they were designed exclusively to satisfy a feature check-list, with no concern for how the elements of the API would work together in practice.

There are probably a lot of other good examples that would resonate with much of the programming community, but I seem to have blocked from my memory the others that I've encountered. I leave extending the list as an exercise for the reader. The important thing about this hypothesis is that when you, the developer, find yourself in a situation where it seems to explain what you are up against, or it might describe another developer's experience with your code, something is wrong. Think of it as a diagnostic tool.

A few of those problems originate with judgement calls with which I've come to disagree. I think most, intentionally or otherwise, are examples of externalizing development costs. In other words, to save themselves a day or week of hard work, developers will cut corners in, or under-build, their code, and, having done so, they'll expose hundreds or thousands of other developers to hours of extra work. So, the original developers save a day or a week, but they cost the community months or years... the larger the community, the greater the damage.

Writing Non-Conformant Code

In this context, the best thing that can be said of an API, library, or other body of code, is that it does not conform to the Programmers Hate Programmers hypothesis, and I have some ideas about how to achieve such non-conformance.

I spent about 15 years writing Macintosh software, and I remember that Apple's engineers used to stress that the most important part of a Mac application was its user interface (GUI), since it's the GUI that defines an application to its user. In order to produce the best possible GUIs, they advised developers to design and, if possible, user-test and refine, their ideal GUI before writing the first real line of code, and then engineer the code around realizing that interface design, because a great many implementation details would be dictated by the requirements of the interface. In other words, the core code must be designed around the demands of the GUI, rather than the GUI being designed around it. Doing otherwise tended to create GUIs that interacted poorly with the user, and were clumsy and/or difficult to use.

While it might have been more honored in the breach than the observance, I believe they had the right idea. Over the years, I have also come to believe that the interfaces provided by GUIs, vital as they are, are not nearly as important to the world as the interfaces provided by APIs, because the effects of bad APIs propagate up into all of the code built on them, including GUIs. In extreme cases, bad APIs will even prevent code from being developed at all by making it appear effectively impossible, or so costly that the coding projects end-up abandoned. In other cases, potentially valuable projects won't be started at all due to past experiences poisoning the developer's cost/benefit guesstimate. (And then there's the matter of programmer burn-out....)

Once we conclude that the quality of APIs is profoundly important, the same advice that made sense for GUIs comes to make sense for APIs (and all aspects of the "developer experience" provided by a body of code): Design something as close as possible to an ideal API first, and then engineer the underlying code to make that API possible. If possible, observe how other programmers interact with the API, and refine it to be as intuitive, comprehensible, convenient, and simple as possible.

Bear in mind how your code is likely to be perceived. For instance, I suspect that many programmers (myself included) tend to conflate the number of lines of code they have to write in order to perform a particular task with the efficiency of the resulting implementation. If it takes a lot of lines, it's likely to be perceived as being less efficient than something that performs the same task in one or two lines. We all know that such a perception has nothing to do with runtime performance, but it can have a lot to do with how we perceive the effectiveness of an API, and our efficiency as programmers while using it. So, convenience is good, and is often worth a hit at runtime, if it's convenient enough. If reversion to a lower-level, but more efficient, API is possible in the optimization phase of development on an as-needed, piecemeal basis, so much the better.

Some APIs may not be able to represent every last feature that their underlying code could supply (or, as mentioned above, they may have to trade absolute performance for developer effectiveness). I believe that it's appropriate to offer both high- and low-level APIs. The high-level API should make, say, the most commonly-used 80% of the functionality as convenient to use, and as easy to learn, as possible. The low-level API should make all of the functionality available as best it can. Ideally, it should be possible to freely mix usage of the two APIs, so that the client code can enjoy the benefits of the high-level API, without foreclosing the option of later embracing, on a piecemeal basis, the more esoteric, or efficient, features exclusive to the low-level API. Even the low-level APIs should be designed to provide the best possible developer experience, but the high-level API should be designed first; its needs are likely to impose some requirements on the design of the low-level API.

(High-level APIs built over pre-existing low-level APIs can still be valuable, but they are in danger of having to adopt severely compromised internal or external designs, because they can't completely work-around the fixed low-level structure. Put another way, the kind of building you can build is severely restricted by the foundation it has to sit on. In order to have flexibility in the building's design, the foundation must be designed around the building's needs. Otherwise, you may be forced to build a shed where you needed a home.)

Criticisms and suggestions are easily produced, of course. If they aren't backed-up by code, they're too easy. For a couple of years, beneficially interleaved with other projects, I've worked on a Java web application development API designed on the aforementioned principles. It's called Qwicap. It's open-source. You can begin using it after reading just one or two (longish) pages of introductory documentation. Most of its API is confined to just two classes. It's a clean-slate design intended to eliminate most of what I perceive as the common tedium of web app. development. It works to fit into the natural flow of your code, rather than forcing you to fit your code into it. We use it in production at The University of Texas at Austin. And, yes, I'm trying to draw attention to it. With more than a hundred thousand projects on Sourceforge, alone, and countless web application development schemes available, it's hard to get noticed. It's even hard to know whether anyone has already done something like it. Much as I detest self-promotion, it seems I'm going to have to try my hand at it. Wish me luck.

That Remindes Me...

If you haven't already seen it, Richard P. Gabriel's book Patterns of Software (free PDF, 1.2 MB) is well worth a read. The problems of abstraction, and the idea of code habitability, are two good topics from the book that spring to mind. I first became aware of him through an article on Sun's site, which is a quick read and also worthwhile. Finally, if you happen to be an old Mac programmer, the book "Revolution in the Valley", by Andy Hertzfeld, is a treat. The web site of Mac folklore from which it was derived includes more material, but the book is a thing of beauty.