Saturday, September 30, 2006

Qwicap Turns Three

Qwicap, my pet project for simplifying the development of Java web applications by allowing them to be structured much like conventional applications (for "conventional" think of interactive command-line apps), turned three years old today.

Actually, it's more like 3.4 years, total, but three years since version 1.0 was released. The releases were internal to U.T. Austin until my boss, his boss, and her boss, all agreed to open-sourcing the product. Then there was the matter of figuring-out whether it could be done at all, given that nobody in my department (the IT dept., no less) had ever done it, or knew of policies that would permit it. Ultimately, I found the lawyer at UT System who dealt with these matters, and discovered that, contrary to all expectations, we actually did have a policy that allowed—and even encouraged—open-sourcing. After reading the policy document countless times, and having many conversations with the ever-helpful lawyer, I even managed to figure-out how to apply the policy to my situation. With that done, there was a lot of paper-work to be filled out, months of waiting for a critical signature while I frantically polished the code and wrote documentation (while also doing all of the things they normally pay me to do), the filing of the paper-work, and, then, figuring-out how to use SourceForge to publish the code.

Finally, Qwicap was posted to SourceForge, and I began trying to figure-out how to make people aware of it. After a few anxious days I was overjoyed to see the news item announcing its release on the SourceForge main page. Wow. All those years of work would not live, die and vanish within the confines of The University. It was actually out there.

That news item lasted three days, and the response to it was deafening, in the same sense that silence can be deafening. Ouch. So, there followed efforts to get Qwicap listed on more topic-specific sites. At the recommendation of colleagues, I started-up this blog, much as it goes against my grain (twenty years ago, it would have been a different matter, but that was then). At the recommendation of my boss, I submitted a presentation proposal to the regional Educause conference. And, of course, I kept working on Qwicap.

The work on Qwicap has gone well. There were a number of 1.3 revisions, then work began on 1.4, the biggest update yet. Several test versions have been released, several of my colleagues have contributed improvements to the project (Kevin, Jay, thanks), and work on it is nearing completion. And we've deployed three more major campus web applications that are built on Qwicap, with very positive feedback from the development team that built them. Most of the team members have baselined Qwicap for their upcoming projects. Since Qwicap's goal was to make life better for developers, that has all been music to my ears.

In stark contrast, the work on publicizing Qwicap has failed miserably. The topic-specific sites ignored it. The blog, which lead-out with my "Programmers Hate Programmers" hypothesis, generated some attention, but if anyone actually did any work with Qwicap as a result, I've never heard about it. The regional Educause deadline came and went without a word, but eventually I was informed that they might be inclined to use the presentation as a backup, in case one of the presentations they actually wanted fell-through. Shortly before the deadline, I swallowed hard and agreed to be a backup, but that never happened. (On the plus side, I never had to write that presentation.) When the national conference came along later, I never quite managed to submit the presentation proposal again. 'Should have, of course, meant to, but still felt the bruises. (On the plus side, I still haven't had to write that presentation, let alone present it.)

I keep an eye on the Qwicap web site, and find that it continues to generate some attention, but most of it relates to the example source code for Java number-guess applications (though not the Qwicap version, naturally). I presume that some people find the examples useful as the interactive equivalent of a "Hello, World" program, but I'm also inclined to suspect that it's the answer to some professor's CS 101 assignment. It's not quite the contribution I'd hoped to make with Qwicap.

Well, Hell.

Last time I looked, there were about 100,000 open-source projects on SourceForge, alone. It seems pretty clear that most of them have utterly failed to make an impression in their niches of the industry, so, statistically speaking, it's no big surprise to be in the same situation. On the other hand, more than a few of those projects seem to have skipped the unpleasant "writing the code" phase of project development, so I kind of thought Qwicap had an edge in that respect. Shows you what I know.

Ironically, when I first proposed open-sourcing Qwicap, I'd been concerned that the product might generate enough interest that I'd have a hard time keeping up with it, so I'd warned my bosses that it might take time away from my usual duties, but they were supportive, nonetheless. Thank you, bosses. [Now there's something you don't hear every day.]

That concern wasn't just wishful thinking. In the late '80s and early '90s I built and maintained the "Gatekeeper" anti-virus system for Macintosh. It was freeware, and developed on my own time. It broke new ground in its field, and achieved a significant level of success; postcards from users arrived from all over the world, those being the days before most people had even heard of email, let alone the Internet. The people who could use email had nice things to say, too, but satisfied users are the quiet ones. The ones you hear from the most are the folks who aren't altogether satisfied; they want features added, or bugs fixed. I encouraged the feedback, and especially the bug reports, of course. Can't fix 'em if you don't know about 'em, and every bug is embarrassing. But, if you haven't had this experience, imagine looking at your email every day for years and finding what comes to seem like nothing but problem reports. Many come with kind words about the value of the product, but the praise is ephemeral, while the worries and the bug hunting and the other work grinds on and on, in every spare moment you can find.

And it wasn't even the case that the Gatekeeper system had a lot of actual bugs. But it seemed like there were, because Gatekeeper had to be implemented as a set of complex, runtime patches to the Mac operating system, and my patches had to coexist with a startling variety of other people's patches, and with all of the undocumented idiosyncrasies of an operating system that was "quirky" on a good day. In practice, the problem-space was limitless and the problems often unreproducible, and uncorrectable. Of course, each bug would be reported many times, and with every version of Gatekeeper (and every OS patch, and every OS version) ever released out there running on somebody's computer, somewhere, they never really went away, even if they'd been fixed years before. And I wanted to represent my handiwork well, so, naturally, I tried to respond to everybody.

After enough years of dealing with all that, day in, day out, I couldn't bear to read the email anymore, or open the letters, or read the postcards. It seemed like there was never any good news, but an endless supply of bad news, all of it directed to me personally, and awaiting my reply. It was relentless, and endlessly depressing. All those years of work, and that was what I had to show for it. The many successes—the product did what I set out to do, and more—couldn't compete in my awareness with the cumulative pounding of the bad news. I had to walk away. Some people were annoyed when their mail went unanswered. I wasn't happy about it, either. Some told me they felt betrayed, which hurt, but walking away had become a matter of survival. (And the product was free, after all. What did they want? Blood?)

The burnout was intense. There came a day when I was contacted by a development house that wanted to turn Gatekeeper into a commercial product. On the one hand, I didn't want to deal with it ever again. On the other hand, it felt like it would be a mistake to have gone through so much and not at least look into this possibility, so I met with a local, commercially successful Mac developer in order to talk about what sort of money I could reasonably expect out of commercializing a product like Gatekeeper. He told me that over something on the order of three years, a product of its kind should be worth about a million dollars. It was an impressive number (still is), but even so I couldn't bring myself to go back to that work.

It felt like about ten years before I could find the will to take on another major programming project, another big idea. So, naturally, I worried about whether I'd go through the same thing again trying to support Qwicap. Turns out that it hasn't been much of a problem. What luck.

One of the nastier traps one can step in, I suspect, is to discover the burn of a really great idea. The kind of idea that you can't not work on, no matter what anybody else thinks of it. At that point, the idea has you right where it wants you, and you have to go where it takes you, while paying mightily for the ride. Great things may come of it, they may make it all worthwhile in the end, but there's only room for so many great outcomes in the world, so the odds can't possibly be in your favor. But you go on anyway.

Maybe Qwicap will make a difference outside The University one day. So far, not so good. Oh, well. I'm not done with Qwicap, and it's not done with me.

'More than I'd intended to say, but I assume I'm talking to myself at this point. If not, good luck with your own great ideas. And have a look at Qwicap.

Wednesday, September 27, 2006

The Screech Owl Nest Box Cam' Temporarily Returns

Temporarily featured on my screech owl nest box cam': A family of fox squirrels. There's mom, and three pups. The pups stay in the nest all of the time at the moment, while mom is out most of the day inhaling the contents of my bird feeder to make enough milk to keep the pups satisfied. The pups appear to suckle in their sleep, so it's surprising she can meet the demand.

The pups were not born in the nest box; Mme. Squirrel moved them to the nest box a few days ago, presumably in response to rain and/or our increasingly cool weather. Past experience suggests that she will rotate the pups among several nest sites, but for the moment they're in my nest box (stop that gnawing!) and therefore observable.

I don't plan on providing daily commentary, or resuming any of my other owl cam'-related work, but you can watch the critters live. My preferred way of watching them is this five-up view. Other viewing options can be found on the nest box cam' main page.