As a UI designer, I'm very aware of the cost of user interaction with a bad UI.. but it doesn't hurt at all to refresh on the details from time to time by listening to a good talk.

Today I was pointed to a video by a guy who really knows his stuff. He has an interesting take on the way we command our machines to do things - rewinding the clock to the command line - but pushing it forward to the 21st century by making it semantic and context sensitive.

"The Death of the Desktop" is an interesting listen and there aren't too many things in there that I found offensive. He even referenced a good document on how one guy redesigned the BART transit system schedule application.

What was interesting is this idea that keeps cropping up - context sensitive search engines for commands, not content. The search engine we're all familiar with is one that finds us a document or spreadsheet or bookmark that we've previously seen or heard about or is out there on the web. There's no action involved here except "find me what I'm looking for". Instead, search engines like QuickSilver and Enso (the presenters application) find you a command to run, not content - they assume you already have content and want to do something with it.

But what? The common commands we're all used to are often available in menus, right click context menus or by shortcut keys. But what happens when you get too many commands that you can't reasonably fit them in to menus or giant icon toolbars. Microsoft's answer was to redesign the toolbar so they could pack more stuff in to it... the Enso and Quicksilver like answer is to bring the power of the search engine to the command.

This is not a new idea - most advanced CAD software these days does it too.. CAD software has so many commands it'd be impossible for them to put them all in to menus. And for us programmers who understand Domain Modeling, we also understand that the commands we send to our objects are our ontology and are thus like commands too. Intellisense, in a way, is like this too - you start typing a command you want to send to an object and it pops up the possible combinations that you might be after. The difference here is that you're running on "instances" of objects.. you're going to do the command -now-, not writing a program. So this is an end user command search engine.. very interesting, it's the command line on crack.

The problem with this kind of user interface is one of ontology and faith. The first problem: ontology. You call it silverware I call it cutlery.. terminology.. someone has to build a big thesaurus of commands to help you all do the same thing when you call it something else. The presenter believes that a big fish in the small pond will set the standard for most common commands and the outliers don't matter so much. In many respects, he's probably right - we all know what cut, copy, paste, undo and redo mean.. but should we? Have we just habitualised those words?

The second problem is one of faith. You have to have faith that you're dealing with Orac, not Oracle - in otherwords, if I ask it to spellcheck my text, it will actually know how to do that.. I have to have faith that if I ask the computer to do something, I won't be wasting my time.. for example, if I say "Spellcheck" .. i assume it will work.. but how am I to know if I write "sum it up and chart it over the fiscal year" won't work when it seems so reasonable to me? Know what the system can and can't do is now pretty much impossible.. at least with menus you knew when you were beat - if it wasn't in the list, it couldn't be done.

Finally, there's the obvious plight.. too much typing right? shortcuts are natural and you'd assume that this sort of interface would let you link up shortcuts too. You'd also expect industry standard ones... but still.. all that typing when often most things on my Mac are one or two clicks away. Some middle ground has to be found - we can't just throw away the last 10 years of desktop design because it doesn't scale. What I found interesting is that while the presenter was so knowledgeable in UI design.. he failed to realize what the desktop was for.

The presenter mentioned the cognitive rules of UI design. A person can handle 7 +/- 2 things at once. What the desktop is for is to custom the most common things a user does, 7 +/- 2 of them, right in front of them so they don't have to go fishing. Now, let's assume that people have too much they need to keep in their heads and the desktop really can't scale for that - sure, then this sort of command approach is paramount. However, you'll still always have a set few things you always want to do and i very much doubt you'll want to write the command out each and every time. Some people just don't type that fast.

However, negatives aside, it's great to see more of this sort of research being done and tested in real world scenarios. The video demonstrates how easy it is to drop a google map into a wysiwyg editor in a web browser using his software without having to modify the editor at all.. that's impressive and clearly demonstrates the ultimate goal - better transparent reuse across applications.

Interestingly enough, that sort of reuse he demonstrated there can really only be pulled off in a web browser right now. When was the last time you saw a live google maps widget dropped in to a PDF or word document? -- it is quite telling that the web browser is once again letting us do things we all expected the desktop to let us do years ago. Why is it that the VHS wins over the BETA so often?

Well, for us Smalltalkers it doesn't matter so much, we're not a beta and we're not a VHS, in fact, we're the guys recording the videos.. so we can happily produce beta or vhs tapes.. or in this case, GUIs and web browser pages. For us, jumping on a bandwagon is as simple as understanding a domain and modeling it. We've done it time and time again and with Seaside we're positioned to pump out some really impressive web apps with far less effort than using traditional approaches.

The interesting thing is that web browsers are merging in to the same realms of technology that GUIs are. Both Firefox and Safari support SVG to a significant extent. Firefox 3 uses Cairo as its rendering engine (as we are exploring it for VisualWorks as well) and there are even discussions on how to do 3d graphics too (and let's face it, when it comes to 3d graphics, the most expensive bit is collisions and making 3d models as vector objects and arrays.. mark my word that some simple way to do this through a web browser will start to appear soon enough, bringing high speed gaming across the internet with stupidly simple protocols like HTTP, CSS derivatives and probably javascript).

The stigmatism of web development will linger for a while. When I talk to people about building web applications as gui applications, I often get a knee-jerk reaction of "eew, why would I give up my native widgets". Mmhm, what native widgets would this be? win32? gdk? cocoa? .. the listen of "common" frameworks goes on and the only that you could really call "native" would be win32.. which is defuncted by WPF anyway. Have you seen Firefox 3 on MacOSX lately? It looks as native as Safari does. The age of high quality "emulated" widgets is definitely here.. in the web browser and in regular apps.. the trick is to bother to do it right and to base it on the right foundations, such as libraries like CairoGraphics (if you're after a cross platform solution).

We haven't even discussed the implications of rich internet applications in Silverlight, Apollo or XUL.. but I may be getting off the track here. In the video, the presenter laments on how everyone reuses the same widgets all the time and don't step out of the box when it comes to UI design. Because they aren't trying to find ways of representing their data graphically well, we end up with the same boring dropdown boxes and tick boxes and input fields we've always had - and this can be a problem. Often, such a user interface is less efficient than simply writing it down on a piece of paper and throwing it at someone across the wall.

That was the crux of the BART transit system user interface redesign - and at about 1:05 in to the video, the presenter starts to show one of the cooler UI redesigns going around. It's called Zoom User Interface. The idea is this: your content should represent your content. If you have a document that you're editing.. it sits around on your desktop in its native form - not as an icon. When you're done with it - you zoom out to see things around it.. and zoom out further to see your entire desktop space. At any point, you can zoom out further to make more desktop space and at any point you can zoom in on an area to see more depth and detail - until you zoom all the way in on a piece of content and are involved in editing it or simply reading it.

The Zoom User Interface really is a cool idea - because it departs from the traditional idea of Applications. Where is no distinction between zooming toward a document and suddenly editing it.. you simply zoom in and start editing it. We throw away the old barriers and let the user organize their information spacially - which is a common human habit. Don't believe me? Take a look at the desk you're sitting at right now - I bet things are laid out on the desk in to certain piles and positions of which they hold some certain meaning -to you-. The same is true of the desktop.

If you ever saw the video for the Self programming language, they began this idea of an infinite desktop that you simply put objects on and manipulate at will. They didn't have the concept of zooming in and out of content to show and hide detail at arbitrary levels.. but they certainly were exploring a new operating system design. They never pitched it that way though - it was pitched as a programming language environment. In fact, it was a networked operating system with shared objects and shared desktop spaces.. fascinating stuff. I'd love to see it revisited. In many respects, that may be what Croquet is trying to achieve. I guess we'll see in time.

I think I've rambled enough now, so I'll end this post with a +1 for the video - go watch and ponder the significances of the things talked about.