When I was briefly attending the Welkind Rehabilitation Hospital for vocational evaluation I had the opportunity to do some minor programming work for speech therapy systems and learned a lot about adaptive computing. This field of computing held my interest for some time after and I was particularly intrigued by the peculiar nature of computers adapted for the blind. It always struck me as ridiculous that most blind user adapted computers -almost always PCs- could not even be turned on, let alone serviced, without a video monitor plugged in. (this, of course, was before the advent of port dongles and special purpose motherboards which today have solved this problem for the sake of file server use) I wondered why it was that, since the first generation of computers were command-line interface based and computers largely work in a text based information environment, no one had created a computer specifically for the blind user, offering all the same capability as any other computer and, more importantly, allowing the blind user to setup and maintain their computer without the assistance of the sighted. With this in mind I toyed with many different ideas for computers and user interface devices designed to be fully capable without reliance on visual displays. But soon other distractions had drawn my interest elsewhere and I put these ideas on the shelf.

Recently, though, these ideas came to mind again when I began reflecting on the difficulty I had encountered with the JoeyPAD project. The problem with such system designs has always been that I could never afford to prototype them on my own. The cost of custom components and engineering tools was just too great. So I began to wonder if there was a way to design a fully capable portable computer whose prototyping might be more within my personal means. The obvious key to this lay with the flat panel display. This is still the single most expensive component in any portable computing device and it tends to consume the lions share of engineering overhead, testing the limits of battery systems, circuit efficiency, and the economy of the rest of the hardware. It occurred to me that if one could eliminate the display, perhaps it would be possible to make a very powerful and capable portable computer at far cheaper cost and with much simpler, easier, engineering.

Today single-chip digital audio systems offer impressive capability at a very low cost, in terms of dollars, circuit board real estate, and power overhead. So a portable computer made to use sound alone as its user interface could pack a lot of capability and running time in a very small low cost package. As any musician knows, the medium of sound has just as much potential information 'bandwdith' as graphics. So in theory an all audio user interface is potentially as capable as any other if its design is sophisticated enough. Speech based computing has long been a popular concept but in practice it has not been very sophisticated. It works well in certain contexts but as a general user interface it is very limited and in most cases has been used only as a supplement with graphical user interfacing, as demonstrated by various attempts at computer based military strategic command center platforms and by voice command interfaces for robotic micro-surgery systems. What is needed is a way for sound to be used in as complex yet structured a fashion as graphics are in a graphic user interface.

Thinking on this I saw a connection between several concepts I had previous explored. Two of these resulted from my interested in blind adapted computing. One was an idea called 'audicons' -audio icons which could be associated with different functions and states. The other was the idea of 'soundscapes' -a means of exploiting stereophonic sound to create a dimensional environment in which audicons and other sounds were presented with a virtual sense of relative location. The other concept originated with a peculiar idea I toyed with some time ago for a Hackers Computer made from recycled Nintendo systems and other digital cast-offs. The basic point of this was the notion of a kind of counter-culture computing platform made from the cast-offs of the consumer and corporate cultures. A family of devices which could form the basis of an amorphous network of secret terminals and servers linked by X.25 based wireless links and simple serial connections. The Hacker's Computer featured an asynchronous distributed computing environment that used an iconographic threaded interpretive language as both a basic means of information encoding and as a programming language. It was like a proto-XML where packets of modular code with embedded graphic icons were integrated with plain text to create message 'threads' that could both serve as software and a simple rebus-like means of communication.

It occurred to me that if one used audicons based on MP3 encoded samples in a fashion similar to the threaded iconographic language of the Hacker's Computer and speech synthesis for the presentation of plain text then one could use the existing platform of XML as the basis of a robust audio interface environment. This would be done using a kind of audio 'browser' that functions as an interpreter for hypertext-like threads of information combining audicons, larger audio samples, text, and command tags. This browser is responsible for establishing a soundscape which allows the information of threads to be presented dimensionally while also establishing a set of common system functions. This is an approach similar to the Tapestry based approach of the JoeyPAD and Distributed Computer, using this audio-XML variant as the basis of all software, both local self-contained software and on-line software accessed through conventional web servers. Thus I arrived at the basic concept for the Mozart computer.

For user response and data input, this system would have several options. The most obvious would be speech recognition. But I've come to the conclusion that this is not an optimal approach. It has a very high processing overhead, has limited reliability, and is not conducive to the level of concentration normally required for intensive writing and computer use -not to mention being very distracting to others. So it seemed to me that a more practical approach would be to rely on the use of conventional and chord keypads as with the JoeyPAD and with word recognition speech synthesis feedback. This also affords the simple use of function keys as a convenient way to enter common words and phrases as well as for switching between different system modes/menus much as reserved keywords are used with speech recognition. It is likely that a finished product would have to include speech recognition capability because of the common expectations for an all-audio device but my suspicion is that the use of a keypad will prove more practical. Also, the use of keys as scrolling and cursor functions becomes a possibility in an soundscape environment, allowing control of 'mobile' cue sounds and audicons that in the virtual sound space much like graphic sprites or as a means of denoting relative position in lists or the like. (much like a position button in a 'scroll bar' cues relative position in a scrolling plane of text)

The complete Mozart computer would, physically, be a very simple device similar in design to any MP3 player but with the addition of audio input port supporting a headset, a built-in mic for note and sound sample recording, a pair or trio of USB ports, and a type-3 PCMCIA card port. Raised icon labeling for ports and audio controls would make the case features readily recognizable to the sight-impaired. The included chord keypad would be similar to the Twiddler product now popular among wearable computer enthusiasts and could be offered in either a conventional 7-key arrangement or in the same arrangement used in braille keypad devices. A pair of headphones with a removable or fold-up microphone would round-out the basic product package. The Mozart would not be marketed as a 'computer for the blind'. It is a general purpose portable computer practical for anyone who needs a compact and economical computing device and doesn't mind a lack of graphics. It just happens to be equally useful to the sight-impaired.

The specific hardware design is not important to this concept, and that's an important key to its potential development. The essence of this product is its audio interface environment and an effective prototype for that platform could be fashioned with any current computing platform, without the need for any custom hardware development. The ultimate hardware platform could be simple or robust, using virtually any combination of CPU and support chip family. But the power of single chip audio systems is such that very robust capability can be realized without a lot of processing overhead, thus very economical CPUs should be quite adequate. Similarly, software storage overhead would be much less than with graphics based computers since most of the user interface audio would be live synthesized while sample based audio is very efficiently compressed with the MP3 format. It is likely, though, that a CD/DVD ROM peripheral will be a popular add-on in order to support interactive audio entertainment applications -audio adventure games- with a very high audio quality.

All in all, it seems to me that the Mozart should be a very inexpensive portable computer to develop and should ultimately offer very powerful and convenient computing capability at a fraction of the cost of typical PDAs and hand-held computers. It should prove particularly useful as a portable word processing and e-mail platform as opens the possibility of a new family of all-audio interactive entertainment that would be well within the means of independent game developers to exploit. But, alas, further exploration of this for me will have to wait for a more stable housing and work situation.