Welcome to the enhancedAggregator tool code tour. The visit will take you to all the components of a typical Usertalk tool for Radio Userland, and will provide you with scenic views of Radio's beautiful news aggregator and its unique driver architecture.
Download enhancedAggregator.root to your Tools folder if you haven't already done so.
Read the general guidelines on how to add new format and module drivers to the tool.
Userland provides general information on tools creation here.
Roger Cadenhead's Radio Userland Kickstart has a whole chapter devoted to tools writing.
The tour starts in
enhancedAggregatorThread.script. This script is launched by Radio as soon as the tool is installed, and executed in the background as a separate thread dedicated to the enhancedAggregator tool.
For this to happen, the
enhancedAggregatorThread.enabled boolean must be set to true.bundle #1 registers the thread in Radio's temp table, I placed it there for debugging purposes, but it has no active role.bundle #2 launches the tool's instal script (colonials please note the proper spelling :-) in a dedicated thread. instal is responsible for hooking all enhancedAggregator's features to their proper locations in Radio's data structure. The script uses the enhancedAggregator table in Radio's temp table as a semaphore: since Radio empties the temp table at startup,
instal is launched only if the enhancedAggregator table does not already exist. This ensures that instal is launched only once at startup, no matter what. The
instal script is run in a separate thread so that it may lead an independant life, even updating itself if autoUpdate is enabled, without any backward effect on the main enhancedAggregator background thread. The main background thread is supposed to live as long as the tool is active, therefore the script's
bundle #3 is an endless loop. Ending the script while the tool is active would cause Radio's scheduler to restart it within 1 minute, which takes longer than dozing.
Now proceed to the
enhancedAggregatorsuite.instal script. Leaving
bundle #1 on your left, first check out bundle #2, which instals the tool's user preferences by calling the buildPrefs local function. The tools default preferences are located in
enhancedAggregatorData.prefs. They are booleans that govern the enabling or disabling of all the tool's features. Since the preferences are user modifiable, we want to store their values in a distinct location, and make sure that we do not overwrite them with their default values if they already exist.
I chose to store the user prefs in an enhancedAggregator subtable of aggregatorData.root, the main database for Radio's aggregator.
Since we are modifying a table outside the scope of our own tool database, we will have to make sure we clean up after ourself when the tool is uninstalled.
I could have chosen to store user prefs in a dedicated database file, or in a user subtable of enhancedAggregatorData instead.
bundle #3 instals our first RSS 2.0 module driver: support for ENT 1.0 topics. Since ENT 1.0 support can be turned on or off in the tool's preferences desktop website page, the actual job is turned over to a dedicated script:
enhancedAggregatorSuite.ent.instal.bundle #4 instals the callback script necessary to include topics in Radio's desktop News page. The way Radio's aggregator is designed, the module driver installed in bundle #3 retrieves the namespace specific tags' content and stores it into a compilation table for later consumption by other programs, while the
storyArrived callback script retrieves the compiled information and updates the aggregator's internal data structure. We are adding the address of our script,
enhancedAggregatorSuite.ent.callbacks.aggregator.storyArrived, to the stack of storyArrived callbacks set up by Radio in aggregatorData.root. Since I've selfishly and unwittingly used enhancedAggregator to label the entry for ENT topics support, you would have to use another string, possibly the module's name or URI, to insert a callback to support any additional RSS 2.0 module.
bundle #5 instals a format driver for Atom feeds. The actual job is handled by
enhancedAggregatorSuite.atom.instal.bundle #6 instals another module driver for ESF events. The actual job is handled by
enhancedAggregatorSuite.esf.instal, similar to ent and atom installations.bundle #7 instals the tool's un-installation code. The uninstallation code performs the reverse of the
instal script. The trick is it must execute even if the tool's database itself, enhancedAggregator.root, is no longer in the Tools folder.
So our first step is to copy the
uninstal script to the scratchpad table in Radio.root: it shall remain behind until needed, whatever happens. Our second step is to instal the address of our newly created scratchpad uninstal script in the afterUninstall callbacks stack in user.tools.callbacks.
This way, the tool's uninstal script is executed every time Radio uninstalls a tool, something that happens under several circumstances, more about this later.
bundle #8 instals the tool's update table in its proper location in user.rootUpdates.servers. We only insert it in if it's not there already.
The update table is part of the tool framework's standard update mechanism.
It defines which server to retrieve updated parts from, and which method to invoke on that server via xml-rpc, through which port.
I've defined all the proper values in enhancedAggregatorData.rootUpdateTable. The update server is a Frontier system operated from Downtown Manhattan by my company, Precision IT Management, Inc.
Finally, our last section,
bundle #9 updates the tool if the autoUpdate user preference is set to true. The
update script is copied to the temp table and the copy is launched in a separate thread, since the update mechanism is liable to download an updated version of the update script, which would then need to replace the original, without interrupting the current thread. The tool is now properly installed, our next station is the
enhancedAggregatorSuite.ui.prefs.edit script. This script is responsible for modifying all the tool's user preferences through a desktop website page.
When a user directs his/her web browser to http://127.0.0.1:5335/enhancedAggregator/, Radio returns a web page created by merging
enhancedAggregatorWebsite.index into enhancedAggregatorWebsite.#template. The
#template outline defines the general structure of the HTML page, with some specific CSS rule definitions in the header section, the {bodytext} macro being replaced by the content of index. The
index text object contains whatever is returned by the enhancedAggregatorSuite.ui.prefs.edit script. The
edit script takes a single argument: the address of the page table, an invaluable data structure holding all the elements of the HTTP request received by Radio's desktop web server. To get a better look at the page table, uncomment debug bundle #1.
The main job of
edit is to return some HTML content to fit into the preferences page. This content is defined in the
enhancedAggregatorData.ui.prefs.template outline, and assigned to the s local variable, which will eventually be returned by the script. The preferences page is mainly an HTML form made of
INPUT type="checkbox" fields. It is a template itself: variables defined between ## tags will be replaced on the fly by the
edit script, according to the current values in the tool's user preferences table: this is the part of the script that executes after the first if statement. Here is the interesting part: the page's form is submitted to the same http://127.0.0.1:5335/enhancedAggregator/ URL, generating an HTTP POST request instead of the HTTP GET request normally issued when the user first entered the URL in the browser or linked from another HTML page.
For more information on HTTP protocol requests, check out O'Reilly's HTTP: The Definitive Guide by David Gourley and Brian Totty.
Hence the importance of the first
if statement: if we are answering a POST request, then the page table's requestBody string is not empty, and we can proceed to retrieving the form's content before returning anything.requestBody always contains something when the form is submitted, even if all checkboxes are unchecked, because of the invisible action input field in the form. If we are indeed addressing a POST request, our first order of business is to organize the content of the submitted form fields neatly in a
postargs table: Radio provides a convenient webserver.parseArgs function for that very purpose. To take a better look at the
postargs table, uncomment debug bundle #2. The name of all checked INPUT checkboxes will appear in the table.
All we have to do is set the value of each boolean user preference according to what we find in the
postargs table, saving the user prefs database for good measure when finished. Once this is done, we activate or deactivate each driver according to the current value of the relevant user pref.
The
displayMode template variable is a CSS stunt, so that we may show the user that his/her modifications where taken into account. Finally, when finished setting preferences, we proceed to returning some HTML content to the client's browser, just as we would for an ordinary GET request.
From preferences setting, the visit carries on to the
enhancedAggregatorSuite.scratchpad.uninstal script. As we saw when taking a look at
enhancedAggregatorSuite.instal, a copy of uninstal executes from Radio's scratchpad table every time Radio tries to uninstal a tool. This happens a lot more than you'd think:
It happens once for every tool found in the
user.tools.databases table every time Radio starts, before the Tools framework installs anything: you got that right, even active tools are uninstalled first before being reinstalled during Radio's launch sequence. It also happens whenever someone deactivates any tool from Radio's desktop website Tools page.
The job of the
uninstal script is to make sure that all driver enabling hooks into Radio's data structure are removed. In addition, it must also thoroughly clean up after itself in case the tool is no longer present in Radio's Tools folder: we don't want to leave untidy litter behind, do we? The callback script receives as its only argument the full pathname to the tool's database file. This is enough to compute the tool's name, and what the tool's path name should be if the tool is active.
First thing: make sure we are processing something for enhancedAggregator uninstallation, if not, quickly make our exit.
Second, provided we are indeed uninstalling enhancedAggregator, remove all hooks to the drivers and callbacks from Radio.root, terminating support for all formats and modules.
Third, delete our tool's table in temp, and kill our tool's background thread, the one started in
enhancedAggregatorThread.script, which is busily doing nothing at this time. We retrieve our background threads id from the aptly named
temp.scheduler.idThreads table. Since the tool has been uninstalled, the scheduler won't try to restart it every minute.
At this point, we've done enough to restart support for formats and modules in the exact state they were operating before the minute someone reactivates the tool, by clicking the now empty enhancedAggregator checkbox in Radio's desktop website tools page for instance.
But what if our user has removed the enhancedAggregator.root file from Radio's Tools folder, being now completely fed up with RSS modules and exotic syndication formats ?
We are still executing the
uninstal script, since we are operating from the scratchpad table in Radio root. We want to remove any trace of our tool's former presence in Radio's installation.
1/ remove the tool's preferences table from aggregatorData.root.
2/ remove the tool's update table from Radio.root.
3/ remove the tool's uninstal callback, we won't need it any more.
4/ remove the tool's entry from the tools database in Radio.root.
Finally, we commit an elegant variant of sepuku, launching a self distruct order for the uninstal script itself from a separate thread: this is the perfect elimination, no evidence left behind :-)
Wait, we've ambled from launch to installation to preferences setting to uninstallation, and we still haven't caught a glimpse of what's actually performed by the tool's payload, the core driver activity!
Well, that's where visitors more interested in Usertalk tool building than syndication formats support can walk off and resume normal activities :-)
Others are welcome to the - at present - 3 format named tables in
enhancedAggregatorSuite. That is if they're not already bored to death :-)
Each tool table has a dedicated
instal script, and a driver subtable with format specific elements. The
enhancedAggregatorSuite.atom table provides support for Atom feeds aggregation. That is it will provide support when a volunteer adds some flesh to the current skeleton.
The
atom.instal inserts or removes a copy of the atom.formatDriver table inside the user.xml.rss.formatDrivers table in Radio.root. The name of the entry must be the name of the top XML element for that format, ie feed for Atom, according to its 0.3 specification.
All formatDriver tables must contain a
compile script. The current version for enhancedAggregator doesn't to anything useful: it merely starts filling the service's compilation table with channelTitle and channelLink elements, which are needed for the proper display of Radio's desktop website Subscriptions page. A complete implementation would flesh out the compilation table with as many ENT elements as possible while exploring the service's
xmlstruct table, then provide a storyArrived callback script to add compiled elements to the aggregatorData.stories table in a manner consistent with what is needed for the proper display of Radio's desktop website News page. The
enhancedAggregatorSuite.ent table fully supports ENT 1.0 topics. It's the only complete driver table available at this time.
ent.instal inserts or removes the address of the ent.moduleDriver table inside the user.xml.rss.moduleDrivers table in Radio.root. The name of the entry must be the URI for the RSS module's namespace, in our case http://www.purl.org/NET/ENT/1.0/.
For additional information on RSS modules driver architecture, check out Jake's description on Userland's site.
The only ENT sub-element of item we are interested in is cloud, which encapsules a list of available topics for each item. The driver is henced implemented in a single script:
ent.moduleDriver.subElementOfItem.cloud. The
cloud script's job is to scan the service's xmlstruct subtable, starting at the cloud subtable pointed to by the adrElement parameter, looking for topic entries and compiling them into the service's compilation subtable, pointed to by the adrItem parameter. You can take a look at the whole service table, designated by the
adrService parameter, when uncommenting the debug bundle before the main scanning loop. I chose to compile topics using the following structure in the
item subtable of the compilation table: An encapsulating
topics subtable if topics exist for this item. One table entry in
topics for each topic id found. Each
topics.[id] subtable holds any number of the three attributes for an ENT topic: its name, href and/or classification. Once topics are compiled by the module driver, they are ready to be inserted into the aggregator's data structure by the
storyArrived callback script. But here's the rub: with the current version of Radio, the callback script is called and executed before the module driver gets a chance to compile the module's payload. Until Userland releases a fix, you can manually open the
system.verbs.builtins.xml.rss.compileService script, and scroll down to the get the items bundle inside the RSS branch of the case format statement. Move the
runModules (item, adrRss, "", adritem) statement you find at the very bottom of the for loop 2 lines up, so that it is executed just before calling the addHistory function. The addHistory function triggers the storyArrived callback sequence.
Compile and save the modified script.
The
ent.callbacks.aggregator.storyArrived script first makes a copy of the compiled topics table, identified by the adrItem parameter, in the proper story entry of the aggregator's table. The aggregator's table is
aggregatorData.stories. Entries are designated by their story number. There is a persistent stories counter available as aggregatorData.prefs.nextStoryNum. As its name shows, at the time of the execution of
storyArrived, the counter is already designating the next entry, so we must substract one its value to compute our story's id. If the
displayTopics user preference is enabled, the script then modifies the storytext element of the story's table by appending a string with an HTML rendering of the topics list. topics' href attributes are rendered as HTML links around their topic's name.
The
enhancedAggregatorSuite.esf table will provide ESF events aggregation some day. For now it's a hollow shell: the
esf.instal script activates or deactivates the module according to the current user preference, and there are as many scripts in the esf.moduleDriver.subElementOfItem table as there are required elements in the ESF specification, but they are all empty. Hopefully, someone at eVectors will soon provide some meaningful content.
I hope that publishing this tour as an outline has made it easier to follow. For more information about outline publishing, check out the activeRenderer site.
