I'm trying out a new approach (new for me, anyway) to writing doc. Over the years I've gotten so tired of messing around with Word. Assuming you get your styles to cooperate the format you wind up with is the format your doc will be in forever. Sure, you can go to PDF using Distiller, but what about other formats, like HTML? Ever seen the HTML Word produces? Yuck!

I've always wanted to write my doc in a simple XML vocabulary and then transform it into the desired format. I took a look at Docbook, and that looked promising, but it seemed like too much for what I needed. Plus, I didn't have a structured authoring tool.

Thursday I came across Apache Forrest. I had seen Forrest before while messing around with Cocoon but had never taken the time to explore it. As it turns out, it was just what I was looking for. It is a simple XML-based publishing system. Under the covers it uses the power of Cocoon to turn a simple XML format into a basic web site, PDF, or whatever format you need. Forrest is simple to get up-and-running, but the format, as simple as it may be, is a bit too painful for unassisted plain-text editing. So I still needed a structured authoring tool.

A colleague reminded me that James Clark wrote a mode for Emacs called nxml that works with RelaxNG Schemas. He also wrote a tool called trang that converts RelaxNG schemas into RelaxNG compact syntax, which is something nxml can understand. Both are easy to install.

So now, thanks to Apache Forrest, Emacs' nxml-mode, and trang, I've got a sweet little XML publishing setup.

9:24:30 PM    

Got charting working. The XMLDB pieces I noted in Step 3 and Step 4 of this post were actually very easy. The syntax for getting XML into Xindice is simple as is the querying. Once I got that going it was just a matter of hooking of the pieces of my pipeline to do what I wanted to do. I did have to tweak the XSLT that produces the SVG. I didn't build it to handle enough data points (bars were too wide, not enough graph area, etc.).

The cron and xmldb samples were really helpful in getting this working, both from a code perspective and from a functional perspective. As I stored XML in Xindice, I'd pop over to the xmldb browser sample and browse my collection to verify that it worked as expected. I used the cron sample OOTB to set up a task to run the DQL queries against Documentum on a schedule. Going forward, I'll need to incorporate an admin/config interface into my app for creating the cron task and browsing the xmldb collections.

6:14:13 PM    

When I dusted off my Documentum-Cocoon integration stuff I had to do a bit of a fix up. It seems that my WDK install had either rearranged some classpath entries (maybe different versions of JARs Cocoon dependend on behind its own) or made the classpath too long. In any case, I had to update the catalina.bat file to remove the WDK entries as a temporary fix.

I then noticed that when I ran any pipelines that used my Documentum-Cocoon components, they didn't seem to be getting called. My loggers weren't showing any entries and the page was just coming up blank. It turned out I had taken a little too much out of my classpath. Obviously, Tomcat needs to be able to find the Documentum DFC classes because my components rely on those. It was frustrating that no one was returning a helpful message to alert me to my blunder.

Something helpful in this situation is the Cocoon Status page in the Samples area. On that page you can show the classpath. If it doesn't see the DFC JAR and the Documentum config directory, you could be in trouble.

1:31:31 AM    

Made some progress on the Documentum-Cocoon stuff over the Christmas break. I'm working on a piece that will allow me to dynamically build charts and graphs based on data stored in the docbase. Here's what I've done and what's left to do:

Step 1. Figure out how to build a bar chart in SVG

First, I needed to figure out how to build a chart using SVG. I found some Java code that builds charts using SVG but what I was looking to do was build the SVG programmatically using XSLT to transform my source data which will be XML coming from my Documentum Transformer.

I needed something I could use to graphically draw the chart to get me started on the SVG. I downloaded a drawing tool called Sodipodi for this. It worked great. It stores everything as SVG. So all I had to do was draw my chart and save it to get a nice XML file to start with.

Step 2: Create the SVG with XSLT

Now that I had a static SVG file I needed to create XSLT that would take source data XML and transform it into the bar chart SVG. First, I copied my static SVG file into my Cocoon install to make sure it was happy rendering it into JPEG and PNG. It was. Next, I downloaded the Batik binaries. I wanted a tool I could use to quickly view my SVG file as I tweaked it. Batik comes with an SVG viewer called Squiggle. In WSAD, I created a test XML source data file, copied my static SVG file from Step 1 into a new XSLT file, tweaked, and then viewed the results in Squiggle.

The first thing I did was add comments so I could figure out which part of the SVG did what. In Sodipodi I had set every object's "id" attribute, so correlating the XML to the object it was describing was pretty easy. Adding comments made it easy to rearrange blocks of XML to work how I needed it to in the stylesheet.

The next thing I did was clean up the SVG by rounding off all of the numbers Sodipodi had used for the coordinates of my shapes. Obviously with a design tool everything has got to be very precise, but for what I needed, whole numbers worked just fine and it made the XML easier to look at.

I then figured out the algorithms I'd need to use to figure out how to dynamically size and position the bars and bar legends in the chart.

Finally, I replaced the hardcoded values with XSLT variables. I then updated my source XML data with new values, transformed, and voila: Dynamically generated bar chart.

Step 3: Grab the data from Documentum on a schedule

The data I need to chart is in Documentum. It changes over time. Because it's historical data I need to get a snapshot from Documentum on a schedule and store the result somewhere. Then, my chart will be built on-demand via a Cocoon pipeline using the data that's been captured for a given time period.

I already have a pipeline that gets the snapshot from Documentum. All I need to do is execute that pipeline on a schedule and store the result somewhere. I've tested out the "cron" feature of Cocoon and it works great. It allows you to execute a pipeline on a schedule. A perfect fit.

Now what I need is to put my snapshot data somewhere. I'm planning on using Xindice. It's an XML database embedded in Cocoon (you can use a separate Xindice install if you want). My pipeline will pull data from Documentum, transform it (I really only need to store a subset/summarized cut of the data), and store it in Xindice.

To complete my work for this step, I need to look at the XML:DB examples in Cocoon to learn how to stick data in Xindice from a pipeline.

Step 4: Build the chart based on the historical data

Now that I've got my historical data summarized and sitting in Xindice, I need to create a pipeline that will query Xindice for the data and transform the query result using the SVG I build in Step 2.

To complete my work for this step, I need to look at the XML:DB examples in Cocoon to learn how to query XML data from Xindice using XPath. The rest should be a piece of cake.

The result

Once this is in place, I'll have an approach and some reusable code I can use to capture and chart source data from Documentum.

1:14:57 AM    

Here's a new minor release of the Documentum-Cocoon integration components. It includes the reader and generator components I referenced a while back and a minor tweak to how I'm handling namespaces. (I'm not convinced I'm handling them the way I should, BTW). I've mentioned needing to develop a Documentum-specific protocol but now that I have a reader and a generator, that doesn't seem as important--readers and generators could be used as "internal" resources within a pipeline to retrieve things like XSL from the docbase to perform transformations.

I'm a little torn on what to work on next regarding this integration. I'd like a more elegant way to handle sessions. I can add single sign-on fairly easily so that when called from a WDK application the credentials would be passed to Cocoon and then used to create a session as the currently logged in Documentum user.

I'd also like to migrate the administrator dashboard prototype I've developed to use the Cocoon Portal UI framework.

Another idea is to use Cocoon scheduler to wake up periodically, execute a pipeline that reads data from Documentum, and persist that data either back to Documentum or into the Xindice XML database. This could be used to run historical reports, for example.

At Momentum I ran across a presentation by a fellow doing XML pipelining with Cocoon and Documentum. We exchanged information. If he posts his components for public consumption I'll link to them.

9:25:01 PM