Jeffrey P Shell: Zope

Moving Toulouse

Sat, 11 Jan 2003 19:24:27 GMT

Industrie Toulouse has moved to A new home (toulouse.amber.org) and a new publishing system. Most of the archives will be there soon. The Radio site will remain up indefintely, but no new content will appear here.

Aggregation versus Inheritance

Thu, 09 Jan 2003 21:37:38 GMT

EuroPython had an interview with Jim Fulton (I can't determine the date, but I'm guessing some time within the past year), in which he gives a good summary of Zope 3's differences over Zope 2 (I've been looking for such a summary):

Zope 3 moves Zope from an inheritance-based framework to a component-based framework. Complexity is managed by splitting responsabilities among many cooperating components, rather than many cooperating mix-in classes. Components are connected using interfaces, which also provide component specification and documentation. Some other big ideas:

It should be possible to use existing Python objects in Zope with little or no change.

Applications can be customized by adding, changing, or removing components.

Site configuration is done by site-managers or deployment specialists without modifying code.

Many CMF technologies will be part of the core.

Acquisition and namespace lookup is wildly more explicit.

I could go on, but I won't.

"Cooperating components instead of cooperating classes" could still be seen as an ambiguous phrase. I think developers who are steeped in component models and aggregation would understand it. But what about those who are used to the static overbearing inheritance model? I guess it could be summed up as "Zope 2 excepts you (you being an object in the Zope system) to provide it lots of information and functionality all by yourself. Zope 3 looks for people to help you". In Zope 2, if you got annoyed that a particular didn't implement the obscure little method to display sizing information in the ZMI, there wasn't much you could do about it without monkey patching. In Zope 3, you write an adapter for that object that implements the ISized interface, register the adapter, and voila! One little object helping another. By developing services to query for adapters (saying "I've got Frank here, can anyone tell me how tall he is?") instead of expecting behavior directly from an object, new doors to interoperability and speciality open up. Susie pops over and says "Why I can tell you, I've got measuring tape right here! He's 5 feet, 11 inches tall." Of course, Frank can say "I can tell you" and report his tallness himself, but he's a busy man and knowing his height at all times just might not be something he needs to do on a daily basis. Of course, if Susie can be replaced with a robot or some other machine or person that can perform the same job (measure how tall people are), she can be replaced without bringing down the system. Frank's genetic algorithm can't be replaced (easily), so if he's suddenly unable to report his height correctly, he's useless in this situation without someone to help him adapt. Hmm.

Zope 3 reStructuredText Document 0.1

Sat, 04 Jan 2003 18:53:11 GMT

With Zope 3X alpha 1 out, I decided it was time to jump in again and see how it would be to write a new content object for it with associated components (adapters, views). I've tried to include comments in the files and README to document what I was doing as I was going along.

It can be downloaded here, and requires a fairly recent docutils and of course Zope 3X Alpha 1.

The current release is intended to focus more on documenting how to write a new content component for Zope 3 than on being a smart reStructuredText client. That functionality may come later.

I have some other thoughts about the process posted on ZopeZen.

Big Alpha Releases for Zope and Python

Fri, 03 Jan 2003 22:22:33 GMT

Two big alpha releases were made for the new year: Python 2.3a1 and Zope 3X a1.

Python 2.3 doesn't make any major changes to the language (unlike previous Python 2 series releases, which brought things like list comprehensions, generators, nested scopes, iterators, etc). Andrew Kuchling, as always, covers the changes extensively. Of interest are - ability to import from .zip files, universal newline support!, a new logging package, a new Set datatype and default Boolean datatype.

Zope 3X (x for "experimental" - a final Zope 3 release without the 'X' is expected to have some Zope 2 migration features) is a completely different beast from its ancestors. Its primary focus is still the Object Publishing mission that's been in place since the days of Bobo, but beneath the covers it is shockingly different from Zope's 1 and 2 before it.

Zope 3 is also dubbed the component architecture project. Its design focuses on many interacting components with well-defined interfaces. It's an aggregation heavy design, as opposed to the inheritance heavy design of Zope 2. Adapters exist to take the burden off of objects for out-of-scope elements such as text indexing or sorting-by-size. As the author of a particular object, you can implement those interfaces (ISearchableText, ISized) on the class itself, or as separate classes. Code that requires searchable text or sizing information asks for an ISearchableText implementation for your class, and since that's the only interface that it's expecting to use, it doesn't matter if it gets an adapter or your class - it only matters that the ISearchableText interface is implemented correctly. Design by Contract finally makes a strong appearance in the Zope world.

Zope 3 should also score big on the packaging and deployment front. Zope 3 uses a new configuration language called ZCML. ZCML is used to register new components at load time, and to load in still further components. It wrests control away from Python's import statements and need for special functions in __init__.py such as the common Zope 2 initialize(context). As such, it can be not only more expressive, but allows control over the order in which items get initialized. Under the mantra of explicit is better than implicit, all new components have to be explicitly added (usually in the products.zcml file). A side effect of this is that heavily customized Zope distributions can be made. Some such distributions exist now with Plone installers and BizarShop. Zope 3's customization and configuration capabilities should allow similar distributions to be made that don't look like Zope at all, while still tying in to all of the available features of Zope.

The combination of heavy Interface use combined with rich configuration options should mean that any part of the system could be kicked out in place of a new one, as long as the expected interfaces are implemented. This is a fundamental feature of component systems, and Zope 3 looks like it will live up to this feature.

Another big feature of Zope 3 is the proliferation of View Components. Views can be individual ZPT pages, attached to a particular content interface via ZCML. But they can also be full-on components in their own right, usually as Python classes. And they're not limited to HTML. Other views may be written to correspond with the other publishing channels (ftp, xml-rpc, and others in the future). I wrote a post back in September about my early experiments augmenting other Zope 3 components, in which I include some example code making an XML-RPC view onto a Job Board component written by someone else. My final paragraph in that post reads:

And finally, Zope 3 should yield a usable scalable means of adapting the works of other developers into new solutions by providing better control of product/service/view configuration and overrides, such as adding a new XML-RPC API to someone elses bug tracking system without having to alter that bug tracking system, or use secret Python hacks to alter behaviour.

A few other positive notes about Zope 3X a1. The "grand renaming" went into effect during the final weeks of the beta. This involved cleaning up the Python package hierarchy. It also involved case-normalization of package and module names. They are now all lowercase, like most other well-written Python packages/frameworks (distutils, docutils, etc), which makes it easy to distinguish classes and other module exports from the modules themselves. Zope 3 also uses Python 2.2 "new-style-classes" and gets rid of ExtensionClasses, since now you can write new extension types for Python in C and subclass them (which was the big point of ExtensionClasses). Zope 3 also (finally) makes use of other long-standing Python technologies such as distutils.

Now for the negative aspects. Since this is an alpha of the Zope 3 "experimental" line, it's not surprising that there is little or no documentation. The code structure is significantly different from that of Zope 2. It takes some time to find ones way around the code layout. Most things are pretty clear though (a benefit of having Interfaces) when you do find them, but there's so much new technology and terminology here that even when you grasp the terminology, it's not obvious to an outsider how to put it all to work.

Zope 3 is heavily reliant on Services, which is a good thing. Services are like Tools in the CMF - well known objects that provide a service to a global or local region. Examples of services are Event Channels (yay! Events!), or something like the Error Log that showed up in Zope 2.6. Other examples of Services are Database Adapters. In Zope 2, most service objects existed in the same namespace as all other objects. Zope 3 manages them in special namespaces. This allows content objects to be managed separately from service objects while still retaining placefulness. There are numerous upsides to a service based architecture. What's the downside? The downside is there's significant "what the...?!" effect when first encountering Services and Configuration objects. I'm sure it will all make more sense as future Zope 3X development releases are made, but right now it's a proverbial kick in the head.

There are some other aspects of Zope 3 that I'm likewarm to right now, mostly because they're difficult to fully comprehend how/when/where to use at the moment. Zope 3 allows for automatic user interface generation to occur. This was a feature of OS/2 (2.0+), and the defining motive of Naked Objects. Zope 2 has a subset of this feature in its use of properties. What Zope 3 seems to have done is to marry Interfaces (software, not user) and Formulator together. In essence, this allows easy edit forms to be generated off of the objects interface description. From the 'src/zope/app/browser/content/configure.zcml' file, here's an example of how an editform is configured for a ZPT Page:

  <browser:editform
      schema="zope.app.content.zpt.IZPTPage"
      name="edit.html"
      menu="zmi_views" 
      label="Edit a ZPT page"
      permission="zope.ManageContent"
      />

And from the IZPTPage interface:

    source = zope.schema.Text(
        title=u"Source",
        description=u"""The source of the page template.""",
        required=True)
  
    expand = zope.schema.Bool(
        title=u"Expand macros",
        )

Whew! That's about all I can cover for right now. If I have time this weekend, I intend to write a new Content Component (probably a reStructuredText Document) to see how different it really is from developing a similar product for Zope 2 or the CMF. I have no promises right now that it will get done this weekend, but it is something I've been wanting to do. Now that Zope 3X is at alpha1 stage, the fundamental structure should be fairly stable.

More Fiddling with Project Management

Sat, 28 Dec 2002 08:01:38 GMT

It should come as no surprise that I'm still fiddling with project management solutions via "Zope". I've started down a new path, but before I explain it, I want to explain the current situation.

General Requirements

We're a very small organization (two people, with loose associations to other very small organizations). As such, we're constantly busy (thankfully) and don't have too much overhead to worry about in regards to project management. But having a to-do list of any sort is always helpful, and having a place to dump notes and flesh out ideas is always good too. Plus, we're decentralized - we work as much from home as we do from the office, and are often working from multiple machines in either location. Sometimes we need to communicate progress with each other, sometimes it's nice just to have a list for our own uses. Ideally, content (which could be proposals, notes, software configuration requirements, etc) and issues should be managed by the same system - or at the very least be close to each other. And projects need to be easy to add in to the system by either main user, from wherever they may be. And most of all - it should offer only just-enough project content management features.

What we've used

Last year, we experimented with a couple of home-grown systems. One was a Zope based SQL backed project / task / contact tracking system. Data went derelict rather quickly. I was working on a much more thorough CMF based system with detailed workflows, nested tasks, role assignment, and more. There were a few key features that were difficult to do to meet our requirements, particularly in the non-containment object relation area, an area where Zope has typically struggled (but should be dealt with in Zope 3). We gave the Roundup (0.4.2) issue tracker a try. While fast, it was still a young product. Too much command line work was involved to set up a Roundup instance, and while there is a Zope-Roundup Gateway available, it didn't integrate too well with Zope based content. I really like Roundup and think it has a lot of potential, but it didn't satisfy our requirements. Ultimately, we landed in the ZWiki 0.7 + Tracker situation. Each project was their own folder or wiki, and usually had a tracker in it. This approach succeeded on the simplicity category and has been used for almost half a year. But it suffers from a couple of problems. The first is that Tracker is an aging old beast. It's generally pretty thorough, but has some rough edges. It's no longer supported, and I've heard claims that it doesn't work with the latest "Zope" release family (2.6.x). The second problem is that I'm no big fan of Wiki's. I never have been. They've been useful, but we don't need whatever supposed collaborative effects Wikis are supposed to have. I don't like Wiki names - they either end up looking funny (with words like OneDotOh or ZoPe) or kick in when you don't want them to. Now that I've done some work with reStructuredText, I've come to loath working with Zope's normal StructuredText. It just doesn't work well with TextAreas. If you're used to Zope's StructuredText and have a good editor on your side, it's not too bad. But StructuredText and DTML (outside of SQL methods and occasional small non-HTML uses) are two Zope technologies I'm happy to leave to the past. Finally, we needed a stronger content management system than a pool of individual wiki's provided. We did (briefly) try out ZWiki 0.12 with setting up the ZWiki based Issue Tracker, but it was a beast to set up and configure (loosing out on the 'easy-to-add-new-project' requirement) and too dependant on DTML for me to debug and make work for us on short notice. I liked the integration of Issue Tracking with the Wiki (gaining points on the keeping-issues-close-to-content) but there was too much pain involved, and too much inolvement with the technologies I wanted to leave behind. And I know Wiki's really really work well for some, but I've never really been able to get comfortable with the Wiki way (ask anyone that I used to work with). I've been able to get by, but that's about it.

The new attempt

Facing the problem of trying to support the Trackers onto post-Zope 2.5.1 software, and facing other issues that stemmed from the basic folder/wiki layout of the site (no intra-project search abilities, poor navigation), it was time to look for something new. That new thing is Plone - a realized content management system built upon Zope's CMF (Content Management Framework). With my knowledge of the CMF (I used to be on the content management team at Zope Corp), I was able to start tweaking the system where I needed to - I made a Project Folder content type to distinguish Projects from other content types without putting too much expectations on the content type. It can contain anything, like documents, links, events, and a CMF Collector (a weaker but still fairly usable descendant of the Tracker). I later designed a quick workflow for Projects so that their status would be more meaningful to allow distinction between active, pending, suspended, completed, and cancelled projects. The workflow has a few simple rules, but nothing distracting. CMFCollector is decent. It has some of its own rough edges, and is missing out on some of the nicer features of Tracker (mainly intra-issue explicit linking). But it has some nice new features, like the ability to automatically assign/accept a new issue. Its interface is a bit more streamlined, particularly in the Plone skin. It uses a very simple plain text markup language it calls WebText. It basically takes plain text, marks up URL's and citations/literal blocks (lines beginning with '>' or white space). A downside to Plone is that it - for better or worse - just builds upon CMFDefault. And CMFDefault has very basic content objects, particularly the default Document, which is very monolithic. I partly blame myself for the design - I could have proposed a better solution back when I was on the team. In my vaporware compound document project, FDoc, not only are the parts of a document componentized, but so are the text handlers. A separate Handler Registry contains handler objects, which take a chunk of text and turn it into an HTML content body. This could be HTML (taking the content between the BODY tags), Structured Text, Plain Text, etc. So today, wanting reStructuredText support, I tried to think of how to deal with this. I had a couple of options: either take the existing FDoc work, which is half baked (the most complete work is in a customer specific project and hasn't been backported yet due to time constraints) and add a reStructuredText handler (and also make a Plone friendly user interface), or make a subclass of CMFDefault's Document class, add in the ability to manage reStructuredText somehow and make it a new Content Type or replace the Document document type (this too would involve some Plone UI work, but it would only be a tweak of the Document's edit form). Neither situation was quite what I wanted - there were too many existing dependencies on the default CMF Document behavior, and I haven't had the time to really learn how to make my own Plone UI pages work. So, I decided to do a hybrid of my original ideas. I made some new FDoc handlers that handled Zope's StructuredText and the reStructuredText formats. Then I made a subclass of the CMFDefault Document and started wiring in knowledge of the handlers and handler registry (based on the FDoc Text Part component, but made to match the default Document's interface). Then, out of some act of devilry, I decided to monkey-patch document by doing some good old fashioned Pythonic class __dict__ comparisons and substitutions. After a few tweaks to the document edit form to list the available text formats dynamically instead of statically, I had it working (aside from one infinite recursion bug my __dict__ tweaking introduced). So now, it's mostly working. We have a decent looking project site that's navigation friendly. It's easy to add new content and to decide its form and format with very little restriction. And I'm finally away from StructuredText (although it still exists as an option). With other CMF and Plone options available from the likes of the CMF Collective, there's a lot of potential expansion if required.

Quicklinks Zope Product

Tue, 10 Dec 2002 20:26:37 GMT

Peter Bengtsson has a really cool "Zope" product, QuickLinks. His product effectively adds a shortcut toolbar for creating new Zope objects. It populates the same line of the ZMI as the "Add List" with icons representing different meta types. Clicking on one of them goes to the add form for that object. If you spend most of your time in the ZMI mousing through the add list to add the same set of objects (ie - folders, python scripts, SQL methods, and page templates) this is a great product. Configuration is done with a new Control Panel object.

State of the desktop?

Tue, 10 Dec 2002 08:21:18 GMT

The theme of today's conference was decentralization....
The theme of today's conference was decentralization. The first time the term appeared in DaveNet was in the first piece of 2001. That's another DaveNet tradition, like the Thanksgiving essays. I try to make the first essay of each year somehow express the most important idea of the year-ahead. It's always a guess. Some years I nailed it, desktop websites were the big idea of 2001, as we prepared Radio 8 for the market (it shipped in January 2002). And Werbach was right to pick it as the theme for the future in software-based technology. There's so much power on the desktops, both in the machine CPU and the human CPU, that isn't being well used in the centralized Internet architecture. [Scripting News]

Pfeh to the desktops. There's a lot of power in them, but I'm positive I'm not the only one using three machines throughout the day. There's nothing worse than realizing that the data you need is held hostage at home or at work - wherever you're not. There are plenty of times when I don't mind the separation - work's work, home's home. But there are certainly things that go in between. I've been a fairly happy subscriber to Apple's .mac services, using my iDisk to ferry a few common documents and preferences (standardizing the subscriptions on all my instances of NetNewsWire Lite, Chimera bookmarks, etc). I'm using Apple's iSync Beta to keep two desktops, a laptop, and an iPod all synced up with the same address book and calendar data (I have a palm too, but I seldom use it any more). Microsoft's "Active Desktop" vision wasn't entirely wrong. The implementation wasn't quite right, and definitely premature. But the lines between the desktop, the local network, and the global network, are fading away. Rendezvous makes local networking absolutely unbelievable in Mac OS X (well, it's believable to those who used the old AppleTalk, but it's remarkable in the internet age). I don't know. The desktop web site idea isn't too bad (I'm using Radio right now), but I'll always prefer the "Zope" "Everything is done through the web" model. It's the comfort of knowing that fixes, updates, etc, can happen from anywhere, with pretty damn good security.

JTracker - Another Zope Issue Tracker

Mon, 02 Dec 2002 16:30:46 GMT

The always pragmatic Jens Vagelpohl has released JTracker, a simplified issue tracker based on the classic Zope Tracker and CMF Collector, but python based and simplified. It's what I've been meaning to write but have never made the time to do it. As soon as I can make time to play with it, I'll post my thoughts.

Zope 2.7 Project (re)initiated, and the joys of error_log

Wed, 27 Nov 2002 15:31:14 GMT

It's gone through a couple of revisions already, but the real project plan for the next "Zope" release, Zope 2.7, is now up on dev.zope.org. Among the list of Proposed Features is Easier Configuration, Installation, and Management, a welcome addition. As for Zope 2.6, all "Zope" developers owe it to themselves to download it for one reason above all others - error_log! The new error_log object gives detailed access to recent exceptions raised. Not only is this beneficial during development, but it has great testing/deployment uses as well. You can keep an eye on a deployed site to see if unhandled exceptions are happening, and where. You can also get detailed technical exception reports during testing that a tester may not be able to report on their own.

Links and Such

Mon, 25 Nov 2002 16:59:42 GMT

I've been absent for an unusually long time. There's been a lot to write about, but family and work responsibilities have kept me occupied. So, here's a few quick links and thoughts: There's some good reading (as always) in the December 2002 Harper's. The November 2002 issue also has some great articles, particularly about the U.N. Sanctions against Iraq, and leads off its Monthly Index with this: Ratio of Japanese killed in 1945's U.S. atomic-bomb attacks to Iraqi children killed due to U.N. sanctions : 1:3. I was first made aware of the inhumanity of the ongoing sanctions through a Michael Moore show a couple of years ago, where they sold gas at discount prices with images of Saddam Hussein plastered all over the gas station "Who wants to buy gas from the whacky Iraqi?". Lots of people showed up for the discount gas, and the money was used to smuggle food and supplies into Iraq. Speaking of Michael Moore, I finally saw Bowling for Columbine this weekend. It was a good film (for however good a film on such a subject can be), but I felt there were a few flaws - namely the exclusion of inner city gun violence. It may be that Moore is pitching his ideas to the masses in the suburbs in hopes that younger people in the suburban high schools will be awakened to these issues. In any case, I recommend seeing the movie. And I also recommend reading this critique on "Moore's urban phobia" from the liberal site/magazine The American Prospect. On the technical side, there is an interesting page/discussion in the "Zope 3" wiki about Trying to unify workflow concepts, primarily dealing with Activity based Workflows and Entity based Workflows.

ZWiki Tracker.

Fri, 15 Nov 2002 06:31:18 GMT

I try and try and try to like Wiki's, and I just don't. ZWiki is a flagrant offender of my "no alarms, no surprises" preference. Maybe it's because it's heavily dependant on the two other big {NA,NS} offenders - StructuredText and DTML. But, we're giving the latest ZWiki a try. Why? Because of the ZwikiTracker. Yes, my search for the perfect "Zope" based issue tracker continues. The ZwikiTracker has some advantages:

Simple interface.
Integration with Wiki content (for better or worse).
Use of ZWiki features such as Comments and Subscribing.

I had hoped for some support for reStructuredText, and barring that, I had hoped that reStructuredText integration with ZWiki would be relatively easy. There are so many supported text formats, I thought/hoped that there was a modular system underneath the ZWiki code that would make adding another text format a straightforward effort. Nope. A ZWikiPage is a very large monolithic object. Extending it could not even be done easily with subclassing, because it would be very tricky to ensure that the subclassed ZWikiPage gets instantiated correctly. sigh... Still so far from my goal for our internal project management systems.

Problems with StructuredText

Wed, 13 Nov 2002 16:50:10 GMT

I was going to write up my own list of issues I've had with Zope's StructuredText (Classic and NG) implementation, but this document sums it up nicely. My expectation of software is, to quote Radiohead, "No alarms and no surprises." "Zope" Page Templates live up to that mantra, as does reStructuredText, so far.

reStructuredText First Impressions

Thu, 31 Oct 2002 05:10:23 GMT

Today I started writing up user documentation for the project I'm wrapping up. It's the first time I've used reStructuredText, and I must say that I'm liking it. As far as structured text markup languages go, it's pretty comprehensive and not *too* obtrusive. It's definitely good enough for most technical documentation needs, and is in fact a PEP Formatting Standard (see also PEP 12 Source). Other items of interest, particularly to Python programmers, are:

PEP 256 "Docstring Processing System Framework" (src)
PEP 257 "Docstring Conventions" (src)
PEP 258 "Docutils Design Specification" (src)

I include the source links to show the relative cleanliness of the markup, which is the point of all Structured Text formats. sigh I thought I'd have more to say, but I'm really too tired. So - quick wrap up: I was pretty productive with this stuff right off the bat. Nice. For "Zope", there is a proposal to integrate reStructuredText into Zope, hopefully for 2.7.

Zope Stories

Mon, 28 Oct 2002 15:15:24 GMT

Zope Stories, another Zope weblog with some very good links and, well, stories.

meta-whine

Mon, 28 Oct 2002 14:10:52 GMT

From Chris McDonough's Meta-Whine:

With only few shining exceptions, these offers almost never pan out and the thing that they offered to do never actually gets done. It's as if people believe that by offering to do something, they've actually done it. [Chris McDonough]

I have to plead guilty to this, with the most recent promises being a release of "FDoc". I learned my lesson long ago - fix it if I can, otherwise, stay quiet. There is rarely enough time in the day to do real work and Open Source work, so why promise? I don't live up to this all the time, but I do try. Speaking of which - it's time to get sparkly clean, shaven, and head to the office. I have a big deadline this week, and I'm sure there are plenty of new projects waiting to attack.

FDoc

Sun, 27 Oct 2002 21:47:53 GMT

I've been asked recently about this FDoc project I keep mentioning. Today, I wrote a new document about it.

Regarding FDoc -- this is the new document, and goes over some of the design ideas as and after they've been put into use.
Compound Zope Documents -- this is an older document, written a few months ago during some of the initial coding of the base framework and a default end-user implementation.

Transitions in waiting.

Wed, 23 Oct 2002 05:55:28 GMT

I'm trying to transition back into posting. But - what to say? I've been working hard during the day (and taking the pleasure of avoiding work when NOT at the office), where I'm away from Radio. I think it's time to move Radio Userland onto my iPod (and go - again! - through the pains of moving Radio) to keep it mobile (the path should stay the same, regardless of whether the iPod is attached to my workstation, my iBook (which is still going unused these days due to a SMC Wireless Barricade whose power adapter with strange requirements is missing in action), or my iMac. Or, I can open Radio up to post to over a blogging API and host it on my always-on workstation. I don't know. I'm never at the machine I want to be on when I have something interesting to say. It may also be time to move to Movable Type or some other server side setup, of which Movable Type seems the most complete. But then there's the issue of moving past posts in Radio to MT. Or I could write up an FDoc based solution for "Zope" and the CMF (FDoc is the pending product name for my compound document project that I've been working on off and on over the past few months), but I don't have the time or will-to-care to do that much coding outside of work. I have other things I'm behind on, like wrapping up the Eucci & Co. entry in No Type's Sine Fiction series, and putting together the More Summer Dress Fire release - I don't even know how big that one's going to be, and whether I should be seeking out putting it on CD, or getting a release on No Type, some other online label, or continue to put it up at Euc.cx. And, ski season is coming up, and I really need to start bringing in more money to support my drinking habit (the Cabana Club has become a second home to me now). It's not that I drink heavily (well...), but that the good Scotch's, the good vodka's, the good beers, etc. And maybe a fine cigar or two. And friends. And then there's all those CD's to import from Japan, such as Pizzicato Five and Yukari Fresh, DVD box sets like The Godfather Series, and the first three seasons of The Sopranos while I sit and dream of a deluxe box set of ALL episodes of Homicide: Life on the Street. And did I mention S k i i n g?? sigh And I didn't even talk about traveling. Oh well, as long as I can continue to do SOME of those, life will be good. There are other aspirations as well. So somehow, "Zope 3" weekend experimenting is a little off the charts right now, but where I get a chance to work with it at the office, I will. And of course, there will be reports here. In the meantime, buy this and this - finest packaging and noises. And in the mean-mean time - the democratic party is dead to me right now, and I'm having a hard time even finding a stomach to cast a vote this year. (there, now this post can land in the political category too).

Projecting What, Exactly?

Thu, 19 Sep 2002 08:07:52 GMT

It's rough working on a project with little or no software design. I'm a little surprised by this, but after a year of being primarily independant or working with a very small company, it's been made very obvious. We try to do something up front in some quick analysis sessions, and even do some agile modeling (of sorts) to sketch out table/object relationships. But it's not long before the typical ruts happen - even the most agile of models can become difficult to update when you're effectively an army of one, when you have no customer contact, and the real mastermind behind the project is extremely busy with their own projects. I can come up with a system out of my ass, but there's so much time being spent doing forced thinking - and most of it off the cuff and right into the application server. Remarkably, most of it is turning out well - just slowly.

A Zope 3 story - augmenting other components

Mon, 02 Sep 2002 08:47:43 GMT

It's very late, so I'll try to make this quick and write up more later. Today, I finally had enough free time in my coffers to download "Python" from CVS (aka - 2.3 pre-alpha) onto my old iMac, and augment that with "Zope 3" - the ComponentArchitecture project. Then I installed the simple but usable Job Board Example. I soon noticed that said JobBoardExample was missing out on a couple of things - an editing interface, and an XML-RPC interface. One of the notions of Zope 3 is that it's supposed to be easier for developers to embrace and extend other components by being able to write new "views" for them. So, I decided to write a JobEditView. I made a new Product (esentially, a plain Python package) called JobBoardEditor, and filled it with a couple of files - EditJobView.py, edit.pt (the page template to edit a single job with), and configure.zcml, the configuration file that ties it into the Zope system. It wasn't long until I could go to a Job object and traverse to my new edit form, and then have the submit go to my 'edit' method. But, at this point, I'm running into security issues that are beyond my measure to figure out, especially at 1:00 am with a beer headache. ";->" My other thing to try out was putting a simple XML-RPC interface on to the JobList. This was particularly interesting to me as I've found that while traditional "Zope" method calls should - in theory - work fine with something like XML-RPC, they seldom do. Sometimes it's because too much HTML is returned from a call, but most of the time lately (for me at least), it's that I want the server to do some processing of the results before sending them out to the client, using the relatively simple data types afforded to XML-RPC. As a result, I've been adding in extra Python Scripts particularly for XML-RPC, and letting them work in the proper places either via acquisition or some other clever tricks. It's not a bad system, but it's not exactly formalized or repeatable, since it's not in a formal Product. In Zope 3, this is different. I wrote a simple XMLRPCView class. 'View' objects in Zope 3 have two state members - 'request' (the incoming request object), and 'context' - the object the View is applied against. So, for me to add in the new functionality whereby I could just get a list of approved Job titles, I wrote the following Python file in my JobBoardEditor product:

from Zope.Publisher.XMLRPC.XMLRPCView import XMLRPCView
from ZopeProducts.JobBoardEx.IJob import JobState
 
class JobListXMLRPCView(XMLRPCView):
    __implements__ = XMLRPCView.__implements__
 
    def listJobTitles(self):
        joblist = self.context
        job_ids = joblist.query(state=JobState.Approved)
        out = []
        for jid in job_ids:
            out.append(joblist[jid].summary)
        return tuple(out)

Then, I added the following to JobBoardEditor/configure.zcml (after adding xmlns:xmlrpc='http://namespaces.zope.org/xmlrpc' to the head zopeConfigure element):

<xmlrpc:view
    name="joblistmethods"
    for="ZopeProducts.JobBoardEx.IJobList."
    factory=".JobListXMLRPCView." />

With this configuration statement, I've added a new View to be published on the XMLRPC channel (which in Zope 3 runs on a different port than normal HTTP requests, which is probably wise) for objects that implement the IJobList interface. I was able to do all of this outside of the JobBoardEx product. I just had to ensure that the Product was added in after JobBoardEx by using the site's products.zcml file to affect the ordering. After restarting Zope 3 (which is a very slow restart on this machine), I was able to do this:

Py$ s = Server('http://localhost:8281/zoo/joblistmethods')
Py$ s.listJobTitles()
['test test test']

I actually had created two jobs through the web interface, but had only approved one. So I ducked in to the web interface to approve the other job, and then made the call again:

Py$ s.listJobTitles()
['test test test', 'the other job']

So, what are some benefits of this? One that comes to mind is issue trackers. Earlier in the day, I was writing a Python script to farm through some Tracker data and format it nicely for XML-RPC, with the intent of writing a simple AppleScript Studio application to list pending/accepted Tracker issues. I did this informally, dropping a Python Script into the folder above a particular tracker. Ultimately, it didn't work - and I think a lot of the blame goes on XML-RPC's utter disregard for semi-proper security. Since Zope's always published objects on the web with regards to web security standards (or embarrasments, depending on your view) such as 'Basic Auth', it's had problems with the "security by passing in arguments" design of many XML-RPC API's. But the point I was really going for was packaging. Even if the Tracker Issue finding Python Script had worked over XML-RPC, now I have to find other places to put it by copying and pasting it, ultimately adding another name onto an ever growing namespace. With Zope 3, however, I could write an XML-RPC view particularly for the application I was designing as a normal Python component, and install and configure it as necessary (and hopefully that configuration work will be better than copying and pasting plain Python Script code). I can name it specially, essentially adding a new namespace for my application. I can distribute it to other users fairly easily. Another example would be implementing the Blogger API not as a new weblog product for Zope, but as a view/adapter component that publishes on the XML-RPC channel, and communicates with other Zope components, either built in ones or third party. In summary - my first tangible Zope 3 experience in quite some time has been hopeful and helpful. There's still a long way to go, but the kernel is shaping up nicely. The pattern driven / "extreme programming" sprint driven development has yielded a decent - but still shifting - code base that should be stronger and simpler than Zope 2. Sometimes, there are enough levels of indirection in Zope 3 to make it look more complex, but so far most code has been easier to follow and trace than Zope 2 (although there are a number of empty folders/packages that probably need some brute force removing from CVS) - at least, it's usually easier to tell what's going on (and even this has an at least - at least, once you start learning the new paradigm). Zope's ancester, Bobo (the kernel on which Zope is still based), was all about abstracting the means of publishing an object on the web away from the object itself. Bobo, and then Principia / Zope 1.x, built upon this, but primarily in a single-UI / single protocol type way. It's obvious today that there are many ways of looking at things on the web, including never actually using the web itself. Zope 3 brings us back to the notion of an object publisher being able to serve up objects on different protocols. Zope 2 offers this today, but there is great difficulty in making WebDAV, normal HTTP, FTP, and XML-RPC all live harmoniously in the current design. And finally, Zope 3 should yield a usable scalable means of adapting the works of other developers into new solutions by providing better control of product/service/view configuration and overrides, such as adding a new XML-RPC API to someone elses bug tracking system without having to alter that bug tracking system, or use secret Python hacks to alter behaviour.

Mac OS X 10.2's FTP and WebDAV issues with Zope

Sun, 01 Sep 2002 02:39:15 GMT

[Mac OS X Hints]: Avoid Finder FTP services in 10.2. Sadly, they're right. FTP Services in the Finder are really sub-par, similar to where WebDAV Services were in Mac OS X 10.0. Even if the FTP Services work, I've had issues with file permissions. I was very happy to mount a Zope server via FTP, go into a Python Script object, and open it up very quickly in Apple's simple TextEdit editor. I made some changes, and tried to save, only to get permission warning after permission warning, with no changes saved. So I tried WebDAV editing of Zope objects from the Finder. I was able to open, edit, and save a file, only to go to the "Zope" server's management screens to see that the SQL Method I was trying to edit had turned into a plain File object. Looking at the Undo Log (Zope's default object database keeps historical transactions that can be rolled back), I noticed that the save command had resulted in a page full of WebDAV operations - and that I had not actually saved to the old object in Zope, but that Mac OS X's WebDAV 'File System' support had renamed the old object, put the new one in as a fresh upload, and then removed the original. As such, Zope had no clue what the new incoming data should be, so it simply made a generic File object. Fortunately, I was able to use Zope's undo capabilities to get my original SQL Method back in place, and I went back to the old standby of editing in the TextArea or via XEmacs' FTP support. sigh. Hopefully a nearby update to Mac OS 10.2 will fix the FTP support so that the upper layers of the OS don't freeze up. And it would be especially nice if the file permissions problem went away too.

Thu, 22 Aug 2002 14:47:47 GMT

A Japanese Summary of yesterdays post, "ZTUtils.make_query". It looks beautiful in Chimera 0.4.

ZTUtils.make_query

Wed, 21 Aug 2002 15:50:42 GMT

Part of the overall Zope Page Templates system is a package called ZTUtils. ZTUtils contains a number of useful functions and objects for Page Template and Python Script usage. Among other things, it contains a range of items for generating and dealing with trees (in replacement of the dtml-tree tag), batching, and assorted smaller helpers. My favorite helper that I use in both Templates and Scripts is make_query. It does what its name says - it builds a query string. But it's more than that - it also properly marshals ZPublisher types, does url quoting, and more. So, gone are the days of building URL's like this:

  RESPONSE.redirect('page.pt?message=All+very+well+then')

And in are the days of:

  from ZTUtils import make_query
  query = make_query(message='All very well then')
  RESPONSE.redirect('page.pt?%s' % query)

While it doesn't look terribly impressive, it's when query parameters start to build up that it's the most useful. And the intent is clearer. A page template example, based on working code:

  <ul tal:define="ztu modules/ZTUtils; MQ python:ztu.make_query">
    <li>
     <a tal:attributes="href python:'./force.py?%s' % MQ(item_id=item.id,status=4)"
     >set all to passed</a>
  </ul>

When the url is generated, it comes out as:

  ./force.py?item_id=IXL0102&status:int=4

Page Template Followup

Tue, 20 Aug 2002 15:56:16 GMT

A couple of months back, I wrote up a series of posts on "Zope" Page Templates. There was "Zope Page Template Joys", "More Page Templates (a buttal of sorts?)", and a screenshot of "GoLive Editing in QuickEdit Mode". As a quick follow-up, I just have to say that with certain exceptions*, I'm never going back to DTML again. With or without a graphic HTML designer, Page Templates are just a much cleaner way to do dynamic pages. * The exceptions are for when dealing with non-HTML/XML dynamic content. Besides SQL Methods (which have special DTML tags that are actually a delight to use), this includes JavaScript and Style Sheets that need dynamic code. Most of the time this tends to be JavaScript code snippets that need absolute URL's (a common Zope-ism), including navigation bars with heaps of rollovers.

Page-Load Handlers for Page Templates

Sat, 10 Aug 2002 04:39:58 GMT

I've been doing a lot of "Zope" and "CMF" work lately, wrapping up a couple of projects and tinkering around with our own internal project site (hence all the recent interest in Bug/Issue Trackers, of which I still don't have a satisfactory conclusion about). There's not much to say except for a fair amount of praise. Zope 2.5.1 is a pretty solid release, and the recent new technologies that have been added to Zope, such as Page Templates, Python Scripts, and long-overdue Session management are pretty much done right. I'm not without a few complaints / desires. The one that really caught my mind today (from an annoyance that I had to bully through under tight deadline a month and a half ago) is this:

Page-Load Handlers for Page Templates

On one recent project, I had to deal with session timeout situations or situations where a user visited a page directly with no session data (their user id or shopping cart contents). The solution that worked out best for me was to just allow certain errors to happen (KeyErrors, etc), and catch that in standard_error_message and report it as "hmm, your session has most likely expired.. Unfortunately, this could mask other errors in the system - testing found most of them, and a "session timeout" is a friendlier error message than most, but I'm still uncomfortable for the situations it may be untrue in. In DTML, it was pretty easy to add a couple of lines of code to catch the situation and raise a redirect error. In Page Templates, it's a bit trickier. And uglier. ASP.NET has a 'Page_Load' event handler a developer could write. It can be used to prepare data for a Page, or to do preconditions. I think that Page Templates might benefit from a similar solution - add a 'Page Load Handler' field that is a TALES expression of its own. I see a fair amount of page templates that weigh themselves down with excessive TALES 'define' expressions at or near the top of a document, sometimes with funky boolean logic to try to prepare data for use later on. Attaching a Page_Load handler that could return data to add to the TALES options context, or return a substitute page in advent of some logic succeeding/failing (ie - a session timeout) could be quite advantageous. It could also benefit situations where Response headers need to be altered. And finally - it would help the separation of Template Logic (aka "display logic") from some of the more programmatic logic that still seeps in to Page Templates from time to time. It's just a thought. There are ways around having such a handler built in, but the ones I've toyed with have proven to be fallible or tricky to maintain.

Another Zope Issue Tracker

Fri, 26 Jul 2002 15:20:53 GMT

I found this Issue Tracker yesterday while looking at ZWiki.org for ZWiki updates. My quest for a good issue tracker is still frustrating. Tracker still comes closest to what I seem to want - something with detail and workflow for team use, but not too difficult for a non-tech person to submit an issue. But I also like the simplicity of this bug tracker by Brent Simmons. I wonder if I might just end up writing my own that's closer to this. Personally, I didn't mind "Roundup" as a bug tracker. It was fast, and the other developer and I that used it during the transition phase of 24Tix 1.0 were pretty succesful with it. But there were some things that I ultimately had too much issue with:

No indication that a post-install schema change was possible. I wanted to add in a couple of fields, such as Expected Version and References, to an already running database. I know now that this is in fact doable.
Configuration and setup of new instances too difficult. Actually, they've done a decent job of this, but really getting Roundup installed requires better Unix familiarity than I have - particularly things like setting up a dedicated Roundup user/group. I'm primarily a developer, not an administrator, and the guy who runs the administrator duties never had time to help me here. This could be a significant issue if I needed to go through him every time I needed to set up a new instance. In Zope, one just adds a new Tracker (Or CMF Collector, or IssueTracker, or what have you). It's a domain I'm personally more familiar with, so I'm going to stay there for the time being.
Difficult to integrate with other Project artifacts. With "Roundup", the issue tracking database was off in its own area, even when using ZRoundup (which is basically a small gateway between Zope and the normal Roundup CGI application). Thus, a few structural and navigational elements I was using in the look and feel of our very basic Projects site would get lost.

Basically, I've come to the conclusion that I'm too steeped in "Zope" for anything else to be useful on the time tables that I have. Tracker is nice now that it's running, but there are still small gotchas and rough areas. Right now, our Project Management site is a loose collection of folders, letting the person responsible for a certain project to decide how they want to track artifacts. Most of my projects use a combination of ZWiki and Tracker, to some degree of success. It's lightweight and easy to get up and running for a new project, and I've had success with this combination before. As things grow, however, real content management might be needed. Perhaps when CMF hits 1.3 and Plone hits 1.0, I'll reconsider CMF + CMFCollector. But the solution we have now works decently.