Wednesday, March 19, 2008

Secret Teachings: The Programmer's One Inch Death Punch How do programmer's get better at their job? Few programmers think of programming as a true profession. For most it's just a job. They go to work, do what's asked, and go home. They don't read. They don't attend conferences. They don't train. In fact, the training business for software is pretty much dead and has been for years. Other professions have different attitudes. Professional athletes train constantly to improve their skills. Even doctors and accountants have stringent continuing education requirements so they stay current. What can we programmers do? Let's first look at how expert performers are created in the first place. That might help us figure out how we can become better. Time in an article called The Science of Experience has a lot of potentially fruitful ideas: The number of years of experience in a domain is a poor predictor of attained performance. Rather than mere experience or even raw talent, it is dedicated, slogging, generally solitary exertion, repeatedly practicing the most difficult physical tasks for an athlete, repeatedly performing new and highly intricate computations for a mathematician, that leads to first-rate performance. And it should never get easier; if it does, you are coasting, not improving. They key is "deliberate practice," by which is meant the kind of practice we hate, the kind that leads to failure and hair-pulling and fist-pounding. You like the Tuesday New York Times crossword? You have to tackle the Saturday one to be really good. Great performance comes mostly from deliberate practice but also from another activity: regularly obtaining accurate feedback. Experts tend to be good at their particular talent, but when something unpredictable happens, something that changes the rules of the game they usually play, they're little better than the rest of us. Entire classes of experts — for instance, those who pick stocks for a living, are barely better than novices. (Experienced investors do perform a little ahead of chance, his studies show, but not enough to outweigh transaction costs.) Researchers found that élite skaters spent 68% of their sessions practicing jumps, one of the riskiest and most demanding parts of figure-skating routines. Skaters in a second tier, who were just as experienced in terms of years, spent only 48% of their time on jumps, and they rested more often. Experience is not only insufficient for expert performance; in some cases, it can hurt. Highly experienced people tend to execute routine tasks almost unconsciously. Experience in a particular task frees space in your mind for other cognitive pursuits,  wondering what's for dinner, answering your cell, singing along with Justin Timberlake,  but those things can distract you from the accident you're about to have. Experience can also lead to overconfidence: a study in the journal Accident Analysis & Prevention found that licensed race-car drivers had more on-the-road accidents than controls did. None of this trips my BS meter. It makes a lot of sense and jibes with experience. We all know people with 10 years resume experience in X who couldn't program their way out of a paper bag while someone with just 1 year of experience continually delivers quality results. And I know I've grown the most when truly challenged to solve new problems. Given the science of experience article I think Coding Dojos seem like a potential solution to the programmer training dilemma. A coding dojo is "a meeting where a bunch of coders get together to work on a programming challenge." It's a form of deliberate practice, which is "not the same as experience gained while doing your job. It is when you actually seek out experiences that will stretch your skills just the right amount, and give you feedback that enables you to learn." For thousands of years martial arts have been taught in a deep mentoring relationship using a long progression of increasing difficulty and challenge. The path from a white belt to a black belt is long, it's hard, but in the end you learn. You learn through constant practice and challenge. It would be interesting to consider how a similar infrastructure could be setup to train programmers. comment[] 3:53:40 PM       digg   reddit

Friday, February 29, 2008

Web 2.0 Suicide Monitoring Using Twitter and Emotional Presence People on anti-depressant drugs--like Prozac--are supposed to be closely monitored for suicidal thoughts that could indicate the drug is having a "paradoxical result." While many feel better on anti-depressants others drop fast and dark into an even worse suicidal depression. Paradoxical isn't quite the word I would use, but we must keep everything clinical. Monitoring allows a doctor to detect if a patient is entering the paradox zone. If so, treatment can be changed and further harm avoided. I was thinking one potentially Web 2.0 way to monitor people's internal subjective state--their feelings and emotions--on an unnaturally frequent basis would be to combine Twitter with emotional presence and a bot that would notify a doctor if certain downward emotional trends were detected. I've been doing some work on a Jabber IM client lately, and I've done some work using the Twitter API, and I've done quite a bit of research on emotion (patent pending), so a mashup of these services seems a pretty natural way of helping people stay alive through their dark times. In IM (Instant Messaging) your presence is broadcasted to your contact list so everyone knows what you are doing and you're availability to others. Using your IM client tells everyone you are available. Don't use your IM client for awhile and and everyone will learn you are away. Pickup the phone, mark your presence as "On Phone" and everyone's IM client will associate your name with a cute little phone icon. And when you close down your IM client everyone will learn you are now unavailable. There's also an idea of emotional presence, often represented by emoticons. If you are happy or sad or angry you can broadcast your emotional presence in the same way you can broadcast your physical presence. Select an option that matches your current feelings and the whole world will instantly know how happy you are that it's Friday and a long weekend awaits. Now let's extend the emotional presence to indicate presence information for thoughts of suicide. I don't know what these would be, but I'm sure doctors could work up something. Say you have a fleeting thought of suicide you could quickly change your emotional presence to indicate your new state. More severe thoughts could have different icons. And so on. Now let's bring in Twitter. Twitter is a microblog. Its purpose is to share brief bits of what is currently happening in your life. That's the perfect match for emotional presence. You could also indicate with each post how you are feeling. These responses can be directed to a channel using the "@reply" syntax in Twitter. Doctors could follow those posts for their patients by briefly taking a look at how they are doing. Or a specially created bot look for certain trends and notify a doctor if a negative trend developed. This would allow a doctor to intervene much more quickly than they could otherwise and the information they are making their decisions on would be much more accurate because it's harder for people to fudge on their self-reports when they are in the moment. With the perspective of time we all do a lot of self-editing, but in the moment you are more likely to be honest. Clearly privacy is an issue. Users need to be able to select who sees what kind of presence information. But that's necessary anyway and exists in some form now as privacy lists. The type of information to block or allow simply needs to be extended to more granular types of data. What's great about this approach is Twitter is everywhere users want to be. On cellphones, browsers, IM, and desktop applications. Users will always be in touch with their emotional presence and doctors can always follow their progress. Wouldn't it be wonderful if we could read a lot less about people on anti-depressants committing suicide? It just always seems so wrong that people who are trying to get help end up dying. comment[] 9:54:48 AM       digg   reddit

Monday, December 31, 2007

The New Mass Transit: People Pod Pool of On Demand Self Driving Robotic Cars who Automatically Refuel from Cheap Solar Our traffic in the San Francisco Bay area is like Dolly Parton, 10 pounds in a 5 pound sack. Mass transit has been our unseen traffic woe savior for a while now. But the ring of political fire circling the bay has prevented any meaningful region wide transportation solution. As everyone scrambles to live anywhere they can afford, we really need a region wide solution rather than the local fixes that can never go quite far enough. Commuters are Satisfied Not Carpooling You might think we would car pool more. But people of the bay don't like carpools and they don't much like mass transit either. In the Metro, a local weekly, they publsihed a wonderful article Fueling the Fire, on how we need to cure our car addiction using the same marginalization techniques used to "stop" smoking. A telling quote shows how difficult going cold turkey off our cars will be: Mitch Baer, a public policy and environment graduate student at George Mason University in Virginia, recently surveyed more than 2,000 commuters in the Washington, D.C., area. He found that people who drove to work alone were more emotionally satisfied with their commute than those who rode public transportation or carpooled with others. Even stuck in traffic jams, those commuters said they felt they had more control over their arrival and departure times as well as commuting route, radio stations and air conditioning levels. Commuters said that driving alone was both quicker and more affordable, according to the study. "They will have a tougher time moving people out of their cars," Baer said. "It's easier for most people to drive than take mass transit." The key phrase to me is: people who drove to work alone were more emotionally satisfied. How can people jostled in the great pinball machine that are our roadways be emotionally satisfied? That's crazy talk. Shouldn't we feel less satisfied? In Our Cars We Feel Good Because We Are in Control Solving the mystery of why we feel satisfied while stuck in traffic turns on an important psychological clue: the more we perceive ourselves in control of a situation the less stress we feel. Robert Sapolsky talks about this surprising insight into human nature in Why Zebras Don't Get Ulcers. Notice we simply need more "perceived" control. Take control of a situation in your mind and stress goes down. You don't actually need to be in more control of a situation to feel less stress. If you have diabetes, facing your possibly bleak future can be less stressful if you try to control your blood sugars. If you are a speed demon, buying a radar detector can make you feel more in control and less stressed as you zoom along the seldom empty highways. If you are bullied, figuring out ways to avoid your torturer puts you more in control and therefor less stressed. Figure out a way to control and an out of control situation and you'll feel happier. That's what I think we are accomplishing by driving alone in cars. In our car we have complete control. Cars our are castles with a 2 inch air moat cushion. Most cars are plusher than any room in your average house. Fine leather, a rad sound system, perfect temperature control, and a nice beverage of choice within easy reaching distance. In our cars we've created a second womb. The result is we feel more control, less stress, and more satisfaction, even when outside, across the moat, a tempestuous sea of stressors awaits. Our Mass Transit System Must Supply Perceived Control Given the warm inner glow we feel from being wrapped in the cold steel of our cars, if you want people to get out of their cars and onto mass transit you must provide the same level of perceived control. None of our mass transit options do that now. Buses are on fixed schedules that don't go where I want to go when I want to go. Neither do trains, BART, or light rail. So the car it is. Unless a system could be devised that provided the benefits of mass transit plus the pleasing characteristics of control our cars give us. With Recent Technological Advances We Can Create a New Type of Mass Transit System New technologies are being developed the will allow us to create a mass transit system that matches our psychological and physical needs. Just berating people and telling them they should take mass transit to save the planet won't work. The pain is too near and the benefits are too far for the mental cost -benefit calculation to go the way of mass transit. The technologies I am talking about are: Inexpensive solar with $1/watt solar panels. Our mass transit must of course be green and cost effective. Breakthrough battery could boost electric cars. Toshiba promises 'energy solution' with nearly full recharge in 5 minutes. Personal transportation pods. A reusable vehicle that can take anyone anywhere they want to go. Self driving vehicles. We are making great strides in creating robot cars that can drive themselves in traffic, better than most humans can drive (low bar, I know). Mix these all together and you get a completely different type of mass transit system. Create a People Pod Pool of On Demand Autonomous Self Driving Robotic Cars that Automatically Refuel from Cheap Solar Many company campuses offer a pool of bicycles so workers can ride between buildings and make short trips. Some cities even make bikes available to their citizens. The idea is to do the same for cars, but with a twist or two. The cars (people pods) can be stored close to demand points and you can call for one anytime you wish. The cars are self driving. You don't actually drive them and are free to work or play during transit. Different kinds would be available depending on your purpose. Just one person on a shopping trip would receive a different car than a family. The pods would autonomously search out and find energy sources as needed to recharge.There's no reason to assume a centralized charging and storage facility. When repair was needed they could drive themselves to a repair depot or wait for transportation. The advantages of such a system are: Perceived control. You have your own person car that you control the destination for, the interior environment, and your own actions. This gets over the biggest hurdle with current mass transit options. Better regional traffic flow. The autonomous cars could drive cooperatively to smooth out traffic jams. Go where you want to go. It would be used because people can go to exactly where they need to go and be picked up exactly where they need to leave from at exactly the time they wish. None of these are characteristic of current systems. Leverage existing road ways. Creating light rail and trains is expensive and wasteful (except for the high speed point-point variety). They don't extend to where people live and they don't go where people go. So it creates a multi-hop mess out of every trip. We already have an expansive road system that goes where everyone wants to go. Using the road infrastructure more efficiently makes a lot more sense than creating hugely expensive partial solutions. And since these cars would be eco-friendly, most arguments against using cars fall away. Cheaper delivery. One force keeping truly distributed manufacturing from blossoming is high delivery costs. A $2 item is simply to expensive to buy remotely and ship because the shipping costs more than the product. An automated transportation system would make this model more affordable. Live where you want to live. Most mass transit systems are based on trying to socially reengineer our current suburbian and exurbian living pattern into a high density live-work pattern. While this should be an option, most mass transit proposals assume this pattern as a given and can't deal with current realities. For the foreseeable future people will not give up their houses or their lifestyles. The People Pod approach solves the mass transit problem and the "difficulties" of having to change a whole populace to behave in a completely different way for less than compelling reasons. Still can own your own car. This isn't a replacement for the current car culture. It's leveraging the car culture. You can still own and drive your own car. Nobody is trying to steal your car away from you. Cleaner and safer. Mass transit is disliked by many because it is perceived as dirty and unsafe. The pods would be safe and clean. Road safety. Our robot overloads will make our lives safer. Hopefully... Funding: Current transportation budgets. There's lots of money that could be redeployed from existing less than successful approaches. Advertising. The outside of vehicles could contain advertising as could the inside, especially from the internal search system. Imagine wanting a new place to eat and asking the pod to suggest one. That's prime targeted marketing. Social networks and massive multi-player games could also be created between pods. Efficiencies. The plug-in cars are electric and efficient and low maintenance. That will save a lot of money. Up sells. Individuals could buy their own pods and trick them out. Also, people could pay for a higher class of pod from the pod pool. Licensing. Technology used in making the pods could be sold to other manufacturers. Create a standardized market so competition and cooperation can erupt. Sponsorship. Companies could buy rights to play music, stock the food locker, use their equipment, etc. Naming rights. The rights to name parts of the system could be sold. Implementation: Challenge prize. Maybe someone with a vision and a dream can put up a $50 million prize to get it going. Something like the Xprize. Government funding. Don't laugh, it might happen. Startup. I'm available if interested :-) With a large enough challenge prize this is a viable model. After a lot of reading on the topic and a lot of self-examination on why I am such a horrible person that I don't use mass transit more, this is the type of system I could really see myself using. It doesn't try to change the world, it uses what we got, and gives people what they want. It just might work. comment[] 11:24:47 AM       digg   reddit

Tuesday, December 18, 2007

Agile Owes More to Aristotle than the Renaissance There's an interesting historical parallel between Agile software development, traditional large organizational software development and Aristotle and the new Renaissance politics of  Machiavelli. Of course the part of Agile development is played by the great Greek genius Aristotle and the part of the evil giant organization development group is played by Machiavelli. Could it be any other way? Machiavelli's The Prince, the longest please hire me resume ever, revolutionized politics by turning the long accepted ideas of Aristotle into compost. Aristotle thought the state was founded on friendship and trust.  To have a state you need a bond. That's how a gang becomes cohesive and turns into a team. That's how soldiers stand side by side together in battle against constant challenge and danger. And the basis of that bond, the basis for the state, must start with friendship and trust. The state can never be sustained by fear. The state succeeds based on personal morality. Sound like Agile now? You thought I was just crazy when I started out, didn't you? In a sure made for TV thriller, Machiavelli flat out contradicts Aristotle. This was a big deal in big M's time, for Aristotle was simply known as The Philosopher, and was assumed right about pretty much everything. It took some big ones to disagree with Aristotle. Machiavelli argues the basis of the state is fear of the Prince and the system of coercion the Prince creates to ensure the continuance of power. A Prince says if you do X here's the punishment or if you don't do Y here's the punishment. There's no weak minded friendship or trust or need for a unifying bond in big M's world. The Prince to stay in power must exercise power. What holds a people together is fear, fear of the Prince. Does this sound something like your usual software development group? Orders radiate down from the top and you are expected to follow orders under pain of death march. Aristotle's notion of political science is empirical (again, like Agile). A study of all the Greek city states was performed with the goal of finding what worked in all those cases so an ideal of how a perfect state is to be run could be established. In my rather strained analogy I'll say this is something like Scrum. An ideal algorithm of how to run a project rather than an exact prescription of every detail. Machiavelli is also empirical in nature, but he is more hard cruel world based. He wants to stay flexible because what works in one situation wont work in another. This sounds a bit Agilish, but it's not. What matters to big M is establishing and maintaining order.  The Prince must do whatever it takes to stay a Prince so you can't be wedded to any quaint notion of an ideal. Personal morality is quite separate from a Prince's morality. The Prince must be free to act in anyway necessary while people must act in strict accordance with the Prince's wishes or there is punishment. This is very similar to the notion that a corporation can justify actions by appealing to a fiduciary responsibility that an individual could never get away with. Who said history wasn't relevant to today? If you look irrationally hard we can see the same struggles faced today writ large in our past. comment[] 5:08:52 PM       digg   reddit

Monday, November 19, 2007

Can Game Theory be Used to Loosen Website API Limits? Let's say Twitter limits me to getting only 20 tweets at a time through their API. But I want more. I may even want to do something so radical as download everything. Of course Twitter can't let everyone do that. They would be swamped serving all this traffic and service would be denied. So Twitter does that rational thing and limits API access as a means of self protection. As does google, yahoo, and everyone else. But when I hit the limit I think, but hey it's Todd here, we've been friends a long time and I've never abused you. Can't you just trust me a little? I promise not to hurt you. I never have and won't in the future. At least on purpose, accidents do happen. The problem is Twitter doesn't know me so we haven't built up trust. We could replace trust with money, as in a paid service where I pay for each batch of downloads, but we're better friends than that. Money shouldn't come between us. And if Twitter knew what a good guy I was I feel sure they would let me download more data. But Twitter doesn't know me and that's the problem. How could they know me? We could set up authority based systems like the ones that let certain people through the airport security lines fast, but that won't scale and I have feeling we all know how that strategy will work out. Another approach to trust is a game theoretic perspective for assessing a user's trust leve. Take the iterated prisoner's dilemma problem where "tit for tat" is a surprisingly simple winning strategy. We start out cooperating and if you screw me I'll screw you right back. In a situation where communication is spotty (like through an API) there can be bad signals sent so if people have trusted before then they'll wait for another iteration to see if the other side defects again, in which case they retaliate. It seems like if services modeled the API like a game and assessed my capabilities by how we've played the game together, then capabilities could be set based on earned and demonstrated trust rather than simplistic rules. A service like Mashery can take this approach a step further because they can assess how a player plays in a wider playing field. If someone vouches for you to a friend then you will likely get more slack because you have some of the trust from your friend backing you. This doesn't work in one on one situation because there's no way to establish your reputation. Mashery on the other hand knows you and knows which APIs you are using and how you are using them. Mashery could vouch for you if they detected you were playing fair so you get more capabilities initially and transit the capability scale faster if you continued to behave. Of course we get the situations like on Ebay where people spend eons setting up great reputation only to trade their reputations in for cash in some fabulous scam. That's what happens in a society though. We all get more for the price of some risk. comment[] 10:19:05 AM       digg   reddit

Wednesday, October 31, 2007

The Internet's Immune System in Arms Race with Digital Rights Management A popular image of the internet it to think of it as a vast interconnected intelligent organism, like this gorgeous map of the internet. If the image is true then who is the heart, the brain, the liver, the nervous system, or the alimentary canal? I'm sure you have candidates for each role :-) The immune system though I think is us, we the citizens of the internet. And we as the internet's immune system react to DRM as an invading organism: destroy ! destroy! destroy! The parallels between the arms race between invaders and the immune system and DRM and netizens are uncanny. The job of an immune system is to detect foreign invaders and destroy! them. This is harder than it looks. Distinguishing between an unwanted visitor and your liver takes some careful work. But once your immune system figures out what's you and not you, it attacks. Some organisms have figured out a way to beat the system by waiting a while for the immune system to detect them as foreign and then rapidly evolving by changing their DNA. This causes the immune system has to start the whole process all over again. We were all brought up on microevolution. Evolution happens by small point/insertion/deletion changes. But the scientist Barbara McClintock discovered something radical, that genes could jump around chromosomes. This means genetic changes could be radical and immediate, not small and slow. Of course Ms. McClintock was ostracized by the scientific community for speaking a heresy. But modern science finally caught up to her, many decades later, and she won her Nobel prize. Why this matter is because invading organisms jump their genes around to cloak themselves from our immune systems. Then in effort to keep up the immune system does the same thing. It hits the big randomizer button in the hope that it can find a way to kill the invader. It's like chasing each other through mirrors. DRM, the internet, and netizens follow a very similar arms race. DRM systems are constantly changed and upgraded to evade we the immune system. But the massive distributed immune system that is the internet community starts simultaneously evolving and attacking the DRM until someone, somewhere finds a way to beat it. Then the process starts all over again. Shouldn't we be smart enough to think ourselves out of this arms race and stop acting like little Darwin machines? comment[] 1:46:01 PM       digg   reddit

Wednesday, July 25, 2007

Announcing My New High Scalability Site The site is http://highscalability.com/. And oddly. it talks about building highly scalable web sites and other systems. People always have questions about how to build a bigger and better website. The information is out there, but it's not easily accessible and often it doesn't make sense if you don't already know what you are doing. So, having some experience in the area, I thought the topic was worth its own site. Please take a look if you have the time. And if you have some expertise please consider contributing. Scalability is a giant field, so the more folks the better. comment[] 11:02:46 AM       digg   reddit

Thursday, July 12, 2007

Gordon Ramsay's Lessons for Software Take Two Gordon Ramsay began a new season of tasty lessons for gourmet software chef's trying to run their own restaurant, err software development group. You may want to read what we learned in the first season of Gordon's tough love school of restaurantary. What new chicken nugget size bits of wisdom have we learned so far? Always put the nights earnings in the bank. Don't cook everything on the same grill. Use separate pans or flavors from last weeks fish will be in today's veggie medley. Don't double book. You can't turn more than one table an hour. Pride and arrogance are what stops you from learning from the provocative lessons dancing in front of your face. Lazy ways of cooking taint everything you make. Don't piss off the locals. If the locals aren't happy then you won't make money during the off season. Create vivid experience based learning sessions. To reduce pride in a charge take them to a bullfighting ring and have them fight an angry bull with nostrils flaring for vengeance. Arrogance fades when a giant multi-horned bull tries to kill you. And once the arrogance melts some learning can occur. Don't be too clever. Use local ingredients and cook cleanly and simply. Let people taste the food. You know you are doing well when you aren't stressed; you are communicating; everything seems easy; dishes come back clean; customers leave happy; and you make money. Eat your own dog food. Eat at your own restaurant and experience what it's like from your customer's perspective. Do you like what you experienced? Run your pub, not your kitchen. It's easy for a cook hideout in the kitchen. You own the business, so run the business, let the chef cook, and let your people do their jobs. Don't try to be something you are not. If you are a pub then act like a pub. A pub serves well cooked, simple, traditional food. Don't be a fancy restaurant if that's not your market. Working a lot of hours doesn't mean you are doing a good job. It just means you are working too hard. Don't hord junk. Keep what you need to do your job and get rid of all the clutter that keeps you from your main purpose. Do any of these lessons apply to building software in a team? I think some of them do. But you're the chef. Hey, Pat Kennedy caught the Gordon Ramsey bug too in his post <A HREF="http://www.gurtle.com/ppov/2008/02/04/gordon-ramsey-is-a-great-consultant/">Gordon Ramsey is a great consultant</A>. Pat's say Gordon exhibits several key traits: confident – the meek might inherit the earth but they’re rubbish at getting the job done experienced – having done it all before he knows what he’s talking about and everyone knows it well rounded – it’s not just about the cooking, to run a successful restaurant you need to know about every aspect of the business eager to teach – he’s not a pompous prat who refuses to share his knowledge and experience, he gives it willingly And he: Demonstrates simple ‘tricks of the trade’ that can be the difference between staying afloat or going under. Breaks the dire straits situation down to individual problems which make answers become simple. Gives the outsider’s perspective is a big advantage. Identifies actions having the greatest impact in the shortest possible time. Gets people skilled-up and self-confident. Ramsey may be a total a**hole, but that doesn't mean we can't learn from him. comment[] 9:26:42 PM       digg   reddit

Monday, July 02, 2007

The Golden Spike Pattern J Wynia wrote an interesting post It's Time to Decouple Your Development Process where he talks about one of the most useful patterns a group of developers can use, it's a pattern I know as the Golden Spike Pattern. The name comes from the 1869 ceremony that drove the final spike completing the transcontinental railroad in the US. The railroad linked the eastern US with the western US, finally meeting up at Promontory in Utah. Constructing a transcontinental railroad is a complex time consuming business. They made it work by researching a general route for the railroad, acquiring the necessary property, and then having two sides building their parts of the railroad independently, The metaphor behind the pattern name isn't subtle. The idea is groups don't have to be in lock step dependency to create a complex system. All they need is a general plan and an agreement of where to meet up. In software a meet can be an API, schema, library, protocol, or a test suite. Once that meet up point is established all the different sides can go about there business without interdependencies. From a lean manufacturing point-of-view this is a powerful way of increasing overall system flow because it's dependencies that cause wasted time. Each side can work out their own issues in their own way, as long as they meet up. And how would you like spend a decade building a railroad and not have it meet up at the right place! comment[] 11:31:14 AM       digg   reddit

Saturday, June 30, 2007

Copernicus is My Favorite Pattern In interviews it's common to ask "What's your favorite and least favorite pattern?" My usual answer for favorite pattern is "keep separate things separate." It's a bit meta  which allows me to talk about a few design principles that are dear to me. My least favorite pattern is the wretched visitor pattern because it binds together different parts of an application that have no business even knowing about each other. It creates a BLOB. After having read "It Started with Copernicus" by Howard Margolis I am going to have a new favorite pattern: the Copernicus Pattern. I will always hate the visitor pattern, so my answer to that question is not likely to change :-) The premise of this book is that Copernicus' discovery of the heliocentric model of the solar system started a fire storm of scientific invention, not because of the discovery itself, but because it spread the idea that we puny humans could think and make big discoveries about the universe using  nothing but our tiny brains. Copernicus gave people permission to tackle big challenges and the confidence that they could expect to meet them. As evidence Margolis lists the major scientific discoveries made before 1600 and the major scientific discoveries made after 1600. Copernicus published his "discovery" that the Earth revolves around the Sun in 1543. In the list of pre-1600 major discoveries there is: nothing. Zip. Nada. After 1600 the pace of scientific discovery blossoms, we discover: the distinction between electricity and magnetism; law of free fall; Galilean inertia; Earth is a magnet; theory of lenses; laws of planetary motion; various discoveries from the telescope, like sunspots; laws of hydrostatic pressure; and synchronicity of the pendulum. All these discoveries were made by Stevin, Gilbert, Kepler, and Galileo all followers of Copernicus. Copernicus discovered discovery using what Margolis calls "around the corner" reasoning. Around-the-corner-reasoning is a habit of mind where you go beyond what is directly seen and take the unexpected step. Stubborningly pursuing a problem even when no clear solution is in sight. Before 1600 most things that can be discovered by direct ways of thinking were discovered. After Copernicus around-the-corner-reasoning helped us start discovering how the world really worked. I highly recommend the book if you want to see a full development Margolis' argument. My already long delayed point follows a similar line of reasoning... What is important about patterns is the idea of patterns and not any particular pattern itself. Many have pointed out the weaknesses of patterns. Patterns can seem like trivial well understood solutions to common problems. Patterns can appear overly formal and lead to complex systems of frameworks that complicate more than they help. Patterns can pollute your code and lead you away from doing the simple thing. For me finding and using patterns is essential because patterns are simply software systemizations. A pattern is that bit of around-the-corner-reasoning that can create massive simplifications and improvements in your system, if you would just take the time to understand your system, see what's really going on, and think a bit how it might be improved. Programming isn't just writing one damn line of code after another. Patterns are the idea that you can continually find ways to  make your system better. If that's using the publish-subscribe pattern, resource acquisition is initialization pattern, or whatever, it doesn't matter. What matters is the idea that a system evolves over time to have inefficiencies and that the system can be brought under control again by  systematically applying patterns of improvement. comment[] 4:29:33 PM       digg   reddit

Wednesday, June 20, 2007

All the world is a DOM. The rise of Identity Based Programming. In the past few years we've seen a huge rise of successful systems built following a Document Object Model (DOM) type of architecture. By that I mean: open systematized models of complex domains that are easy for applications to specialize and extend in a cooperative manner. This approach has quickly taken over the tradition libary + statically compiled language paradigm in amazing products like Eclipse, Aspect Oriented Programming (AOP), DOM + Javascript + DHTML, and in Content Management Systems (CMSs) like Drupal. While we must pay homage to Smalltalk for its unified image based development environment, the most familiar example of the DOM architecture runs inside your favorite browser, using the dynamic trio of DOM + Javascript + DHTML. The olden days of interface writing is represented by the applet. You combine all your libraries into a single application and download it to an interpreter. This is basically the same as compiling to an image and running the resulting executable on your favorite OS. Look how different is your browser's DOM based approach. The DOM provides a really rich and functional model of a document, your web browser, which is made available to all applications running in your browser. Multiple libraries can be directly loaded into the browser and build on layers of each others functionality. Applications can add new methods, events, and properties directly to the model. And powerful meta tools like jQuery are creatable because of the dynamic power and regularity the browser environment provides. One of the most stunning success stories is Eclipse's DOM type architecture of an IDE. If you would have told me you could make a world class development platform by defining a plugin architecture and then weaving together a myriad of plugins from a bevy of developers, I would have said you are crazy. But it works and Eclipse is now the premier IDE on the market. They succeeded by creating an IDE model, which in my world is a DOM, and then made the IDE buildable by composing parts formed from a generic plugin architecture. That this works so well was surprising to me and very enlightening. Another interesting application of the DOM type model is in CMSs like Drupal and Joomla. A CMS is like a big web site construction kit allowing you to create your personal idea of a website from the work of 100s of other people. Drupal makes this possible through their module system that allows modules to be composed together. Drupal's role is to provide an open model of what it means to be a CMS, a set of standards about how everyones modules can play nicely together, and mechanisms for composing the modules together in meaningful ways. For example, forum systems allow readers to make comments on posts. Usually that comment system only works for forum posts, but Drupal takes it one step further and allows the ability to create a comment module that can be attached to any other part of the system, like blogs or polls. Taking extension yet another step beyond is the Content Construction Kit (CCK) module for Drupal. CCK allows you to create new content types and extend existing content types with new features in a type specific manner that is understandable by Drupal. For example, let's say I am creating an event system and I want my event to support geolocation. With geolocation we can show events on Google maps and do other cool operations like calculate how far apart events are. Traditionally I would build location attributes and functionality into my event system and this would take a long time, even using a third part library. I would have to change the database, make a module, make it configurable, add theming, and so on. With CCK I don't have to start from scratch anymore. I can fold into my event the nifty preexisting location module that already exists. CCK will add the location fields, hooks, and widgets into my event in such a way that Drupal can see them as one thing, even though they aren't one thing. I can take this even further. Let's say I want people to be able to comment on my event, so we weave in a comment module. I also want people to rate the event, so we weave in a rating module. Next I want people to able to submit events like Digg and vote them into a front page, so we weave in the voting module. A very complex event has been created by weaving in different aspects together in a unified whole. This may sound a bit like AOP but it is much more powerful because what is being weaved together happens within a complete CMS system model. New content types are natively able to take advantage of CMS features like theming, installation, configuration, upgrading, security, user permissions, and database configuration. It's not just AOP's attachment of a bit code at an insertion point, you are creating a niche in a complete interdependent ecosystem. So what is an event really if it's defined primarily by weaving together different aspects? Here's where we get to the "Identity Based Programming" (IBP) part of our tour. Why IBP? Because all programming ideas these days have to have 3 letter acronyms where the middle letter is preferably a "D" for driven or "O" for oriented. I am really bucking the trend here using "B" for based. An object is usually said to have identity, state, and behavior. I have always contended an object simply has identity, state and behavior arise from relationships with other objects. For example, the date of an event is usually considered an attribute of an event object and this is part of its state. But in reality the date is in a 1-1 relationship with the object and thus isn't part of the object. This is the same for all attributes of the event, like location, comments, votes, and ratings, along with their associated behaviors. What ties the state and behaviors together is the identity of the event they are in relationship with. And here's where DOM type architectures and IBP meet. A DOM sets up the system model, the extension points and facilitation protocols. The process of composing and gluing them together to create new applications is what I am calling Identity Based Programming. Applications in Eclipse, your browser, and Drupal are all built this way. You can say I am not brining anything new to the party and you are absolutely right. I just think it's very interesting to see this approach evolve separately and almost independently in several different application spaces. This approach is fundamentally different than what has gone before, but it's very difficult to pin down exactly what is different. I just thought I would take a stab at defining what is different and to me one of the most important ideas is that identity is what holds all this independent parts together when traditionally we have tried to force these parts to be one whole object. Biological life works by accretion. Parts keep being added on to existing systems rather than being thrown away and redesigned from scratch. In your brain you'll still find the brain of the lizard, mammal, and the primate. Haman brains were made by adding on. With Identity Based Programming we see the same process happening in building software. comment[] 12:31:35 PM       digg   reddit

Tuesday, May 15, 2007

Morning Scrum Meetings are Like the American Family Dinner In the US family sit-down dinners are a cherished tradition. Many Americans have fond and powerful memories of sitting down around table with their family at night and eating together. I realize other cultures have very different dinner traditions, but I was struck how much like a family dinner is the morning scrum meeting. In a family dinner there is a sense of community. You are surrounded by people who support you, who are there for you, and who will help you when times turn tough. During the day you battle the world and then you come home to the comfort of the family circle that is the dinner table. You gather together. It's a time to talk to each other, reconnect, and figure out how everyone is doing. Every gathering is a ritual act of building community. Scrum meetings create a very similar sense of family and community. Without a regular meeting we are just individuals carrying out tasks like machines. It's being part of a team that helps elevate work to life. comment[] 9:40:29 AM       digg   reddit

Wednesday, April 18, 2007

Top 10 Things to Do Now that Your Blackberry has Crashed WNBC reported a major outage affecting 100% of Blackberries in the US. What might dedicated crackberry users do with all this unscheduled downtime? 10. Solve world hunger. You now have the time. 9. See a movie all the way through. No interruptions. 8. Go for a run. Without your crackberry you weigh less and you'll be able to run farther and faster. 7. Contemplate the transitory nature of the universe. If an essential service like the crackberry can fail, what else in your life might fail you? 6. Have a drink. Surviving off the grid is stresseful. Did my team win last night? What time is that meeting at corporate? How is my portfolio performing? Did Jughead really sleep with Veronica? Gaping holes are bound to open up in your digital life without instantaneous answers to important questions like these. So just relax. Have a pop or two. 5. Keep twirling your thumb wheel. You want to be in tip top shape once it's back up. You'll be way ahead of the other kids who have spent their time less productively. 4. Play hall hockey. Crackberries slide really well on the floor. Get two teams together, setup two goals, and see who can make the most goals with your new puck. If you are alone find a lake and see how far you can skip your crackberry. I bet you can't slice more than 5 hops. 3. Remember the 5 stages of grief: denial, anger, bargaining, depression, acceptance. Let's help you dial through the stages quickly. Yes, your network is down. Mistakes happen. It's part of life. The digital gods cannot be appeased, so don't even try. There are no tasty binary cookies or digital flowers that can patch panel this one. You feel depressed now, see (2). The last stage is bunk. Deny deny deny. That's how we do it. Acceptance is for losers. 2. Schedule an appointment with your therapist to help you cope with your loss. But, wait, your crackberry is down! Noooo! The irony of it all! 1. Make a pair of glasses using two tubes from empty toilet paper rolls. Viewing the world outside the small window of your crackberry can be disorienting at first. You'll want to transition slowly to view the world in full resolution. Every hour slice an inch or so from each roll so you'll gradually see more and more of the real world. comment[] 7:52:51 AM       digg   reddit

Tuesday, March 20, 2007

I Thought of Twitter for Eclipse, Too Late, Drats! A few years ago I had the idea of a Twitter like add-in for Eclipse. Too late now I guess. He who codes first wins! The idea was pretty simple. Eclipse would broadcast out to all team members every major action that was completed in Eclipse. So when you created a new class, a new method, etc everyone on your team would know. Likewise, if you were starting something new or wanted everyone to know how close you were to done you could just broadcast it to everyone else's Eclipse instance. And if you had a question you could just blast it out. The idea was to reduce the cycle of feedback to almost nothing. Information flow dominates development time and the more you can reduce blocking on a project the faster you'll move. How it would work, for example, is if I read that you created a SerialLine class and I knew of one already, I could tell you immediately that one already existed in package XYX and you could reuse it. Or if I was interested in knowing how we handle regular expressions in a project I could just type the question in. This kind of informal communication is less scary to team members. Many people are very concerned about their image so they won't ask a question on an email list. They are afraid of looking dumb or weak. The quality of communication is inversely proportional to the risk of ego destruction. Programming is more macho than football in many ways. So quick, short bursts in Eclipse would make communication flow more freely. The result should be pretty fast convergence on solutions because of the tight team work involved. Developing becomes more of a team sport than isolated outposts throwing Morse code at each other. IM has filled this role to a large extent so I stopped pursuing the idea. But I still think the integration with Eclipse would be powerful. It would help everyone on a project build a mental model of the software as it was being constructed. comment[] 3:48:20 PM       digg   reddit You Can't Twitter at Relativistic Speeds Twitter is entraining the technorati on an unbreakable hedonic treadmill. The treadmill gorges itself on an infinite supply info mediated dopamine hits. Addiction, divorce, 12 steps, and the grief cycle are sure to follow . But what really should concern twitterites is their global stream-o-conscious will shatter once we travel in space at near light speed. Let's say you're accelerating towards Vulcan in your new Mercedes X Series Space Coup and you type in your latest bon thought: I really need to upgrade my materializer. The pate was runny. Your thoughts will stream out at a constant speed of 186,000 miles an hour and nobody will hear you! And you will not hear them! You will ache. It will 1 millisecond without a info mediated dopamine hit. Then another. And then another. Until you go entire days without sharing the barely conscious thoughts of the twitter-sphere. Then you are in hair pulling, drano drinking withdrawl. Oh what a glorious future it will be! I do see a market in relativistic hermitages however. In time no place on earth with be safe from ads or phones or other information radiators. The only safe place to hide will be in a space capsule near the speed of light. Only then will you be alone with the strange sensation of your own thoughts. comment[] 12:45:47 PM       digg   reddit

Monday, March 12, 2007

What if Cars Were Rented Like We Hire Programmers? Imagine if you will that car rental agencies rented cars like programmers are hired at most software companies... Agency : So sorry you had to wait in the reception area for an hour. Nobody knew you were coming to today. I finally found 8 people to interview before we can give you a car. If we like you you may have to come in for another round of interviews tomorrow because our manager isn't in today. I didn't have a chance to read your application, so I'll just start with a question. What car do you drive today? Applicant : I drive a 2002 Subaru. Agency : That's a shame. We don't have a Subaru to rent you. Applicant : That's OK. Any car will do. Agency : No, we can only take on clients who know how to drive the cars we stock. We find it's safer that way. There are so many little differences between cars, we just don't want to take a chance. Applicant : I have a drivers license. I know how to drive. I've been driving all kinds of cars for 15 years, I am sure I can adapt. Agency : We appreciate your position, but we can only take exact matches. Otherwise, how could we ever know if you could drive one of our cars? Applicant : Oookay. I've driven a Taurus before. You probably rent those, don't you? Agency : Indeed we do. What year did you drive? Applicant : It was 2005...but I don't see how that ma... Agency : Oh sorry, we use the 2006 model. We can't possibly let you drive a later model. Applicant : But, but they aren't that different. Surely if I can drive a 2005 I can drive a 2006. Agency : Sorry, sir. Our requirements clearly spell out that you must be able to drive a 2006 model. Applicant : I've driven a 2006 Escort. Do you rent those? Agency : Ah, excellent, you are in luck. We have one in stock. Applicant : Great. Can I rent it? Agency : No, no, no. We have to go through our interviews now. I'll go try and find the first person. Interviewer#1 : Sorry I was late, I was in a meeting I couldn't get out of. I like to ask technical questions to get a feel for your competency as a driver. What color has the middle wire feeding into the distributer cap? Applicant : What? What does that have to do with driving? Interviewer#1 : If you have experience as you say driving an Escort then you would certainly know the color of that wire. Applicant : I know how to drive. Why don't you ask me questions about driving? Interviewer#1 : I assure you I am. Are you this way with everyone you rent a car from? Nevertheless, I'll ask another question. What is the total weight of an Escort just after it has been washed, but before it has been dried? Applicant : Hand dried or blow dried? Interviewer#1 : It doesn't matter. Applicant : I know. Interviewer#1 : Well then. Thank you very much. We are done. I'll find the next person. Interviewer#2 : Sorry I am late. They never told me I had an interview today. I see on your application that you've driven a lot of different cars and you have a lot of experience driving. It's a shame you only drive that 2006 Escort, that's what we use here. So, how would you fit a SUV through the eye of a needle? Applicant : What? What does that have to do with driving? I know how to drive! Please ask me some #$*&! questions about driving! Interviewer#2 : Sorry, I have a meeting to go to. Let me get the next interviewer. Interviewer#3 : Do you have an exact itinerary of where you will drive and park? Applicant : Not really. I just thought I would drive around and explore. I know I plan on going to the tech museum downtown. Interviewer#3 : I believe that's on first street. That's good. It's on our approved list of streets. Have you ever driven first street before? Applicant : Hm, let me think, no, don't think so. But I am sure I can find it. One city street is pretty much like any other, so it shouldn't be a problem. Applicant : Oh I am sorry, our policy is you can only rent you a car if you've driven on an approved street that you've driven on before. We just can't take a chance that you won't be able to drive on new and different streets. Applicant : I don't believe this. I know how to drive, navigate, diagnose and fix minor problems, ask for help, find out anything I need to find out, and learn anything I need to learn. I know everything I need to know to rent this car because I've done it successfully a hundred times before! Interviewer#3 : How excellent for you. But it's policy. We need the exact experience to be sure. No exceptions. You may be very skilled, but you don't have the specific skills we require...that will be all. Agency : Sorry, but interviewers #4 - #8 were called to an emergency off site with upper management to reformulate policies on policy formation. Applicant : Bows head forward, looks at the water spot on the desk, and sighs. Agency : We might or might not let you know in a couple of weeks if we'll rent you a car. Applicant : But I need a car now! Agency : Very well. It was close, not everyone wanted to rent you a car, but we will rent you a 2006 Escort. How much did you pay for your last rental car? Applicant : I don't see how that matters. What are you charging? Agency : We like to know what you paid before so you get a fair rate. Applicant : I paid market rates. Agency : Sorry, we must know how much... Applicant : Gets up and walks out of the interview room in total frustration, wondering how anyone ever rented a car at this agency. comment[] 9:58:46 PM       digg   reddit

Monday, February 12, 2007

The Input-Output Political Philosophy Be conservative in what you do and be liberal in what you accept. comment[] 9:36:01 AM       digg   reddit

Monday, February 05, 2007

If You See the Buddha in Your Scrum, Say Hi Around 2,500 years ago the Buddha may have created one of the first and longest lasting intentional communities, that when looked at in a certain light, looks an awful lot like a modern agile organization. In Karen Armstrong's excellent book, Buddha, we learn soon after the Buddha became enlightened a rich merchant gave him some land and built a few huts so the Buddha and his followers could make a permanent camp. This organization was called a Sangha and was the forerunner of the Buddhist monasteries we see today. Until this time the Buddha and his followers were always on the road and took shelter wherever they could when the monsoon season rained in. Few would travel on the monsoon muddied roads so the Sangha became a place where Buddha and his followers could wait out the monsoons and continue their studies. Soon Sanghas were being created everywhere as appreciative students donated land to the Buddha's cause. Whenever you get people together you need rules of governance. How should people interact? Who should be in charge? What are the duties and obligations of someone who has joined? What are the Buddha's answers to these questions? I was surprised. The Buddha advocated decentralized administration and individual authority and responsibility. Since Buddhism is based on taking personal responsibility for finding ones own enlightenment, this organization makes sense, but for some reason I thought he would be more of a waterfall guy than an agile guy. When one particularly nasty crisis involving different factions hit the Sangha, the Buddha refused to take sides. He thought the solution must come from combatants themselves for it was the egotism of the parties involved that made it impossible for each side to see each others point of view. Based on the idea hatred is never appeased by more hatred, he said treat both sides with respect, but that they had to work it out for themselves. Both sides could be defused with friendship and sympathy. And eventually they did patch things up. The Sangha had no central authority and there was no real organization. All the members of the Order were equal because the Buddha refused to be a ruler that controlled everything. The Buddha was never much concerned about having a central leader because he taught every person was responsible for themselves. The Buddha thought coercion was against the spirit of the Order and that if one wanted to live in a way different than the Order then they are perfectly free to leave. Monks must make up their own minds and not be forced to follow anyone else's directives. People who left and came back were to be accepted with open arms. What members of the Order did share was the same lifestyle and the same teachings. Every six years the different groups would come together to recite a common confession of faith, called the "bond." It's purpose was to bind the different groups together. What they pledged was: Refraining from all that is harmful. Attaining what is skillful. And purifying one's own mind; This is what the Buddha's teach. Forbearance and patience are the highest of all austerities; And the Buddhas declare that Nirvana is the supreme value. Nobody who hurts another has truly "Gone Forth" from the home life. Nobody who injures others is a true monk. No faultfinding, nor harming, restraint, Knowing the rule regarding food, the single bed and chair; Application in the higher perception derived from meditation -- This is what the Awakened Ones teach. OK, it's not exactly the Agile Manifesto, but there's a shared spirit. This pledge was important because it's all that united the different Orders together. What did the Buddha think would ensure the survival of the Order: Be mindful, spiritually alert, energetic and faithful to the meditative disciplines that alone can bring you to enlightenment. Avoid such unskillful pursuits as gossiping, lazing around, and socializing. Have no unprincipled friends and avoid falling under such people's spell. Do not stop half way in your quest and be satisfied with the mediocre. Become self reliant so you need not rely on any authority. I am not trying to say there's a one-to-one correspondence between the Buddha's organizational style and Agile, but there are interesting parallels. I am not saying the training for finding enlightenment is the same as producing software, but there are interesting parallels. And this organization has survived almost every other for over 2,500 years and is still going strong. That's not a bad model. comment[] 2:58:25 PM       digg   reddit

Saturday, February 03, 2007

Smackdown #2: Scrolling Crushes Paging After 2000 Years of Dominance Scrolling is now enjoying a historical renaissance over 2000 years in the making. Once upon a time all books were lovingly drawn on papyrus scrolls. Jewish Rabbis would have read the Old Testament from a scroll. Early Christians, perhaps as way to differentiate themselves from Jews, preferred a different book form, the codex. The codex is the same book style we use today: two sided pages held together with a binding. As Christianity rose to power the codex rose with it and scrolls fell out of popular use. Fast forward 2000 years into the future and scrolls are once again becoming the presentation form of choice. Why? Because web tech makes scrolling better than paging. But that wasn't always the case. Early web design continued the codex form. If you read most of the advice on how to design early web sites (circa 1994) the codex form was still king. Web pages were supposed to be cut up into little chunks and readers slogged through the text stream one slow click at a time. Small pages were faster to load, scrolling was new to most people, and scrolling in web pages was clumsy. So it was thought most readers would not scroll. Pages were the better design. All that has now changed. ClickTale, a web site usability service, has found people are scrolling and that web designers are now designing pages to feature scrolling. The User Inter