Griddle Noise

New Blogging Pastures

2008-09-03T07:27:00.004-06:00

With a fair amount of success, I've pulled in the entries from the old Griddle Noise into a Squarespace account. I've long been dissatisfied with Blogger... It was good enough for a free service, I guess, but I just couldn't take it any more. It was hard to use Markdown, and it was hard to use its "wysiwyg" editor on Safari. Results were unpredictable. So I'm giving Squarespace a try.

It's really quite a nice service, and it will definitely do for now. The big issue is to either:

Make time to write
Write smaller

So head over to eucci.squarespace.com/griddle-noise. Gods and time willing, I'll get around to knocking out my tech thoughts again on topics such as:

Python
Zope 3 Component Architecture
Buildout and the realization of the component dream (almost)
Git
Object publishing
etc...

Remembering Pausch

2008-07-25T19:26:00.002-06:00

There are three things that I remember about IPC 8 (International Python Conference 8), besides it being the only Python conference which I've had the luxury of attending.

Riding from Fredericksburg to somewhere outside of DC in a major Virginia snowstorm, only the first or second such snowstorm I had seen since moving to the area in late 1997.
Meeting Tres Seaver.
The final keynote, masterfully given by Randy Pausch.

Pausch, who passed away earlier today (July 25, 2008), was charismatic, smart, and opinionated, and his keynote was funny and engaging. I must admit that I forgot Randy's name, but I'll also gladly admit that many of the points he made have remained with me and I still think of them often. There are even a couple of anecdotes which I repeat to others. Not many people can give such memorable speeches and presentations. Thanks, Randy. I'm glad to have met and heard him in person once, and wish I had been able to do so again to thank him for some great insights and inspirations.

OmniFocus Revisited

2008-07-16T12:35:00.000-06:00

I've been using OmniFocus for a while now on my Mac, and made occassional use of it's predecessor, Kinkless GTD. As an implementation of the "Getting Things Done" method, or just as a personal "what am I doing?" manager, OmniFocus 1.0 was pretty good. I, personally, liked it better than the alternatives I tried, but some of that may be due to familiarity with the OmniOutliner heritage. And OmniOutliner remains one of the greatest Mac OS X applications ever written. I believe it's one of the greatest outliners ever written. Sure, it doesn't have Mind Mapping support and it can't run a slideshow (although its data can be exported to OmniGraffle or Keynote). OmniOutliner is a joy because it's fast, beautiful, ridiculously easy to use, while still having a fair amount of power and flexibility. Kinkless GTD was actually a set of AppleScripts that made OmniOutliner act like a specialized GTD application (which OmniFocus now does natively).

But when it came to OmniFocus, I only used it at work, and even there I only used it intermittently. But with the 1.1 "Sneaky-Peek" version of the desktop client (still under development), OmniFocus supports synchronization via MobileMe or any WebDAV server. This not only enables me to share this information between the office, home, and the rarely used laptop, but it enables sharing with the awesome new OmniFocus for iPhone.

Second only to the iTunes Remote, OmniFocus for iPhone is the coolest mobile application I've seen so far on this young platform. OmniFocus for iPhone (or iPod Touch) can synchronize over Wi-Fi, Edge, or 3G. That alone makes it very useful. Cooler still is that it's location aware. "GTD" has a strong concept of contexts. A project may require picking up supplies at an office supply store, assembling at home, and mailing out items at the post office. With OmniFocus, you can add location information to any context. This location information may be based on where you currently stand (using Location Services), on a manually entered address, on an address from a Contact record, or on a business search. Then when looking at the "By Location" screen in OmniFocus, available actions get grouped by their location in relation to where you currently stand. "Grocery Store: 1 Mile; Post Office: 3 Miles..." Very Cool.

This is the first electronic implementation of GTD that actually appears usable. Even if you don't follow GTD religiously (I certainly don't), the projects/contexts combination is an effective way of organizing actions. The location-aware contexts in OmniFocus for iPhone help answer the question "what can I do based on where I am?" When used well, it should make it harder to forget those little items that you needed to do or pick up for some small project.

iPhone 2.0 and the iTunes Remote

2008-07-12T00:27:00.003-06:00

Like many others, I was bit hard by the server failure that crippled the big iPhone 2.0 update process this morning. It robbed me of said morning, and I opted to take the day off work. It's been a bit of a tough week, and last night I did my first show in months, and had company over until past 3AM afterwards. Since things are about to get crazy at work, I decided that I needed a day to catch back up on sleep while waiting for my phone to work again.

A few hours later, the phone worked.

One of the most surprising and enjoyable elements is the iTunes Remote. Full and comprehensive access to my fairly large iTunes library on the iMac: all playlists, etc, with the ability to control volume, jump around in songs, see artwork - just like the 'iPod' iPhone application! Sometimes, particularly in the morning, I might turn on Front Row and carry the nice little Apple Remote around with me to have some control over songs while getting dressed in the upstairs loft, but using the remote from up there often required reaching far over the edges and trying to point the remote in the direction of the iMac, just to be able to skip forward a track or two.

But now, I can control it all from the iPhone, without needing a line of sight! In fact, this might be what pushes me to pick up an Airport Express or two (one for speakers in said upstairs loft, maybe one to bring music out on the patio or the bathroom). Full access and control of 29 and a half days of available music playable out of real speakers and controlled by an untethered iPod size device - beautiful.

Looking to a Snow Leopard Winter.. er... Summer.

2008-06-10T14:34:00.001-06:00

I’m a bit excited about Mac OS X “Snow Leopard”. Few user-visible changes, with a focus on fine-tuning and giving developers better access to capabilities of modern hardware. It appears that Apple’s experience in making a lightweight Mac OS X “Core OS” for the iPhone will also drive this release.

One of my favorite operating system releases was OS/2 “Warp” (OS/2 3.0). OS/2 2.0 was a fascinating creature - completely divorced from Microsoft, OS/2 2.0 delivered an aggressively object-oriented runtime built on SOM (a desktop implementation of some of CORBA 1.x, I believe). It was radically different from Window 3.x. It’s hardware requirements were a bit high for the times, but it was a solid OS.

What impressed me about OS/2 3.0 “Warp” was that it’s system requirements were in some cases significantly LESS than OS/2 2.0, while performing better. I don’t know of any majoro user-visible adjustments (this was before operating system releases became the giant dog’n’pony shows that have been expected since both Windows 95 and Mac OS X).

I think that even though desktop and laptop hardware continue to get better, the rapid growth rates seen between 1995 and 2005 are slowing down. Now the pressure is on connectivity, portability, and storage storage storage for all of those mp3s and movies and photos. I think both Windows XP and Vista, along with Mac OS X 10.4 and particularly 10.5 have been a bit cavalier about their usage / expectation of resource availability without doing a good job of cleaning up afterwards. Removing a ‘TemporaryFiles’ folder used by Apple’s “Soundtrack Pro” program gave me back 25 GB of disk space. 25GB! I expect that when doing lossless audio work, I’m going to leave a lot of turds behind. But not that many. That’s an accumulation over only a few months. Now some of that may have been due to crashes brought on by the instability in Mac OS X 10.5.2’s audio subsystem (particularly in relation to some USB audio devices). But still - 25 gig! Over the course of just a couple of months!

I think that Apple is at a good place to do this. Good housekeeping is required - otherwise you end up with situations like Mac OS ‘Classic’ or even Windows Vista, where there is so much old baggage, bad hacks, outdated mentalities, etc, all in play; it makes it difficult to move the platform forward. Some companies and developers have always been mindful of this, electing to keep their products lean and fast, always (see Ableton Live - hands down, the most impressive audio application out there). Other companies don’t support that philosophy for whatever reason - backwards compatibility, rush to market, a combination of the two, etc.

This far into the Mac OS X life cycle, there’s not many new dog’n’pony features to add. The API’s have stabalized, the developer tools offer more than they ever have (Interface Builder 3 is a terrific update), the Finder and Spotlight are actually fast and usable; applications and utilities from both inside and outside of Apple are going to really shine on Mac OS X 10.5 with all that it offers to developers. A new age of PDA’s are upon us, whether it’s a device like an iPhone, an ultra-mobile Asus Eee-PC style portable, or even the Macbook Air: secondary and tertiary devices are really taking off.

I think that an underlying aspect part of the ‘Snow Leopard’ plan is to allow such devices, made by Apple (naturally), to proliferate. When it was announced that the iPhone was built on Mac OS X, I was surprised - Mac OS X has been a pretty wasteful OS - or at least, one that would consume more resources than realized (often for caching, interestingly enough). A standard install is full of crap that may be useful, but often takes up space. How many gigabytes of printer drivers now? To take the fine tuning and resource management ideas from the iPhone variation of OS X into the main system is what I think will allow for Apple to finally make the Eee PC style portable that everyone wanted the Macbook Air to be.

I’m putting my money on some kind of small device, priced around $600-$800, coming out at or around the same time as Snow Leopard. Combined with Mobile Me and Snow Leopard Server’s increasingly Exchange-like feature set (but better priced and more understandable for small organizations), the ubiquitous-data-access capability is there.

Today’s full-featured laptops (MacBooks, Inspirons, whatever) are their own entities; my aging iBook gets used rarely as I just don’t have as much data or software set up on it, and it’s sometimes too big of a pain to keep in sync.

The XO and Eee-PCs (or whatever they’re called) are also separate from the rest of one’s life; useful as a fun or educational toy, or as a geek’s favorite gadget to see what they can get running on such a little device. Most of the other developments I’ve seen in this area have centered around “how cheap and how small can we make a laptop/portable that will run (Linux/Windows XP)”. But outside of education, if this is the only focus being given, then these companies are going to be making nothing more than the next round of casual gadgets that get tossed or buried after a few months - especially if a key factor of what made Palm devices so popular (for a while) is completely neglected.

The Macbook Air is deliberately designed as a complementary computer, using the master’s optical drive even. While sexy, I think the Macbook Air misses the mark on a few items. But I think it’s an indication of things to come - laptops deliberately designed to complement your main machine. Smaller devices, from the Palm to the iPhone, have done this. And they’ll also be designed to work with your (or your company’s) data, which the Blackberry has done (and the iPhone will do when its new ‘enterprise’ support rolls out). Getting this onto other devices, without being constrained to an enterprisey system like Notes or Outlook, is where things really appear to be headed. It’s certainly something that I’d like to have. And the more I look at Snow Leopard, the more I believe that Apple is sneaking ahead of the crowd into delivering this into the hands of consumers. They’re skating to where the puck is going to be.

Granted, Windows “Live Mesh” looks to be heading in the same direction. But after Vista, Microsoft needs to reign in the Windows kernel and distribution. Windows Server 2008 and some of what has been leaked (or speculated) about “Windows 7” seem to indicate that Microsoft is aware of this. And how could they not be? But I think that even with their vast resources, Microsoft has a long ways to go to catch up - even though it appears that they’ve been playing in this area (tablet computing, ultra-mobile pc’s) for a while. A deep cleansing of the Windows core is desparately needed. And then a deep re-implementation of the UI may be needed.

Apple had a terrific luxury (and great idea) with the iPhone. While sharing the same kernel and many same APIs as the desktop (and server) Mac OS X, it has an entirely new UI that is dedicated to its intended use. Windows CE, on the other hand, tried to bring the Windows 95 look and feel to tiny devices and now I’m really not so sure it was a good idea. It allowed Microsoft to punt on some usability and design issues by falling back on the way things work on the desktop. I still see this, even in some of the newest and fanciest “iPhone killers”: some of these have a very fancy launcher app; some even have a very fancy phone and contact app that spins around in 3D and responds to gestures. But then, suddenly, you’re in the tiny-font, tiny-scrollbar, pixelated, stylus-driven world of the interior. It’s like going into a grand building like The Plaza (back when it was a hotel, at least), and finding the inside full of grey linoleum floors, flickering flourescent lights, and cinderblock walls reminiscent of an old hospital or elementary school. Quite the let-down (a lot of courthouses are like this, actually).

I also think Apple was smart to NOT have an SDK at the launch of the iPhone. I bet they would have liked one, but I think the iPhone had to launch when it did, and perhaps not-quite-everything was ready yet. If one looks back at the classic Macintosh and Palm devices and operating systems, you see systems that pulled of very clever hacks to fit within the price and size constraints of the time. The Lisa was much more than a $10,000 Macintosh - it had many features from power management to an OpenDoc style multi-tasking document based UI. But to offer those features, it was priced well out of reach. The Macintosh squeezed as much as it could down into a 128K Ram machine, and the compromises they had to make in order for that to work would end up haunting the company until its near-death. The Palm, too, took the ideas of the Newton and other tablet devices and stripped them down into a size and price point approachable by the masses. And like Apple, the design decisions that were made to make that work have crippled the Palm OS so much that even Palm sells half of its devices with Windows CE (or whatever CE is called these days). Those compromises are bad enough to deal with on your own - but when having to support third party developers and then provide some degree of backwards compatibility, it can just kill you.

By taking the time to put the SDK into beta, to polish up the OS and its APIs, I think Apple will avoid a repeat of that story. Instead of having to support every little exposed compromise that may have been made to get the iPhones out the door last June, Apple could tidy them up. By using a beta period for the SDK and next major release of the software, Apple can respond to feedback and make changes and adjustments before they become permanent.

The Hafler Trio, Dislocation

2008-04-29T00:48:00.001-06:00

I just received the Korm Plastics re-release of The Hafler Trio’s Dislocation. I’ve been waiting for this one more than any of the others thus far. The Korm Plastics re-issue series is a magnificent undertaking wherein a large chunk of Hafler Trio material that has long been unavailable or available only in “wrong” versions is being re-issued in definitive form. The packaging for each has been stunning: each release in a booklet sized “wallet” form, with full booklets, posters, post-cards, and more. The releases are all wrapped in a vellum (or vellum-like) material with additional writing. The packaging alone is of the highest grade - far higher than what these releases saw in their original forms (even ones that were also in wallet/booklet form).

I first became interested in The Hafler Trio when the Hafler Trio / Nurse With Wound collaboration was issued on CD, which I picked up in … 1995? 96? I was a big Nurse With Wound fan at the time and, thus, gobbled this one up. I was blown away. While I recognized many Nurse With Wound sounds, the overall production and flow was of a different nature. Similar, yes, in it all being (perhaps) some form of Musique:Concrete. It just sounded a bit more … polished? Subtle? In any case, it remains a personal favorite.

At this time, I was 20-21 years old, poor, bla bla bla. Most of these CDs that I liked were all import and out of my price range (when they could even be found). But one day around this time, I was in a used cassette shop, and came across the original Hafler Trio Dislocation cassette. For $4! It was a Staaltape release, in a bag (my second one - a couple of years earlier I picked up what turned out to be a fairly rare Legendary Pink Dots cassette in the same style of packaging, mostly because I didn’t yet have a CD player). Not knowing much of The Hafler Trio at the time, I knew that this was something I should get, as it was unlikely that I would easily come across such a find for years to come.

I liked the tape, but I admit that it took a while to grow on me. It was very quiet and subtle, with a lot of field recordings. It was something that I enjoyed, but I found that I had to give it more attention than similar recordings from other artists. I loved having it on tape, though. It heightened the “found sound” feeling. And I loved what was inside the bag (besides the unlabeled cassette). Inside, there was heavy paper with fragments of maps on one side and various clippings on the other. It was far more interesting than some of the cheap shock imagery (or just cheap packaging) used, again, by somewhat similar artists that interested me at that time.

However, as fascinating as it all was, that would be it for me and The Hafler Trio until 3-4 years later when, after moving to the east coast, I’d find a copy of A bag of cats at Printed Matter’s old Wooster Street location in NYC. After that find, I’d lap up whatever I could.

So Dislocation has been held in a special place, and I’d been watching that Korm re-issue series list tick along, waiting for this one to show up. Now it’s here. And it’s much more impressive than imagined. First, the booklet is a new heavy-stock foldout variation of the same maps found within the cassette version. When folded up… I can’t even describe it, really. It’s just beautiful and very different from the booklets that have accompanied the rest of the re-issue series.

But what’s really been impressive has been the audio. This time, I have no prior CD to compare; only tape. And I love tape. But now, more than any other re-issue done thus far, I can really see how well done the audio restoration has been. Remarkably well done. This album deserved (and required) it. As much as I loved the “found sound” fuzzy feeling I got from the tape, there are a lot of nuances here that I may now appreciate. Goddamn well done.

What a DVCS gives me

2008-04-07T23:28:00.004-06:00

I've seen some posts floating around about what a distributed version control system might give you. For me, these are the key elements:

Committing changes is separate from sharing. While the phrase "you can edit on a plane!" gets thrown around quite frequently, I think this is the far more important aspect of that vision. As a developer, you don't always know how a particular path of development might impact the code base. With a purely centralized system, you have to think first about where a path may be taking you as it could affect everybody else. Time and time again, I've seen developers work without any revision control safety net for days or weeks at a time because they don't want to "break the build", and they don't have time to look up the policies for branch naming, merging, etc. Not that such a thing should take a long time, but when under pressure, it's the last thing one wants to deal with. And the untracked changes keep getting bigger, and bigger. And when I say "I've seen developers..." here, I include myself.

With a DVCS, I can commit immediately. I took heavy advantage of this on a recent project where the set of work for which I was responsible was in no state to be set up on other systems. It required new configuration and possibly new tool installations and I didn't have time to help everyone else install and update their sandboxes. They didn't need my code anyways. Instead, I was able to pull in and rebase my work on updates from my co-workers while my personal branch was in development. When my code was mature, it was easy to merge into a more centralized branch. Very easy. In fact, it was just a fast-forward (in Git parlance), since I was rebasing my changes on top of those by my colleagues.

Again, this is possible with a purely centralized system, but it would have required me to realize the significance of my changes and their impact on others. My sandbox was in a "guts on the table" kind of state. Just about every commit I made was stable, but sharing them would have made it harder on other team members to do their work due to the changes made in tool and configuration dependencies.

In essence, a DVCS inverts the control back to individuals. As a developer, I can commit my changes whenever I want. With a purely centralized system, I have to think more about what I'm about to commit, since it immediately impacts all other developers.

A DVCS encourages experimentation. Being able to commit my changes whenever I want and being able to make local branches so easily makes it easier for me to start playing around with new ideas. Whether that's doing a big refactoring or restructuring of code, experimenting with a new feature of third party library, or working on updating the code to run well on the latest release of Foo, I know that I can experiment and SAVE those experiments in small chunks, without impacting anybody else. I can choose when and how I want to share my results. I can choose to throw my experiment away and not have to be reminded by its grizzly branch name for years to come.

We have an internal toolkit that bridges SQLAlchemy and Zope 3 and provides various other useful features and integrations. Late last year, I started looking into updating the code to work with SQLAlchemy 0.4 and to also clean up some ancient hacks. We were still using CVS at the time. I can't remember when I made the branch, but I knew at some point that I was heading down a potentially long path and that a branch would be required. Other priorities were coming up and I'd have to leave this work aside for a while.

Well, there's 2-3 days of feverish work, all held hostage within a single check-in. I've been wanting to pull some features out of that branch and into the current mainline branch, but since it's all in one big check-in, I can't do that easily. This is the second time I've done that (the other time was on a feature branch that lead to a dead end, but along the way I had some good ideas that I'd love to be able to extract now).

If I'd been using Git, I would have been making more commits, more frequently, and in much smaller sizes. Using Git, I would then be able to quite easily cherry pick individual commits out of that branch and apply them to the mainline.

Finally, a DVCS makes it easy to vet changes through the system. We don't have to give new employees the keys to the kingdom, particularly when their skill set is focused on a specific area. Instead, the code can go through review channels. They make commits in their local repository, and tell someone (like me) that they've made some changes. Using Git (or any other tool - but Git's named remotes makes this hella easy), I can pull in changes from their repository, verify that they're good, and push them to the canonical repository. I can merge them into other branches, if required.

Imagine this in a team situation: team members can share their repositories with each other as needed, giving each other chances to do code reviews and fixes before sharing those changes with the larger group or division; all without requiring permission to touch the central repository. Suddenly whole new workflows open up, based on the "networks of trust" inherent in all of us: a team leader collects commits from their team, and shares those changes with other team leaders. Those team leaders pull together changes from all of their teams (while sharing said changes across team lines) and push those on to a QA / Testing division. The QA / Testing division then puts their seal of approval on things by being the ones who control pushing to the "canonical" repository from which builds are based.

There's just so much more that can be done with a DVCS, and we're in an age now where there are very usable and useful tools for this job. A DVCS restores individual responsibility, encourages experimentation, enables adaptive workflows, and I believe it fits more naturally into how we humans organize our interactions. Whether this is in a rigidly defined corporate structure or a loosely connected set of worldwide open source contributors, the peer to peer nature combined with getting the whole repository enables people to step up and do bold things without having to go through channels to get any coveted "write access".

Sunday Great Openings

2008-04-06T10:49:00.000-06:00

In my last post, I revisited an earlier write-up of a personal experience involving The Hafler Trio’s so-called Trilogy in Three Parts. I’ve been on a bit of a binge lately, trying to grab up a lot of Hafler Trio releases that I let slip by over the past few years. Right now, I’ve got two thirds of the Exactly series of long-form dual disc releases revolving around the voice of Jónsi Birgisson (Sigur Ros), with one of those releases being in the form of the deluxe packaging from Simply Superior. I’ve been starting to pick up vinyl releases as well and will soon have all three evidences according to Wolf Sheep Cabbage, among others. I even managed to find a well priced vinyl copy of what has become one of my all time favorite albums: A Thirsty Fish.

But on this here Sunday morning, I find myself ill at ease. Shoulders are tense, bones are cracking, mind is dazed yet burning. To try to find a spot of calm, I first went through what I have so far of the Exactly set, but it wasn’t helping today. So I returned, again, to this initial trilogy. I haven’t listened to it much since my grand encounter four years ago. That was just a little too powerful. But up until that point, I got a fair amount of comfort out of the series. It’s still a remarkably powerful. The trick is not to get too lost in it when not wanting great and terrible visions. All I need right now is a warm, sustained, and calming presence. These can do that job rather well - enough to get me through the morning and into a much needed hot shower followed by a cold walk with the dog and some lukewarm coffee.

A Trilogy Revisited

2008-03-24T07:57:00.004-06:00

I was killing late night minutes by reading through pouring empty into the void - a collection of reviews of Hafler Trio material - when I came across a review of the entire trilogy in three parts. The writing was more personal than many of the other reviews, which were written for larger audiences (magazines, record shops, etc). And then I got to the final paragraph:

Deep into No More Twain, Of One Flesh: 11 Unequivocal Obsecrations, I was deep into trance and ripped apart by my own twain — two entities guiding and dividing my life right now — and had to fight my way out of such deep and frightening consciousness and back into the body. Scratches and cold medal were put to use to reinvigorate the breath and to bring myself down out of the super present to the merely present.

"Wow," I thought, "this guy had an experience similar to mine!"

And then I clicked on the source link - and it was, in fact, my own. April 6, 2004. What a wonderful, horrible spring.

And what the fuck has happened to my brain when I look back at many of the things produced / released at that time and wonder how the hell I could do that (and find time to write about it in fairly eloquent terms). Now I have a hard time making a single, simple, goddamn web page for a simple four track online album. How anyone's ever going to hear Like the city, we're bound to last is beyond me at this point.

Quit blaming Nader

2008-02-24T12:47:00.002-07:00

A friend of mine says, “Ralph Nader is an idiot”, just because Nader is running for president again.

I fucking HATE this line of thinking. Nothing pissed me off more than when watching the documentary on Nader recently and seeing how everyone fucking turned on him in 2004 after championing him in 2000. At least Nader has principals and stands by them. Nearly every ‘liberal’ voice that I had respected proved to me that they had no fucking principals.

Michael Moore in 2000 at a Nader rally: “If you’re voting for the lesser of two evils, you’re still voting for evil!” Four years later, Moore was on the anti-Nader bandwagon.

If we believe in democracy and having choices and voices, then anyone should be able to run for president. 2000 was not Nader’s fault. Gore, and the democrats in general, should have run a better campaign. Just like when teams complained about the Patriots running up the score on them: it’s not the Patriots job to take Tom Brady off the field and stop scoring. It’s YOUR defense’s job to get on the field and stop them (the Giants did it; Philly and Baltimore nearly did it). The democrats should have run better campaigns in 2000 and 2004. They could have stopped Nader from running in 2004 if they had picked some of his causes (just some simple ones) and championed them. But the democrats didn’t.

Honestly - whenever I hear this vitriol against Nader, a man who has championed consumer rights far more than anyone currently running or serving this country, it sounds like people wanting a fucking dictatorship. It’s like the movie ‘moon over paramour’ (or something like that) where there are two posters for the same dictator - red and blue. One guy asks the other who he’s going to vote for, “red or blue”. The other guy says, “it doesn’t matter, it’s a free dictatorship.”

I have the right to vote for whomever I want. And on principal, I will cast my vote for whomever I feel best represents what I want out of a leader. I’m not going to vote for someone just because they where an elephant or donkey lapel pin. I don’t care if my one single vote hands the election over to another eight years of Bush tyranny.

The two party system sucks big time fucking hairy nutsacks. I think South Park represented it best with the “Giant Douche versus Turd Sandwich” vote for school mascot (one of the greatest, and saddest, South Park episodes).

What sucks even more than the two party system is the electoral college being based on representative seats plus senate seats. This causes a vote in Wyoming to count four times as much as a vote in california. Now that’s not fucking fair at all.

The problem isn’t Nader. The whole system is flawed and corrupt and I feel like the anti-Nader rhetoric comes from people who would like to see it stay that way as long as they think they can rig the system to swing to their guy who may be the lesser of two evils.

It’s still evil.

And honestly - seeing the response of so many people against Nader in that film made me absolutely sick with the democratic party and the people who claim to champion its causes. They do not believe in freedom or standing for principals. They’re the selfish, self-absorbed, narcissistic pricks. “Oh, look at me, I write for the nation.” Bit whoop. Nader actually fucking fought hard for things like seatbelts in cars. Big auto wouldn’t do it. They resisted like hell. We all take this for granted now. But it took someone with actual balls to stand up and fight for something so seemingly trivial (now) but which has been a big deal when it comes to saving lives. What has Eric Alterman done for us lately?

The general democratic response to Nader shows that most of these people have no spine, no guts, and they don’t deserve any glory. They certainly don’t deserve my support.

Blaming Nader is a sign of weakness, and a sign that the general democratic party and pundits can't own up to their own fucking mistakes. If they can't do that, they can't lead.

Distributed VCS's are the Great Enablers (or: don't fear the repo)

2007-12-02T22:58:00.000-07:00

The more I play with the new breed of VCS tools, the more I appreciate them. The older generations (CVS, SVN) look increasingly archaic, supporting a computing and development model that seems unsustainable. Yet most of us lived with those tools, or something similar, for most of our development-focused lives.

When I speak of the new breed, the two standouts (to me) are Git and Mercurial. There are some other interesting ones, particularly Darcs, but Git and Mercurial seem to have the most steam and seem fairly grounded and stable. Between those two, I still find myself preferring Git. I’ve had some nasty webs to untangle and Git has provided me with the best resources to untangle them.

Those webs are actually all related to CVS and some messed up trunks and branches. Some of the code lives on in CVS, but thanks to Git, sorting out the mess and/or bringing in a huge amount of new work (done outside of version control because no one likes branching in CVS and is afraid of ‘breaking the build’) was far less traumatic than usual.

One of those messes could have been avoided had we been using Git as a company (which is planned). One of the great things these tools provide is the ability to easily do speculative development. Branching and merging is so easy. And most of those branches are private. One big problem we have with CVS is what to name a branch: how to make the name unique, informative, and communicative to others. And then we have to tag its beginnings, its breaking off points, its merge points, etc, just in case something goes wrong (or even right, in the case of multiple merges). All of those tags end up in the big cloud: long, stuffy, confusing names that outlive their usefulness. It’s one thing to deal with all of this for an important branch that everyone agrees is important. It’s another to go through all of this just for a couple of days or weeks of personal work. So no one does it. And big chunks of work are just done dangerously - nothing checked in for days at a time. And what if that big chunk of work turned out to be a failed experiment? Maybe there are a couple of good ideas in that work, and it might be worth referring to later, so maybe now one makes a branch and does a single gigantic check-in, just so that there’s a record somewhere. But now, one can’t easily untangle a couple of good ideas from the majority of failed-experiment code. “Oh!” they’ll say in the future, “I had that problem solved! It’s just all tangled up in the soft-link-experimental-branch in one big check in and I didn’t have the time to sort it out!”

I speak from personal experience on that last one. I’m still kicking myself over that scenario. The whole problem turned out to be bigger than expected, and now there’s just a big blob of crap, sitting in the CVS repository somewhere.

With a distributed VCS, I could have branched the moment that it looked like the problem was getting to be bigger than expected. Then I could keep committing in small chunks to my personal branch until I realized the experiment failed. With smaller check-ins, navigating the history to cherry-pick the couple of good usable ideas out would have been much easier, even if everything else was dicarded. I wouldn’t have to worry about ‘breaking the build’ or worry about a good name for my branch since everyone else would end up seeing it. I could manage it all myself.

This is the speculative development benefit that alone makes these tools great. It’s so easy to branch, MERGE, rebase, etc. And it can all be done without impacting anyone else.

One thing that I often hear when I start advocating distributed VCS’s is “well, I like having a central repository that I can always get to” or “is always backed up” or “is the known master copy.” There’s nothing inherant in distributed VCS’s that prevents you from having that. You can totally have a model similar to SVN/CVS in regards to a central repository with a mixture of read-only and read/write access. But unlike CVS (or SVN), what you publish out of that repository is basically the same thing that you have in a local clone. No repository is more special than any other, but that policy makes it so. You can say “all of our company’s main code is on server X under path /pub/scm/…”.

And unlike CVS (or SVN), really wild development can be done totally away from that central collection. A small team can share repositories amongst themselves, and then one person can push the changes in to the central place. Or the team may publish their repository at a new location for someone else to review and integrate. Since they all stem from the same source, comparisons and merges should all still work, even though the repositories are separate.

Imagine this in a company that has hired a new developer. Perhaps during their first three months (a typical probationary period), they do not get write access to the core repositories. With a distributed VCS, they can clone the project(s) on which they’re assigned, do their work, and then publish their results by telling their supervisor “hey, look at my changes, you can read them here …” where here may be an HTTP or just a file system path. Their supervisor can then conduct code reviews on the new guys work and make suggestions or push in changes of his own. When the new developers code is approved, the supervisor or some other higher developer is repsonsible for doing the merge. It’s all still tracked, all under version control, but the source is protected from any new-guy mistakes, and the new-guy doesn’t have to feel pressure about committing changes to a large code-base which he doesn’t yet fully grasp.

But perhaps the most killer feature of these tools is how easy it is to put anything under revision management. I sometimes have scripts that I start writing to do a small job, typically some kind of data transformation. Sometimes those scripts get changed a lot over the course of some small project, which is typically OK: they’re only going to be used once, right?

This past week, I found myself having to track down one such set of scripts again because some files had gotten overridden with new files based on WAY old formats of the data. Basically I needed to find my old transformations and run them again. Fortunately, I still had the scripts. But they didn’t work 100%, and as I looked at the code I remembered one small difference that 5% of the old old files had. Well, I didn’t remember the difference, I just remembered that they had a minor difference and I had adjusted the script appropriately to finish up that final small set of files. But now, I didn’t have the script that worked against the other 95%. When I did the work initially, it was done in such a time that I was probably using my editors UNDO/REDO buffer to move between differences if needed.

Now if I had just gone in to the directory with the scripts and done a git init; git add .; git commit sequence, I would probably have the minor differences right there. But I didn’t know such tools were available at the time. So now I had to rewrite things. This time, I put the scripts and data files under git’s control so that I had easy reference to the before and after stages of the data files, just in case this scenario ever happened again.

I didn’t have to think of a good place to put these things in our CVS repo. I just made the repository for myself and worried about where to put it for future access later. With CVS/SVN, you have to think about this up front. And when it’s just a personal little project or a personal couple of scripts, it hardly seems worth it, even if you may want some kind of history.

Actually, that is the killer feature! By making everything local, you can just do it: make a repository, make a branch, make a radical change, take a chance! If it’s worth sharing, you can think about how to do that when the time is right. With the forced-central/always-on repository structure of CVS and SVN, you have to think about those things ahead of time: where to import this code, what should I name this branch so it doesn’t interfere with others, how can I save this very experimental work safely so I can come back to it later without impacting others, is this work big enough to merit the headaches of maintaining a branch, can I commit this change and not break the build….?

As such, those systems punish speculation. I notice this behavior in myself and in my colleages: it’s preferred to just work for two weeks on something critical with no backup solution, no ability to share, no ability to backtrack, etc, than it is do deal with CVS. I once lost three days worth of work due to working like this - and it was on a project that no one else was working on or depending on! I was just doing a lot of work simultaneously and never felt comfortable committing it to CVS. And then one day, I accidentally wiped out a parent directory and lost everything.

Now, in a distributed VCS, I could have been committing and committing and could have lost everything anyways since the local repository is contained there: but I could have made my own “central” repository on my development machine or on the network to which I could push from time to time. I would have lost a lot less.

There are so many good reasons to try one of these new tools out. But I think the most important one comes down to this: just get it out of your head. Just commit the changes. Just start a local repository. Don’t create undue stress and open loops in your head about what, where, or when to import or commit something. Don’t start making copies of ‘index.html’ as ‘index1.html’, ‘index2.html’, index1-older.html’ ‘old/index.html’, ‘older/index.html’ and hope that you’ll remember their relationships to each other in the future. Just do your work, commit the changes, get that stress out of your head. Share the changes when you’re ready.

It’s a much better way of working, even if it’s only for yourself.

Broken Bulb

2007-11-20T14:44:00.000-07:00

With apologies to Johnny Cash: "Flash, I hate every inch of you."

Falling for Git

2007-11-07T21:26:00.000-07:00

You know what? Git must have come a long way in the last year. I keep reading that Git is hard to learn, has rough documentation, etc. But it’s really been quite nice in comparison to many things.

It’s especially nice once you quickly learn that the HTML man pages for Git follow a simple pattern (as I guess many online man page collections must). Just change the end of the URL from git-cvsimport.html to git-push.html or git-pull.html to look up documentation.

That I’ve been able to play around with Git quite successfully and easily just makes my frustration with some Python tools (like easy_install and zc.buildout, particularly its recipes) even more …. frustrating.

And, I’ve totally fallen in love with Git. Yes, I know there are alternatives written in Python that are quite comparable. But Git’s actually been easier to install and figure out (particularly the CVS interaction that I must currently suffer). And people who know me know that I’m no “Kernel monkey”. I’m really impressed with Git’s implementation and general behavior. Very impressed with the implementation.

By the way: if you’re having to work two ways with a CVS repository, this post has been absolutely invaluable. This collection of Git articles has been invaluable in getting some good defaults established, and some tips for building on Mac OS X (with a nice tip to download and untar the man pages directly instead of trying to build them with the asciidoc tool and its terrible dependency on troublesome XML libraries. Goddamn, how I hate XML).

AppleScripting Across the Universe

2007-11-01T01:52:00.000-06:00

After a long day at work, I wrote a long message in Basecamp about what I had accomplished, how to access it, etc. But I forgot to submit the message! Crud. I wanted to send it out before morning and didn’t want to go into the office. I couldn’t get any screen sharing connection to go between the machines. I just had a handful of SSH leaps.

AppleScript to the rescue!

This is probably the most AppleScript that I’ve ever written. Fortunately, Safari supports the command do JavaScript ... in tab. After some floundering around with a similar setup on my local machine, I finally figured out AppleScript’s interesting reference notation and was able to ferret out the window and tab containing the unsent message, add some text to the message’s textarea element, submit the form, and return the extended value.

tell application "Safari"
    set message_tab to current tab of window named "Web site > New message"

    set extended to ".... Fun fact - i wrote this before i left the office and forgot to submit it. as a result, i now know how to submit forms like this via AppleScript."
    set post_body_value to "$('post_body').value"
    set extend_value to post_body_value & " += '" & extended & "';"

    do JavaScript extend_value in message_tab
    set body_value to do JavaScript post_body_value in message_tab

    do JavaScript "document.forms[0].submit();" in message_tab

    return body_value
end tell

I pasted the above code into VIM and ran it with the command line osascript command. Worked like a champ.

And because sleep is for the weak, I decided to track down how to do the equivalent in Python. Mac OS X 10.5 provides a “Scripting Bridge” for Python and Ruby (and potentially others), which causes many frameworks and other objects to be dynamically exposed. Without the need (for better or worse) of yet-another-virtual-machine. Anyways, I cobbled the following together:

from Foundation import *
from ScriptingBridge import *
safari = SBApplication.applicationWithBundleIdentifier_('com.apple.Safari')

def find_window_named(name):
    for win in safari.windows():
        if win.name() == name:
            return win

window = find_window_named("Web site > New message")
message_tab = window.currentTab()

print safari.doJavaScript_in_("$('post_body').value", message_tab)
safari.doJavaScript_in_("document.forms[0].submit()", message_tab)

There may be a better way to do the find_window_named method, but I didn’t have the time to track it down. As it was, I was able to do do the above by playing around with everybody’s favorite Python tool, dir(), which verified my suspicion that many of the commands exposed to AppleScript were also available via the Scripting Bridge. This is evidenced by the currentTab() method of a Safari window, analogous to the current tab of window ... AppleScript. And I imagine most of these are just Objective C methods. And since AppleScript editor’s Dictionary browser told me about the do Javascript [v] in tab [t] command, it stood to reason that it would exist on the Safari object. It was there when I did pprint(dir(safari)), and I knew that I’d need to pass in a Tab object.

In any case, it’s awesome that Apple has embraced Python and Ruby and has tied them in to the Cocoa runtime. Historical note: the first Python - Objective C bindings that I know of where commissioned by a NeXT Developer who wanted to use Python and Bobo (zope.publisher) to do web work with NeXT’s Enterprise Objects Framework, without the weight and cost of WebObjects. I think that means that Python was bridged into the Objective C runtime and NeXTStep frameworks before Jython ever got going. I believe that work was done by the developer who later released Objective Everything which bridged into Perl and TCL as well as Python.

Of course, traditional MacPython from the classic Mac OS was also natively tied in to the AppleScript of that era; AppleScript has always supported other dialects (FrontierScript was a common one).

But it’s nice now to see support coming out of both Apple and Microsoft (and Sun too, I guess) for these languages. The above scripting of Safari was surprisingly easy. As was an earlier experiment to fish around my calendar store for incomplete To-Do items. Quite nice.

But what’s especially nice is that I was able to SSH into my office Mac and tell Safari to submit that form that I had neglected earlier.

Java Crybabies

2007-10-29T22:29:00.000-06:00

So Mac OS X 10.5 (Leopard) doesn’t ship with Java 6. And now Java people are all sad and mad and yelling at Apple for dropping the ball on this.

Why should Apple go out of their way to provide Java 6? After the aborted Java - Objective C bridge experiments, what else is there to do? “Native” Java applications have still never come close to feeling like a native or near native piece of the operating system. Why should Apple keep throwing engineering efforts at this system?

And whatever happened to OpenStep for Solaris by the way?

Honestly, Apple has a dynamic object runtime environment heavily tied to a C based language already. It’s done much of what Java and the .Net framework are now doing, and has been doing that since the latter half of the eighties. It’s interesting to look back to the mid nineties and at the criticisms of NeXTStep/OpenStep. “Why are you using this Objective C thing? Why not C++?”

Because what NeXT understood that others didn’t was that the runtime is what’s important. NeXTStep came closest to providing a Smalltalk-style runtime of dynamic collaborating objects without being the alien and self contained environment that Smalltalk can often be. Initially tied in to a Unix operating system, it later went multi-platform (at least to an extent - I think FoundationKit, Enterprise Objects Framework, WebObjects, and PDO (Portable Distributed Objects) ran on HP-UX and Solaris, while all of those plus AppKit ran on NT, along with D’OLE (distributing COM/OLE while Microsoft was still struggling to provide DCOM)).

In those same mid nineties, there were other attempts to provide some of the power of the NeXTStep / OpenStep platform. IBM perhaps came closes with their CORBA (Corba 1.x) based SOM, which was also to be at the heart of OpenDoc. SOM was the heart of the fascinating OS/2 2.0 and Warp. It also was used in the classic Mac OS. Of course, it was there in OpenDoc’s brief life. But beyond that, SOM was used to provide contextual menu plug-ins, interestingly enough. But it was still fairly heavy, as CORBA could be. Too much wringing and wrangling to help non-dynamic languages function in a semi-dynamic world.

And then there was Microsoft’s Cairo. Never shipped. Some of its technologies found their way into NT… But the big features? One of the big features was to be an “object oriented file system”. This resurfaced as WinFS for Vista. Twelve-plus years later, and it’s still not done.

And of course, there was Taligent. Initially, Taligent was going to be an all-new Operating System, aggressively object oriented, etc. Apple and IBM together, to make a NeXTStep for the rest of us, perhaps? Except instead of a dynamic language, they decided to go for C++. But they apparently had to pump in a lot of work to overhaul the C++ runtime and try to provide some of the dynamic loading options (of Smalltalk, NeXTStep/Objective C, etc). It was a lot of time wasted, I’m sure. And they eventually had to pull back from the all-new operating system plan. That was probably wise, considering the environment of the time. Apple never was able to complete Copland and Gershwin, and Microsoft never got Cairo finished; BeOS never found a substantial market; and even early NeXT-era Apple wasn’t able to sell the idea of a NeXTStep based Mac OS until they provided the Carbon migration path for the classic Mac APIs.

So Taligent shifted to providing, like OpenStep, differing layers that would provide these object features on top of differing host operating systems. Still never happened. Which is a bit of a bummer - they had some paradigms that would have been interesting to see.

In 1994, Jon Udell wrote a short article titled “A Taligent Update,” subtitled Will systemwide object frameworks reinvent programming?

Well, while Taligent never delivered; and OpenStep faded into WebObjects (providing the OpenStep developer tools on NT, NeXTStep Mach, and Mac OS X Server 1.x, aka Rhapsody); this seems to have actually, finally, come to pass. Cocoa is a killer framework for Mac OS X, with many fans. There are bridges to many other languages (Ruby, Perl, Python, among others). It’s not quite the pervasive system-wide framework that it was in the NeXTStep days, but in Mac OS X Leopard Cocoa looks as though it’s reclaiming its position as king of the hill. (For a while, there were many Carbon APIs that were a bitch to use from Cocoa; or at least those used to the comparative ease of Cocoa programming).

And Microsoft’s .NET framework has delivered similar in the Windows market. Of course, it doesn’t have anything like Interface Builder; but it still seems to have a much better share-and-reuse model than anything that’s come before it in Windows programming. And it’s built on: dynamic object systems. Unlike Java (which I’ll get to in a minute), the .NET framework and core languages (C#) appear to be taking cues from Objective C and purely dynamic languages/systems like Smalltalk: dynamic class extension, for example, is a new feature in C# 3.0. It’s also been possible in Objective C. This can be a dangerous feature; but also quite useful and usable. But it’s nice to see this in languages and systems that try to combine C, which has the benefits of familiarity, with the power of dynamic object-filled worlds.

And it’s much better than the heavy and strained world of COM and CORBA.

So anyways - NeXT, and now Apple, has been ahead of this game for quite some time. Granted, if it weren’t for Apple, NeXT would be another blip like Taligent. Except with a shipping product. But still - they survived. And their system wide dynamic object framework idea seems to have been vindicated.

So what of Java?

Java is the bastard child here. I’ve never been comfortable with it. It’s not cross platform: Java IS the platform. And it’s awkward. Even in its best desktop guise - Eclipse - it’s still a foreign environment on Mac OS X. Even Firefox is starting to feel more natural (and Firefox 3 looks to be trying even harder in this area). Why would we want it? Sun has never seemed to care that much about Java on Macs, except to try to showcase their “see, multi-platform!” message. But Windows has always gotten the lions share of the attention, even though Microsoft long ago stopped caring about Java.

On a side note, we’re seeing the same thing happen now with Flash. Ugh. It’s a memory hog on Mac OS X. Makes poor use of resources. If Adobe and Sun don’t seem to care enough about providing a truly killer Mac experience, is it any wonder that they’re being kicked off of the island? Apple’s got the dynamic object system language and frameworks (Objective C, Cocoa); and it’s got an increasingly impressive web environment (WebKit, with Canvas support); it has bridges into and out of AppleScript, Python, Ruby, and Perl, all included with Leopard - bridges too!

One just has to look at how huge the “Java in a Nutshell” books have become to know that Java is no small undertaking. And again, I think that Apple probably has stopped caring. They’ve tried to be good Java citizens - from the Java - Objective C bridge to the all-Java implementation of WebObjects; they’ve tried to be make the Cocoa framework appealing to Java developers and probably tried to make Java appealing to Cocoa developers. But it must not have ever happened. It’s all deprecated now.

And I don’t know what’s going on, really. I just see Apple as having other priorities. It’s not like they’re (purely) a not-invented-here company. They have, after all, built in Sun’s DTrace technology. And Apple builds on and gives back to Open Source, with projects like launchd, the new calendar server, bridge support, WebKit, etc.

But Java…? Apple has no real stake in it any more. The last Java application I ran was Eclipse, months ago, just trying to see what life in a fancy IDE would be like. It was disappointing. Desktop Java just doesn’t figure into a Mac user’s life all that much. I’d rather see Apple focus on improving their primary object language (which inspired Java), focus on improving their APIs and offering more features for programmers (all done handily in Mac OS X 10.5 - from a programmer’s standpoint, it’s extremely impressive) and providing a smooth, fast, and natural platform experience (again done handily in Mac OS X 10.5 - see the still-unequaled Interface Builder 3.0; see CoreAnimation; see a sea of new UI object offerings from Apple).

Why would they spend their time fighting uphill to support a platform whose chief aim is to be an anti-platform? At best, desktop Java on the Mac could mean “runs a windows-like-application-almost-as-good-as-windows, maybe.” Well, we now have Boot Camp and Parallels/VMWare for that. Apple wants to provide a killer alternative platform, and they’ve learned that the best way to do that is to be in control. When Mac browsers were suffering - IE on Mac OS X was slow and strange (compared to near excellent behavior in OS 9), Camino and Firefox were big and slow and non-native (Camino did a decent job, but it had different widget implementations for browser skin and in-page rendering), OmniWeb looked beautiful but its support for new HTML and Javascript was far behind, etc - Apple took control of their destiny by building Safari. And they built in on a toolkit that would let them plug in the right widgets and behavior for a native experience. Hence, all Safari users have long enjoyed having spell checking support in their TextAreas. We get that from Cocoa. As of Mac OS X 10.5, we can turn on grammar checking as well. And now, WebKit is leading much of the HTML 5 charge, recently announcing preliminary support for client side database storage, and were among the first (if not THE first) to put forward tags like canvas which may make SVG usable and make ultimately take care of many uses of Flash (remember when Flash was purely an animation tool?). Apple has all of this in their control. They don’t have to put up with lackluster players / viewers from Adobe.

And I think it’s become pretty clear that Apple’s preferred solution at this time for rich cross-platform(ish) UI and Code is - the web. HTML, CSS, Javascript. It powers Dashboard; it’s been a big selling point of the iPhone (although it is admittedly laughable that Apple said “Web 2.0 apps are your iPhone API!”, but it’s still impressive; people have built some very impressive apps with that very system).

Why does AJAX / DHTML succeed where Applets have failed? And even Flash does poorly? It’s not just because it’s “everywhere”, without a need to install and deal with a JRE. It’s because AJAX is part of the web page. It’s not a self contained rectangle that can do really cool things - within the realm of that rectangle. It’s because AJAX leverages the browser’s toolkit, so that a text field in IE behaves like a text field in IE; and a text field on a Mac behaves like a text field on a Mac. Java performance has never been that great on the Mac. Which must mean that no one cared enough to really try to make it shine, or that the technology is really heavy and inferior. I remember groaning every time I used to check the snow report page at one of the ski resorts because of the extra amount of time taken to load up Java and all that accompanies it just to render some scrolling-headlines; scrolling headlines whose text didn’t match (or even anti-alias with) the surrounding text. It’s absolutely useless and pointless. Granted - I have seen some impressive Java applets in the science arenas, and IBM had some cool chess ones in their old Deep Blue v Kasparov (?) challenge. But again, those have been few and far between.

The two desktop Java apps that I’ve used heavily at points in the past, besides my experiments with Eclipse, were with a UML tool, and then LimeWire. Both of these used on Macs. The UML tool was tolerable, but barely. Limewire was terribly slow - click and wait instead of click and point. Desktop Java is dead to me. I don’t know why you’d want to write in it. Cocoa is just an all out balls out better environment, especially for UI programming as (again) Interface Builder remains peerless. And .NET, especially with the Mono implementation, is a far more interesting playground that seems keen on taking in new dynamic, declarative, and functional features, (LINQ, F#, etc), while Java just feels like a big stack of alphabet soup and static typing and not much else. (Although I do understand that Java 6 has started to break this mold).

And as far as languages go: please. Python, Perl, and Ruby alone offer better cross platform capabilities than Java: especially on the server side where Java is supposed to shine. I didn’t have to wait for Apple’s blessing to use Python 2.5 on my desktop Mac. I didn’t have to wait for Apple’s blessing to draw native widgets with it either. I didn’t have to wait for Apple’s blessing to use Python 1.x to control other applications across the scripting bridges of Mac OS 7, 8, and 9.

Now that Java is Open Source (it is, isn’t it?), maybe the Java community can look into what it would take to provide a good Java experience on Mac OS X. I think that is its only hope. It has to become leaner. The dynamic “scripting” language crowd have all been able to find ways to take advantage of different platforms. Why is it on Apple’s head to provide Mac Java? Going back to what I asked earlier - is Sun going to try to get OpenStep going again on Solaris? Is Microsoft going to provide Java 6 for Windows? Is Apple going to provide .NET 3.5 for the Mac?

A response I’ve seen in the Java community is that Apple is arrogant for not shipping Java 6 with Leopard, and for withdrawing any development downloads and many topics related to Java on the Mac. I think that it’s arrogant of the Java community to think that they matter enough for Apple to continue to sink engineering resources into the platform. They’ve sunk a lot in over the years, and there’s never been a huge payoff. I see no reason for them to continue. They have far better alternatives.

IBM or Sun or anyone else out in the Java / Open Source community should take it upon themselves to provide a good platform if they really care. They’re obviously doing it for Linux and Windows.

Sorry, this is a long and rambling post and now it’s quite late at night when I swore I would be going to bed early. But seriously - Java has not mattered to me as a developer or Mac user for years. It’s a dead weight for Apple. Support for Mac OS “Classic” got the boot in Leopard, and even Carbon is looking like its days are numbered. If those two can be cut off, what the hell chance would Java have?

Catching Up

2007-10-12T14:50:00.000-06:00

These periods between posts keep getting longer, don’t they?

I’ve got nothing earth-shattering to talk about. Work’s been very busy, and we continue to be served well by Zope 3. I’m still royally confused by things like setuptools and eggs, mostly in regards to how they work in a Zope 3 world when you’ve already got long entrenched ways of doing software. I could not get a good answer from anyone I asked (in fact, I often got wildly competing opinions). So I’m sticking with our internal make-rake-like-ish toolkit which is primarily helpful for automating checkouts from internal and external repositories. I did have some success with zc.buildout, but I don’t yet foresee a time when I can use it to deploy whole sites/applications. I can barely see a time when I can use it on anything but small projects that are relatively stand-alone. There’s just a big gap between The Way Things Have Been Done and The Way That It Seems That Maybe Things Should Be Done In The Future.

Of course, neither setuptools nor zc.buildout seem to have “proper” releases. zc.buildout is in an endless 1.0 beta (beta-30 at this point), and setuptools is at 0.6c7. Does that mean that it’s not even at release 0.6 quality yet? None of this instills confidence in this hurried developer.

The big problem is the legacy code, which is in CVS. Some of it is being extracted out into individual packages that have the proper ‘setup.py’, ‘buildout.cfg’, etc. Finally. But I have no idea how to apply it to the bigger picture, and I’ve found very little written words that target our situation.

The biggest downside of being so busy with customer related work is that it’s very difficult to keep up with discussions, conversations, plans, etc. And I’m sure that my frustrations with lack of documentation, seemingly unfinished releases, and so on, are really the fruit of other hurried developers. I admire them for at least releasing something. It’s more than I’ve done in a long time. It’s more than I see myself being able to do for quite some time.

Anyways, the revolving door of Javascript toolkits keeps turning. I’m now deeply enamored with jQuery. “Write less, do more”. I like it. I like that it doesn’t trample all over Javascript, and thus plays well with others (especially others that play well with others, like MochiKit). MochiKit is just so big… I think I might make a stab at writing, at least for internal use, a lightweight version that brings many of its best concepts out without overlapping jQuery’s functionality. MochiKit brings many wonderful Python-ic functions and tools to the Javascript table that make general development much easier.

I’m also deeply enamored with zc.resourcelibrary which is a Zope 3 add-on that makes it much easier to manage javascript and CSS resources and their relations to each other. Among other things, it helps save resources when they’re not needed. For example::

if len(rendered_boxes) <= 3:
    return self.just_render_the_damn_boxes(rendered_boxes)
else:
    zc.resourcelibrary.need('fancy.scrolling.library')
    return self.render_the_advanced_widget(rendered_boxes)

I’ve also adjusted my coding style, returning to the underscore_separated_words style instead of the camelCasedWords style, at least for functions, attributes, and methods. This is closer in style to PEP 8 (the main style guide for Python code). The Zope style guide differs on this point, using camelCased instead. And PEP 8 does say that it’s OK, if not downright preferred, to stay true to the style around you.

But one thing I learned from looking through Rails code was that the underscore_style was easier to read, since the underscore acts like a space. And I’ve become a big fan of writing code that communicates intent; that reads like a story (somewhat). Extract Method is your friend. I’ve grown very distrustful of excessive nesting, or of having very long bodies inside of a ‘for’ or ‘if’ block.

That’s about it. Hell of an update, huh? Well, work’s really started to become work, and is quite enjoyable. I’ve got a good flow going and don’t feel I have as much need (nor place) to be an advocate or crank. As I’ve mentioned before, we’ve gotten incredible levels of code re-use by building our internal libraries and applications on top of Zope 3, and we’ve been able to grow them so much that they’re really the first level of framework. It was such a struggle to do this in Zope 2, but in Zope 3 it does fall (fairly) neatly into place. Nothing else in the Python web-framework-whatsit world comes close.

The only toolkit that’s even better? SQLAlchemy. It’s pretty much the only way I’ll interact with RDBMS systems in Python from this point out. And I don’t mean I’ll be writing every RDBMS interaction as an object-relational mapping. SQLAlchemy is great because it provides a good connection / pooling infrastructure; a good Pythonic query building infrastructure; and then a good ORM infrastructure that is capable of complex queries and mappings (as well as some pretty stone-simple ones).

Numbers

2007-08-16T07:18:00.000-06:00

I don’t know when I last used a spreadsheet for its actual spreadsheet capabilities, on a sheet I designed myself. I think it may go back to AppleWorks (on the Apple II)! Sure, I’ve used sheets like time cards and travel requests that others had made where I just had to fill in the holes. And I’ve received more than my fair share of spreadsheets used like an outliner / lightweight database / structured note list. But I don’t remember how long it’s been since I used a spreadsheet to figure out a budget, to track expenses, or any other dumb mundane thing like that. Until last night.

I was getting ready to pay my mid-month bills, and I was trying to figure out how much I could pay towards one of my credit cards and still have enough cash to cover the expenses remaining in the month. I also realized that I’ve been spending quite a bit at the iTunes Music Store and hadn’t been tracking any of it. I decided that this would be an excellent time to try out Apple’s new spreadsheet application, Numbers. I found a downloadable time trial of Apple’s iWork ‘08 suite, and immediately got to work.

Numbers is pretty damn cool. I don’t know if there are other spreadsheets that behave like this, but in modern times, it seems so obvious: instead of having the big set of cells in one large table, you work in small floating spreadsheets / tables. This is a big deal for so many reasons, with the most obvious being layout. Another great reason is that each table/spreadsheet can be more focused on its job. Already, Numbers felt a lot more intuitive than anything I had used in a long time.

When doing my simple rest-of-the-month budget, my main question was “how much can I pay on this card and still have enough cash on hand for the rest of the month?” Numbers made it easy with its slider option. For just this one cell, I was able to quickly configure it to give me a slider with a range of -700 to -500. When I got the rest of the budget entered, I could then play with the slider and watch its impact on the total-leftover cell. In previous months, I’ve generally done this calculation in my head, or compared where I stood the prior month after paying this particular bill. It was much nicer to whip up a simple spreadsheet where I could make this one particular number interactive and see the results immediately.

So I was able to get a couple of simple but nice looking spreadsheets together quickly that gave me actual data. I could easily play with this data, or just be embarrassed by it (I have spent quite a bit on the iTunes Music Store).

There are still a lot of old-style spreadsheet rules in play, at least in formulas and the like. That’s made a bit easier by being able to use header names (ie, =SUM(Total) or =MINA(Date Purchased)). I think it was Lotus’ Improv, which first appeared on NeXTStep, that worked this way. In fact, I think with Improv, it was the only way you could work: there were no A/B/C/D columns or rows. This was part of a cool feature of Improv wherein you could drag and drop header representations and regroup the data visually without impact on the calculations. I still think that was one of the most forward-thinking spreadsheet applications. But, it’s gone. I believe there’s some open source variation on the idea, possibly written just for GNUStep…?

Still, Numbers is pretty decent. I love the free-floating tables. It does make it much easier to compose complex spreadsheet pages out of multiple tables and data types. It’s pretty easy to refer to other tables as well. And it’s nice to have non-tabular data (text, graphics, etc) floating free from those numbers, making it easier to adjust layouts without impacting cells.

I’m impressed enough that I’m quite likely to buy iWork ‘08, just for Numbers alone. I have a small need for Pages and almost no need for Keynote, but I do find myself needing to get on top of my finances and similar data. Numbers is the first tool I’ve encountered that I think will let me handle my odd needs without requiring a degree or summer course in spreadsheets.

Yahoo TV Is Now Useless

2007-07-18T20:26:00.000-06:00

So, I used to use “My Excite” as my little personal portal. It had good TV listings, which is always important. But many years ago, as Excite was was burying its content under more and more ads, I switched to “My Yahoo!”. Which was even better. Great TV listings. So even now, when I don’t have cable or satellite, I found it valuable.

But then they changed. They upgraded to super fancy TV listings full of AJAX-y action.

But do you know what sucks?

THEY CAN’T GET TIMEZONES RIGHT!

It’s been quite a few months now, and they still can’t get my timezone right.

For seven or eight years, I had no problem with localized listings. Although for a few of those years I was in the blessed Eastern Time Zone. But even when I moved out west - no problem.

But now, apparently Yahoo! has invested all of their resources into flashy features but they can’t tell me when a tv show is on according to the time in my area.

Absolutely useless.

For a while, I’ve gotten by with using the TV listings on the “My Yahoo!” page. It didn’t have all of the fancy features, so somehow it managed to get the times right. But as of today, I’m told that I have to use the new fancy-ass “My Yahoo! Beta”.

First thing that happens when I visit the My Yahoo! beta?

“We don’t support Safari….”

Goddammit. Many of their other upgraded YUI based sites work fine in Safari. I can understand them saying this months ago, which was the last time I looked at the new beta. But to still be having an issue? While trying to be a shiny partner of the lovely iPhone? What’s up?

Anyways, I clicked the “live on the edge” button to see the new page. There were the TV listings. In the wrong time zone!

I submitted an email months ago about this and got the “yeah, we know, we’re working on this” response.

I know that time zones are a bitch to work with, but come on: this shit has worked for years. And now it’s been how long since the launch of the fancy new Yahoo TV section? Six months? Seven? Surely someone could have worked this out by now.

I’m so disappointed. Yahoo! was always one of the most reliable web sites. And I appreciate what they’ve given to the developer community with YUI! and other tools. But this little TV listing issue just takes the cake. It makes it absolutely useless, and now “My Yahoo!”, which has been my ‘home page’ for years, is all but useless as this became such a valued resource as their other sources stopped working.

Very frustrating.

And I’m still looking for a good TV listing site. But all I’ve come up with, so far, is pretty much bullshit. Wrong channels, inability to properly remember channels, too many ads, hard to access listings, slow display…. Augh.

Pissed.

Traits / Roles as Alternative to Abstract Base Classes

2007-05-08T14:29:00.000-06:00

While digging through the Python-3000 development list archives, trying to figure out the state of thought circling PEP’s 3119 and 3124, I came across this gem:

“Traits/roles instead of ABCs”, by Collin Winter.

With ABCs refererring to Abstract Base Classes (pep 3119).

Winter’s proposal is similar to my recent post, which is that this sort of “capability inference” should be dynamic, and not bound to the rigid nature of the class hierarchy. In my post on this subject, I showed a number of different implementations of a single interface (specification, role, whatever) - only one implementation followed the basic class-instance scenario. All others provided the exact same outward appearance, while internally they were implemented as module-level functions, class or static methods, or a dynamically composed single-use object (a brainless instance was made and had methods dynamically attached).

Winter’s roles/traits system, which refers to roles in Perl 6 and traits in Squeak, is along the same lines. I hope to hell it gains traction.

ABC may be easy as 123, but it can't beat zope.interface

2007-05-04T10:45:00.000-06:00

I guess the deadline may have come and gone for getting in PEPs for Python 3000. Guido’s already written up a PEP Parade.

Of particular interest to me has been the appearance of PEPs for Abstract Base Classes (PEP 3119) and the more exhaustive PEP 3124 which covers “Overloading, Generic Functions, Interfaces, and Adaptation.”

Both of these aim to provide ways of saying “this is file-ish”, “this is string-ish,” without requiring subclassing from a concrete “built-in” type/class. But I think they both fall short a little bit, while zope.interface (from the Zope 3 family) provides the best solution.

PEP 3119 (Abstract Base Classes) has a section covering comparisons to alternative techniques, and it specifically mentions “For now, I’ll leave it to proponents of Interfaces to explain why Interfaces are better.” So this is my ~~brief~~ attempt at explaining why.

A quote from PEP 3119 that I particularly like is “Like all other things in Python, these promises are in the nature of a gentlemen’s agreement…” The Interfaces as specified and used in Zope 3 and some other systems are the same way. They are not “bondange and discipline” Interfaces. They are not the ultra-rigid Eiffel contracts, nor are they the rigid and limited Interfaces as used by Java. They are basically a specification, and they can be used (as mentioned in PEP 3119) to provide additional metadata about a specification. There are some simple tools in zope.interface.verify to check an implementation against a specification, but those are often used in test suites; they’re not enforced hard by any system. The agreement might be “I need a seekable file”, which might mean it expects the methods/messages ‘read’, ‘seek’, and ‘tell’. If you only provide ‘read’ and ‘seek’, then it’s your fault for not living up to the agreement. That’s no different than the Python of today. What Interfaces and Abstract Base Classes aim to provide is a better clarification of what’s expected. Sometimes “file-like” in Python (today) means it just needs a ‘read’ method. Sometimes it means the full suite of file methods (read, readlines, seek, tell). Same thing with sequences: sometimes it just means “something iterable”. Other times it means “support append and extend and pop”.

Another side benefit of Interfaces as specification is that they provide a common language for, well, specifications. Many PEPs propose some sort of API, especially informational PEPs like WSGI (PEP 333) or API for Cryptographic Hash Functions (PEP 247). I’ll use PEP 247 as an example for my attempt at explaining why Zope 3’s Interfaces are Better.

A problem with Abstract Base Classes is this: they’re limited to classes. Even when PEP 3119 mentions Interfaces, it does so like this:

“Interfaces” in this context refers to a set of proposals for additional metadata elements attached to a class which are not part of the regular class hierarchy…

It then goes on to mention that such specifications (in some proposals and implementations) may be mutable; and then says that’s a problem since classes are shared state and one could mutate/violate intent. That’s a separate discussion that I’m not going to go into here.

What is important is this severely limited focus on classes. zope.interface works on objects as well, and not just normal ‘instances of a class’ object, but on classes themselves, and also modules.

There are two important verbs in zope.interface: implements and provides. provides is the most important one - it means that this Object, whatever that object may be, provides the specified interface directly.

implements is often used in class definitions. It means “instances of this class will provide the specified interface”. It can also be thought of in terms of Factories and/or Adaptation - “calling this object will give you something that provides the desired interface.”

“What does that matter?” you might ask. Well, there are all sorts of ways to compose objects in Python. A module is an object. It has members. A class is an object. An instance of a class is, of course, an object. Functions and methods are also objects in Python, but for the most part what we care about here are Modules, Classes, and Instances.

Because when it comes down to actual usage in code, it doesn’t particularly matter what an object is. In PEP 3124, the author (Phillip J Eby) shows the following interface:

class IStack(Interface):
    @abstract
    def push(self, ob)
        """Push 'ob' onto the stack"""

    @abstract
    def pop(self):
        """Pop a value and return it"""

Ignore the @abstract decorators, as they’re artifacts of the rest of his PEP and/or related to PEP 3119. What is important is the use of self.

“self” is an artifact of implementation that is invisible to use. Sure, you can write a Stack implementation like this. (Note: I’m going to use zope.interface terminology and style from here on out):

import zope.interface

class Stack(object):
    zope.interface.implements(IStack)

    def __init__(self):
        self._stack = []

    def push(self, ob):
        self._stack.append(ob)

    def pop(self):
        return self._stack.pop()

But when it’s being used, it’s used like this:

def do_something_with_a_stack(stack):
    stack.push(1)
    stack.push(2)
    # ...
    top = stack.pop()

stack_instance = Stack()
IStack.providedBy(stack_instance)
# True
IStack.providedBy(Stack)
# False

do_something_with_a_stack(stack_instance)
# works fine
do_something_with_a_stack(Stack)
# raises an exception because `Stack.push(1)` is passing `1` 
# to `self`.. unbound method, bla bla bla.

Notice that there is no ‘self’ reference visibly used when dealing with the IStack implementation. This is an extremely important detail. What are some other ways that we may provide the IStack interface.

One way is to do it with class methods and properties, effectively making a singleton. (This isn’t a good way to do it, and is just here as an example).

import zope.interface

class StackedClass(object):
    zope.interface.classProvides(IStack)

    _stack = []

    @classmethod
    def push(class_, ob):
        class_._stack.append(obj)

    @classmethod
    def pop(class_):
        return class_._stack.pop()

IStack.providedBy(StackedClass)
# True

do_something_with_a_stack(StackedClass)
# this time it works, because `StackedClass.push(1)` is a class method,
# and is passing `StackedClass` to the `class_` parameter, and `1` 
# to `ob`.

Another variation of the above is using Static Methods:

import zope.interface

class StaticStack(object):
    zope.interface.classProvides(IStack)

    _stack = []

    @staticmethod
    def push(ob):
        StaticStack._stack.append(ob)

    @staticmethod
    def pop():
        return StaticStack._stack.pop()

Again, StaticStack.push(1) and StaticStack.pop() work fine. Now lets try a third way - in a module! Let’s call this module mstack (file - mstack.py)

import zope.interface

zope.interface.moduleProvides(IStack)

_stack = []

def push(ob):
    _stack.push(ob)

def pop():
    return _stack.pop()

Then in other code:

import mstack

IStack.providedBy(mstack)
# True
mstack.push(1)
mstack.push(2)

print mstack.pop()
# 2

So whether we’re dealing with the instance in the first example (stack_instance), the classes in the second two examples (StackedClass and StaticStack), or the module in the last example (mstack), they’re all objects that live up to the IStack agreement. So having self in the Interface is pointless. self is a binding detail.

Jim Fulton, the main author of zope.interface, taught me this a long time ago. Because in Zope 2, you could also make an IStack implementation using a Folder and a pair of Python scripts. Well, those Python scripts (as used in Zope 2 “through-the-web” development) have at least 4 binding arguments. Instead of ‘self’, the initial arguments are context, container, script, traverse_subpath. Just like self is automatically taken care of by the class-instance binding machinery, the four Zope Python Script binding arguments are automatically taken care of by Zope 2’s internal machinery. You never pass those arguments in directly, you just use it like push(ob) and pop().

So there it is - many ways to provide this simple “Stack” Interface. And I believe that both [PEP 3119] and [PEP 3124] are short sighted by focusing on the class-instance relationship exclusively (or so it appears).

And since many objects, particularly instances, are mutable, one could compose an IStack implementation on the fly.

class Prototype(object):
    """ Can be anything... """

pstack = Prototype()
pstack._stack = []

def pstack_push(ob):
    pstack._stack.append(ob)

def pstack_pop():
    return pstack._stack.pop()
pstack.push = pstack_push
pstack.pop = pstack_pop

# Now we can say that this particular instance provides the IStack
# interface directly - has no impact on the `Prototype` class
zope.interface.directlyProvides(pstack, IStack)

pstack.push(1)
pstack.push(2)
print pstack.pop()
2

# We can remove support as well
del pstack.push
zope.interface.noLongerProvides(pstack, IStack)

Examples of dynamically constructed objects in the real world - a network services client, particularly one that’s in an overwraught distributed object system (CORBA, SOAP, and other things that make you cry in the night). Dynamic local ‘stub’ objects may be created at run time, but those could still be said to provide a certain interface.

So now let’s look at whether it matters that you’re dealing with a class or not:

@implementer(IStack)
def PStack():
    pstack = Prototype()
    pstack._stack = []

    def pstack_push(ob):
        pstack._stack.append(ob)

    def pstack_pop():
        return pstack._stack.pop()

    pstack.push = pstack_push
    pstack.pop = pstack_pop
    zope.interface.directlyProvides(pstack, IStack)

    return pstack

@implementer(IStack)
def StackFactory():
    # Returns a new `Stack` instance from the earlier example
    return Stack()

import mstack
import random

@implementer(IStack)
def RandomStatic():
    # chooses between the two class based versions and module
    return random.choice([StackedClass, StaticStack, mstack])

All three are factories that will return an object that provides an IStack implementation, which is exactly the same as the Stack class in the first example. That also claimed that it implements(IStack). When the class is instantiated / called, a new object is made that provides the IStack interface. In Python, another thing that doesn’t really matter is whether something is a class or function. All of the following lines of code yield a result that is the same to the consumer. The internal details of what is returned may vary, but the IStack interface works on all of them:

Stack()         # class
PStack()        # 'Prototype' dynamically constructed object
StackFactory()  # Wrapper around basic class
RandomStatic()  # Chooses one of the class/static method implementations.

And whether we’re looking at the class implementation, or any of the factory based implementations, the result should be the same:

IStack.implementedBy(Stack) # class
# True
IStack.providedBy(Stack)
# False
IStack.providedBy(Stack())
# True

IStack.implementedBy(PStack)    # Factory
# True
IStack.providedBy(PStack)
# False
IStack.providedBy(PStack())
# True

No matter which method of instantiation is used, they should all pass the verifyObject check, which checks to see whether all of the specified members are provided and that the method/function signatures match the specification

from zope.interface import verifyObject
verify_stack = partial(verifyObject, IStack)

all(verify_stack, [Stack(), PStack(), StackFactory(), RandomStatic()])
# True

Now the class-based options will fail on the implementedBy check, because it’s the Class that provides the implementation, not an instance like with Stack

IStack.implementedBy(StackedClass)
# False
IStack.providedBy(StackedClass)
# True
IStack.providedBy(StackedClass())
# False

“OK”, you might say, “but still, why does it matter? Why might we really care about whether these abstract specifications work only with classes? It seems smaller, simpler.”

The main advantage is that specification should (generally) make no assumptions about implementation. If the specification, aka “gentlemen’s agreement” is generally met, it shouldn’t matter whether it’s provided by a Class, an instance, a module, an extension module, or some dynamically constructed object. The specification language should be the same

Going back to PEP 247, the “cryptographic hash API”: there is a specification in that module about what the ‘module’ must provide, and for what the hash objects must provide. Consider also the WSGI spec, the DB-API specs, and all of the other formal and informal specs that are floating around just in the PEPs. Using zope.interface, those specifications can be spelled out in the same fashion. WSGI just cares about a particular function name signature. It can be provided by a single function in a simple module, or as a method from an object put together by a large system like the full Zope 3 application framework and server. It just wants a callable. This is a little bit ugly in zope.interface… but in reality, actually, I think it works. Here’s how it could be specified:

class IWSGIApplication(Interface):
    def __call__(environ, start_response):
        """ Document the function """
    # and/or use tagged values to set additional metadata

This just means that a WSGIApplication must be a callable object taking environ and start_response arguments. A callable object may be a function (taken from PEP 333):

def simple_app(environ, start_response):
    """Simplest possible application object"""
    status = '200 OK'
    response_headers = [('Content-type','text/plain')]
    start_response(status, response_headers)
    return ['Hello world!\n']

Or a class (the __init__ is what is callable here). Maybe the WSGI spec might also state that the result “should be iterable (support __iter__)” Maybe that’s loosely enforced, but the following example shows how the class can make separate declarations about what the class directly provides, and what its instances implement. Instead of using any decorators or magic-ish “class decorators” (the implements, classProvides calls above), we’ll make the declarations for both AppClass and simple_app in the same manner, which matches the style in PEP 3124.

class AppClass(object):
    def __init__(self, environ, start_response):
        self.environ = environ
        self.start = start_response

    def __iter__(self):
        status = '200 OK'
        response_headers = [('Content-type','text/plain')]
        self.start(status, response_headers)
        yield "Hello world!\n"

from zope.interface import directlyProvides, classImplements

# Both 'simple_app' and 'AppClass' are callable with the same arguments,
# so they both *provide* the IWSGIApplication interface

directlyProvides(simple_app, IWSGIApplication)
directlyProvides(AppClass, IWSGIApplication)

# And we can state that AppClass instances are iterable by supporting
# some phantom IIterable interface
classImplements(AppClass, IIterable)

What are the benefits of this, beyond just having a common way of spelling specifications? Instead of, or in addition to, abstract base classes, the core Python libraries can include all of these specs, even if they don’t provide any concrete implementation. Then I could have a unit test in my code that uses verifyClass or verifyObject to ensure I stay inline with the specification.

def test_verifySpec(self):
    verifyClass(ICryptoHash, MyHashClass)

Then, if the specification changes in a new version of Python or in a new version of someone elses library or framework, I can be notified.

Of if the specification undergoes a big change, a new spec could be written, such as IWSGI2Application. Then by process of adaptation (not covered in this post) or interface querying, a WSGI Server could respond appropriately to implementations of the earlier spec:

if IWSGI2Application.providedBy(app):
    # Yay! We don't have to do anything extra!
    # ... do wsgi 2 work
elif IWSGIApplication.providedBy(app):
    # We have to set up the old `start_response` object
    # ... do wsgi 1 work
else:
    raise UnsupportedOrUndeclaredImplementation(app)

Adaptation could provide a means of doing the above… (still, not going into the details.. trying not to!)

@implementer(IWSGI2Application)
@adapts(IWSGIApplication)
def wsgi1_to_wsgi2(app):
    return wsgi2wrapper(app)

# And then, replacing the `if, else` above:
wsgi_app = IWSGI2Application(app, None)
if app is None:
    raise UnsupportedOrUndeclaredImplementation(app)
# ... do wsgi2 work

When you have both specification and adaptation, then you can write your code against the spec. In the above example, the main code does IWSGI2Application(app, None) which means “for the object app, give me an object that provides IWSGI2Application, or None if there is no means of providing that interface.”

If app provides that interface directly, then app is returned directly. Otherwise an adaptation registry is found, and it’s queried for a callable object (an adapter) that will take ‘app’ as its argument and return an object that provides IWSGI2Application.

Another example: knowing that Python 3000 is going to change a lot of core specifications and implementations, such as the attributes for functions (func_code, func_defaults, etc). If an IPy2Function interface were made (and zope.interface or something like it was added to Python 2.x), then code that works with function object internals could program against their preferred spec by adding a line of code:

func = IPy2Function(func)
if my_sniffer(func.func_code):
    raise Unsafe(func)

On Python 2, you’d get the regular function straight through. In Python 3000 / 3.0, an adapter would translate __code__ into func_code, for example. I don’t expect this to happen in reality, but it’s an example of how migration paths could be made between two major software versions, allowing code to run in both.

By taking advantage of this system, my company has seen more re-use with Zope 3 than at any time in our company history. And because (most of) Zope 3 is programmed against specification, we’ve been able to plug in or completely make over the whole system by providing alternative implementations of core specs. This is very hard to do in native Zope 2 (the CMF, on which Plone is based, was probably the first Zope system that started these concepts, which Plone and others were able to take advantage of by providing new tools that matched the provided spec).

At the heart of it, again, is the gentlemen’s agreement, but brought out in full: it doesn’t matter who you are or where you came from (ie, it doesn’t matter what classes are in your family tree or if you are a simple module), as long as you get the job done. There’s a simple contract, and as long as the contract is fulfilled, then everybody is happy.

But if the gentlemen involved can only come from the class system, then there’s still a nasty aristocracy that excludes a large chunk of the populace, all of whom can potentially fulfill the contract. Let’s not cause an uprising, OK?

plispy

2007-04-30T11:32:00.000-06:00

Sometimes, it just happens...

>>> pprint(
...     sorted(
...         map(
...             linecount,
...             path('.').walkfiles('*.py')
... )))

Python's Make Rake and Bake, another and again

2007-04-24T23:33:00.000-06:00

Ian Bicking wrote a post recently titled “Python’s Makefile”. He advocates using / re-using distutils… er… setuptools. (I can’t keep them straight - they’ve both become absolute nightmares in my opinion). He then goes off about entry points, separate setup.cfg files, and other things that still go way over my head. The example he shows is convoluted, and I’m ultimately not entirely sure what he’s really advocating (besides the idea - which isn’t bad - of using the near-standard setup.py file/system instead of re-inventing).

But he mentions, earlier:

Because really people are talking about something more like rake — something where you can put together a bunch of code management tools. These aren’t commands provided by the code, these are commands used on the code.

We do have the infrastructure for this in Python, but no one is really using it. So I’m writing this to suggest people use it more: the setup.py file. So where in another environment someone does rake COMMAND, we can do python setup.py COMMAND.

For me, having an easy way to say bla bla COMMAND isn’t as important as having a good system for automating common tasks that I and/or my colleagues do frequently. As we started to depend on more and more code from internal and external repositories, due to our increased re-use when building on Zope 3, I really needed to automate checkouts and exports. Not everything was neatly packaged as an egg, or the released egg didn’t have a bugfix applied, and I still don’t understand how to make eggs work well with Zope 3 in a manner that I’m comfortable with.

I was initially excited about zc.buildout as a way to automate the monotonous but important tasks that revolve around setting up both deployment and development environments. But I didn’t like how zc.buildout specified its tasks/commands in INI format. It was relatively easy to write new ‘recipes’, so I wrote some recipes to do Subversion and CVS checkouts/exports.

But the INI format just pissed me off. It didn’t fit my needs, basically, wherein I needed more conditional control. More code control. And managing complex sets of parameters required making new top-level sections instead of nesting. Before long I was staring at a very long and very narrow file. And in the end, it was building Zope in a way that wouldn’t work for us. So I abandoned it.

I briefly looked at some tools that let you write these task files in “pure” Python. In this way, Scons appeared to be the closest thing in Python to Rake, which uses Ruby. But Scons seemed far more focused on general compilation issues (compiling C, Java, etc), but that’s never a problem that crosses my path.

I just wanted something like rake. What I liked about every Rakefile that I’ve seen is that it’s been quite readable. Rake makes common file / path commands readily available as Ruby methods, classes, and objects. Rake takes advantage of Ruby’s syntax, particularly blocks (and optional parenthesis) in a way that makes it not seem like, well, Ruby. It looks like something makefile-ish, something shell-scripting-ish, etc. That’s what I wanted; but, of course, in Python.

So I came up with a system. It’s not yet released to the world - far from finished, and there are many competing ideas out there that I don’t feel like competing with - but it’s already proven to be very useful internally. Generally, it’s been used to automate what I mentioned above: retrieving software from multiple repositories, both Subversion and CVS, and placing them in the proper directories. In particular, we try to stick with certain revisions for third party dependencies, and I got tired of trying to capture this information in READMEs and other files that we could refer to when installing certain configurations. It’s even been useful for downloading such software and applying internal patches::

patch = Command('patch')

@task('mysqldbda')
def mysqldbda():
    """ Installs mysqldbda from subversion and applies patch """
    svn = Subversion('svn://svn.zope.org/repos/main')
    svn.co('mysqldbda/tags/mysqldbda-1.0.0', target='mysqldbda')

    # patch mysqldbda
    log.info("patching mysqldbda")
    patchfile = path('fixes/mysqlda.1-5-07.patch')
    if patchfile.exists():
        print patch.read('-p1', '-i', patchfile)

@task('formencode')
def formencode():
    svn = Subversion('http://svn.colorstudy.com/FormEncode')
    svn.co('tags/0.6/formencode')

task('install', ['mysqldbda', 'formencode'])

It’s also been useful for tasks like getting MochiKit and generating all sorts of packed versions. A lot of what makes this possible is the path.py module, which provides a more object-oriented interface over os, os.path, and other Python file utilities.

ROCKFILEPATH = globals().get('ROCKFILEPATH', path('.'))
MOCHIKIT_LIB = ROCKFILEPATH/'libs'/'mochikit'
MOCHIKIT_DL = ROCKFILEPATH/'mochikit_dl'
MOCHIKIT_SRC = MOCHIKIT_DL/'MochiKit'
SCRATCH = MOCHIKIT_LIB/'_scratch.js'
mochikit = namespace('mochikit')

@mochikit.task('get')
def getmochikit():
    if MOCHIKIT_DL.exists() and bool(MOCHIKIT_DL.listdir()):
        return
    svn = Subversion('http://svn.mochikit.com/mochikit')
    svn.co('trunk', target=MOCHIKIT_DL)

@mochikit.task('clearmochilib')
def clearmochilib():
    for jscript in MOCHIKIT_LIB.files('*.js'):
        jscript.remove()

@mochikit.task('make-noexport')
def makenoexport():
    info = Subversion().info(MOCHIKIT_DL)
    src = NOEXPORT.safe_substitute(**info)
    file(MOCHIKIT_LIB/'NoExport.js','w').write(src)

@mochikit.task('build', ['get', 'clearmochilib', 'make-noexport'])
def mochi_install():
    for source in MOCHIKIT_SRC.files('*.js'):
        log.info('copy %s -> %s' % (source, MOCHIKIT_LIB))
        source.copy(MOCHIKIT_LIB)

# Javascript Packing tools (JSPack not shown - essentially it's a wrapper
# around combining and piping Javascript through Dojo's custom_rhino.jar
# to use its compression system)
def packmodules(sourcedir, modules, target):
    mods = [ (sourcedir/mod) for mod in modules ]
    log.info('Packing %s modules', path(target).name)
    JSPack(mods, target).run()

    if SCRATCH.exists():
        SCRATCH.remove()

def jsmin(sources, target):
    packmodules(MOCHIKIT_LIB, sources, MOCHIKIT_LIB/'min'/target)

@mochikit.task('minimize')
def mochiMinimize():
    """
    Generates packed versions of most individual MochiKit files, while
    combining a few core ones together.
    """
    mindir = MOCHIKIT_LIB/'min'
    for jscript in mindir.files('*.js'):
        jscript.remove()
    jsmin(['NoExport.js', 'Base.js', 'Iter.js', 'DOM.js'], 'base-iter-dom.js')
    jsmin(['Style.js', 'Signal.js'], 'style-signal.js')
    jsmin(['Async.js'], 'async.js')
    jsmin(['Color.js'], 'color.js')
    # ...

mochikit.task('install', ['build', 'minimize']).comment('INSTALL!')

I don’t think this falls under the jurisdiction of setup.py (distutils/setuptools). Nor would I want to specify these as zc.buildout recipes and have a separate configuration file to then name all of the files and directories. And, being Python, I don’t really have to deal with compilation steps so I don’t need wrappers around gcc and friends. I’m not (yet) specifying how to build large deployment scenarios. I just need to automate some development tasks, and I need to be able to write them easily. I want to write them in Python, but I want to ensure that they don’t accidentally get imported into normal projects (hence, the files above don’t have a .py extension). And as this is a specialized task, I’ll allow myself to get away with Python shortcuts that I would never touch in normal development, such as import *. In fact, it’s the import * that gives me a lot of the common commands/tools, such as the classes for interacting with Subversion and CVS, managing working directories, etc.

This really stemmed from reading this article by Martin Fowler about people wanting to replace ant with Rake with the advent of JRuby. In the post, Martin states:

The thing with build scripts is that you need both declarative and procedural qualities. The heart of a build file is defining tasks and the dependencies between them. This is the declarative part, and is where tools like ant and make excel. The trouble is that as builds get more complex these structures aren’t enough. You begin to need conditional logic; in particular you need the ability to define your own abstractions. (See my rake article for examples.)

Rake’s strength is that it gives you both of these. It provides a simple declarative syntax to define tasks and dependencies, but because this syntax is an internal DomainSpecificLanguage, you can seamlessly weave in the full power of Ruby.

At that point, I decided that this was the way to go: use Python decorators to wrap ‘task’ functions. The wrapper maintains dependency links, comments, and other things of interest to the internal system; and the wrapper allows the task name to be independent of the function name, allowing easier-to-type tasks for use from the file system. But the ‘task’ function is plain Python. Or, like some of the examples above show, task can be called without the @ symbol that makes it a decorator. Multiple callable actions can be added to a task, potentially allowing for more ‘declarative’ style:

mochikit.task('minimize').using_action(
  JSMinMap(
    {'style-signal.js': ['Style.js', 'Signal.js']},
    {'async.js': ['Async.js']},
  ))

Useful, I imagine, for very common patterns. Er. “Recipes”. In any case, it’s a very useful kind of tool. Beats setup.py, INI, or XML based automation language any day.

Cock Radio

2007-04-18T11:41:00.000-06:00

This whole Don Imus issue has confused the shit out of me. Talk radio is full of that kind of, um, talk. Anyways, it all feels like the first episode of the latest season of South Park. Sometimes, South Park can crank out a new episode in response to a very recent event, but that didn’t happen here. This episode aired weeks earlier.

As for how or why this Don Imus issue exploded in the way that it did - I just don’t understand (and now I feel like Stan Marsh at the end of that South Park episode). There are so many similar things said all the time by many radio “personalities.”

Media Matters has an excellent post up chronicling the many slurs of Glenn Beck, O’Reilly, and more: It’s not just Imus.

The response to the whole Imus situation just seems wrong: a cause celeb on which everyone can jump. The latest distraction. How the hell did it get so out of hand? Who did it really offend? Why this “nappy headed ho’s” statement? Why not “ghetto slut” (Boortz)? “Turbanned hoodlums” (Savage)?

Imus is probably far less offensive than many of these other radio people, and neither his firing nor all of this special attention is going to make anything better. Nor did it solve anything. It just provided everybody with some bullshit theater.

What about search?

2007-04-17T16:06:00.000-06:00

From my tumblog: but I don't want my search engine to be a slide show!

Google - remember that search engine of yours? How about making it better by offering some options like result filtering ("I don't feel like shopping right now, I'm trying to research")?

Reuse and non use

2007-04-17T11:55:00.000-06:00

We’ve been using Zope 3 in earnest for just over a year and a half now. I would like to report that in that year and a half our little company has achieved more re-use than at any time in our history. This is real re-use too: libraries of tools and objects that are easily shared among both horizontal and vertical markets, yet customized for each customer as needed. Benefits for one are fairly easily shared with all.

In the Zope 2 days, we tried hard to achieve this. But we were constantly having to re-invent the kind of architecture that I believe really makes this work: adaptation, which also brings dynamic view binding, dynamic UI generation (ie - registering a ‘tab’ for a particular object / interface and having it show up in the UI as necessary, etc. We had to spend a lot of time making the frameworks that would let us make frameworks.

“Frameworks for making frameworks?” - you heard right. Let’s face it: most web work is custom development. Sometimes custom development is best served by tools like Ruby on Rails or Pylons, or even by plain old PHP. But sometimes you know you’re going to have at least five customers all needing variations on the same thing in the coming months; and potentially more after that. You’re going to need to at least make a library or two.

See, Model-View-Controller isn’t just about “separating business logic from presentation”. It’s about separating it in a way that you can take business objects and logic (the ‘model’ layer; or models and services) and put more than one view on them. And by “more than one view”, I don’t mean “more than one template.” I mean putting wholly different user interfaces on it. I mean being able to take a base library and override a few select options (or many select options) as they appeal to a customer.

We tried to achieve this on some of our Zope 2 products, but it was hard to extract frameworks. We did OK, however, but I think that the most re-use we ever got was about three or four customers on one toolkit. That was over a three or four year span. We re-used patterns and snippets quite often, but it took a lot of work to extract an e-commerce toolkit from a particular customer’s site, and more work still to make it adaptable and workable for different customer requirements.

In the year and a half since using Zope 3 full time, we’ve had double that - and with far greater results. It’s not an easy system to just start using from scratch, but it can be quite worth it.

Being back at work on some legacy Zope 2 projects has made me all the more appreciative.

By the way: for a simpler Zope 3 development experience, check out Grok.