Van Couvering Is Not a Verb

Now this is a vacation

2008-07-16T18:52:00.001-07:00

Well, it's wonderful here. We are thoroughly enjoying our visit. We are in a huge beautiful house, we are with a friend and her son as well as my wife's aunt, we are visiting places, hanging out, cooking delicious meals, going out for food, and just soaking in the sunshine and the fact that we left all of our many-layered obligations behind us for three weeks.

It was a lot of work getting our house ready, but it was worth it.

The picture above, by the way, is from Las Estacas water park. This is a park surrounding a river that bubbles up from the ground, and then you can ride down in an inner tube or a life jacket and look up at the incredible lush greenery or look down into the crystal clear water. Or you can swing from a rope and jump straight into the water, or wander through the lotus garden.

To add to the romance and drama, every night there is thick moist stillness and crickets, then a rumble of far off thunder, then wind and lightning and BOOM a huge thunderstorm, power sometimes goes out, rain pours down in sheets, and the next morning all is clear again.

Did I mention the mangoes, the bunches of tiny bananas, delicious Mexican food, a massive kitchen, an inner courtyard, and the fact that our friend sharing this home with us is a chef?

What can I say? This really is the life...

Someone is having fun with me

2008-07-01T10:37:00.000-07:00

We are getting ready for a wonderful three week vacation to Cuernavaca, Mexico. We are doing a home exchange with a family of four who have a home there. Their place is huge, has a swimming pool and a trampoline, and we are going to have loads of fun.

But then there is getting ready. We really want the place to work well for our guests, and in the past things have happened that were no fun, like the sink clogging up. And I guess Someone Out There is having a little fun, because in the weeks before leaving, we've had one thing come up after another, and for each one, I am so glad that it didn't happen to our guests:

Our sprinkler system got cracked and required a complete overhaul (yes, I did this myself, many trips to the hardware store for pipes and valves and glue and mud and getting sprayed in the face and...)

Our car battery ran out of steam and we kept having to get jumped (we're doing a car exchange too), so the car is at the shop today getting its battery replaced

The day before we were going to get its wings clipped,one of our cockatiels escaped and flew up into a neighbor's tree and wouldn't come down. We drove out there with the bird cage and the other bird, treats and a pillowcase. No luck, and this morning we had to try again. This time it was in another tree, and we finally coaxed it down with a millet stick (his favorite treat). We were very lucky. We lost our first cockatiel for good this way (these two are from an animal shelter).

We had mice and rats in our garage and our food was being raided. Took two weeks to get rid of them, I'll spare you the details

At 4:30 am two nights ago I awoke to a roaring sound. I stumbled downstairs to the kitchen and walked into a pool of water that spread from under the kitchen sink and all the way into the living room. The ominous roar came from under the sink, and taking a deep breath I opened it and water was spraying everywhere. Of course, the shutoff valve was at the back of the opening, so I dove in and spluttering and being smacked by water I got it shut off. Nice way to start your day... Linda and I spent the rest of the morning into lunchtime cleaning up and replacing the water filter, which at that moment had decided to give up the ghost in a dramatic way.

Can you imagine any of that happening to our guests while we were away? Or worse yet (with the water thing) happening when we had no guests? So as crazy as it's been, I actually feel blessed that all these things happened now, instead of later.

Theo Schlossnagle - Traffic spikes through the heart of your app

2008-06-30T13:52:00.000-07:00

Theo Schlossnagle's blog is a very interesting read indeed.

What isn't entirely obvious in the above graphs? These spikes happen inside 60 seconds. The idea of provisioning more servers (virtual or not) is unrealistic. Even in a cloud computing system, getting new system images up and integrated in 60 seconds is pushing the envelope and that would assume a zero second response time. This means it is about time to adjust what our systems architecture should support. The old rule of 70% utilization accommodating an unexpected 40% increase in traffic is unraveling. At least eight times in the past month, we've experienced from 100% to 1000% sudden increases in traffic across many of our clients.

That's something to think about.

One likes to think that cloud computing is the silver bullet for scalability - use their stuff, build a system that assumes scale, and no worries, mate. But this makes it clear that there are basic operational issues that need to be kept in mind. Doing it manually may not cut it.

Amazon for example provides no tools for handling traffic increases automatically. It's your job to monitor and kick off new instances. But if this is happening in a matter of 60 seconds, you'd better be very sure you know what to do, and quickly.

Personally, I've never had to deal with this, so I have no tips for you. If I were to start up a new site, I'd go find people who do and have a long talk with them.

New DB Features coming in NetBeans 6.5

2008-06-27T10:45:00.003-07:00

Some cool stuff coming down the pike for NetBeans 6.5 database tooling.

SQL Editor code completion - lots of you have been asking for this, and it's in, with more on the way. Note how completion brings up the columns if the SELECT statement already specifies the table name.

SQL History - You can now look at the history of all SQL you've executed, and can to text filtering and/or filter by connection URL, and then select and insert a statement.

Editable, sortable results, multiple result tabs

This isn't integrated yet, but I had to show it to you. With this you can sort the results by double-clicking on a row header (we should have done this a long time ago), modify a row, insert and delete rows, there is full pagination support, and you can optionally create a new tab for each statement that returns results, very nifty for comparing results back and forth.

This feature actually came from the SOA team under a completely different part in Sun - open source works inside as well as outside. Thanks to Ahi and Nithya and Nilesh for this!

Tired of typing in a JDBC URL?

Many of our users, and they're growing, aren't even doing Java, but when they want to connect to a database, the dialog is all JDBC-ish. We've fixed that - now you just type in the parameters and we figure out the URL for you.

It turns out this is no walk in the park, as each vendor has their own parameters and URL format. So for now it works for MySQL, PostgreSQL, Java DB, Oracle and Microsoft SQL Server. For the rest, sorry, you still need to type in the URL, until I can get to them.

BTW, if you still want to see and edit the URL, you can, and we'll update the fields automatically. Nifty.

Jerry Seinfeld on George Carlin’s Life and Comedy

2008-06-24T10:01:00.001-07:00

I loved George Carlin as a kid. I once thought of a great joke - what is the worst ice cream flavor you ever heard of. I went to a Carlin concert, and he had the same joke, with the best answer: "bologna swirl." Carlin already did it.

http://www.nytimes.com/2008/06/24/opinion/24seinfeld.html?em&ex=1214452800&en=33d98b4f11507cbf&ei=5087%0A

Query languages for Hadoop, AND tooling for the non-relational stores

2008-06-20T16:26:00.000-07:00

Tom White just blogged about three, count them three, different query languages being built for Hadoop.

Pig, from Yahoo! and now incubating at Apache, has an imperative language called Pig Latin for performing operations on large data files.
Jaql, from IBM and soon to be open sourced, is a declarative query language for JSON data.
Hive, from Facebook and soon to become a Hadoop contrib module, is a data warehouse system with a declarative query language that is a hybrid of SQL and Hadoop streaming.

As someone working on database tooling, I have been thinking about what we at NetBeans might do to make it easier for developers to build apps against these new platforms/products like Hadoop and CouchDB and Amazon services.

These query languages are an example - I can see an interactive query editor that lets you write and test queries directly against your Hadoop or CouchDB engine, much as you can today with SQL tools.

Another interesting question is - if I write my application in terms of domain objects, what things can a tool do to make it easy to map these domain objects to an underlying non-structured store like Hadoop or CouchDB or SimpleDB.

Definitely some opportunities here to make developers' lives easier.

Kaj Arnö - Happily using NetBeans to build MySQL

2008-06-19T20:28:00.003-07:00

After getting a demo from a NetBeans engineer on the C/C++ extensions to NetBeans, Kaj was converted - "now this was enough even for a sceptic like me to become eager."

http://blogs.mysql.com/kaj/2008/06/18/netbeans-as-ide-for-developing-mysql-itself/

Alex Bunardzic » Software Development Detox Part 5: Session

2008-06-19T20:28:00.001-07:00

Someone who is in my boat about session. Although I still wonder if you are overburdening the client if you say "on the web each request contains all the information necessary for the server code to make a decision on what to do next."

http://jooto.com/blog/index.php/2008/06/20/software-development-detox-part-5-session/

Alex Bunardzic » World Wide Web is About Self-Serve

2008-06-18T23:57:00.001-07:00

Interesting blog challenging us to question if there is a need for services, when really the web is basically about resources. I think the real service is in quality of service, making sure the resource is available when you need it.

http://jooto.com/blog/index.php/2008/06/19/world-wide-web-is-about-self-serve/

AOL Radio

2008-06-16T09:12:00.000-07:00

I have been a long-time member of the "AOL sucks" club. I signed my name in blood as a member of this club when I tried for three days to get Internet access working for my laptop in someone's home that had AOL as the Internet provider. AOL has historically provided a horrific user interface which constantly thinks it knows better than you and gets in your way whenever you want to do anything different from the standard.

But I've found one piece of AOL that is just, what can I say, nice... It's AOL Radio, which I bumped into when reading a NY Times article about how AOL Radio will soon be available on the iPhone. I tried the Acoustic Rock station, and it's really quite high quality, and the user interface, is, well, simple and - gasp - easy to use!

I have tried a number of interfaces for free Internet radio, and they are either impossible to use, have a horrible selection, or are filled with ads. AOL Radio has none of these limitations. And all you need is Flash to get going. They do seem to have song-skipping disabled, but that's supposed to come back soon.

Pretty nice, and from such an unexpected source! :)

Facebook's Thrift in Apache Incubator

2008-06-13T00:45:00.001-07:00

From the Facebook blog:

Thrift is a lightweight software framework for enabling communication between programs written in different programming languages, running on different computers, or both.

It includes support for C, Java, C++, Ruby, Erlang, Perl, Haskell and many others.

It sounds very interesting. I first heard of Thrift because it's used by a very interesting Amazon-like stack called Thrudb. What's nice is because it works with Thrift, you can use it from any language.

However, I looked at the interface for calling the Thrift APIs, and it's a bit ugly, or at least it takes some getting used to.

Anyway, something to keep an eye on, it may have legs.

Rhapsody on my Mac

2008-06-12T23:14:00.000-07:00

I really like the Rhapsody music service. I have been trying various ways to get this to work on my Mac.

There is a plugin for the browser that lets you access Rhapsody natively on the Mac. The only problem is that it regularly crashes Firefox, and Rhapsody support clearly has fixing this as a very low priority. I tried Safari, and that was working OK, until I upgraded to OSX 10.5.3, and now Safari regularly crashes my entire machine! Sorry, no go.

So I finally gave in, and am happily running Rhapsody natively on my Win XP VMWare instance. I reduced the memory on the instance to 512MB otherwise if I have two instances of NetBeans up my system memory starts thrashing. But once I did that I am happy as a clam. Mozart is lilting to me as I code, and all is well with the world.

Open Source EC2 - the beginning of a Scale Stack

2008-06-10T14:37:00.001-07:00

In my last post I mentioned that what I wanted to see was the industry coalesce around an open source "scale stack" [1].

We may be seeing the beginnings of this. The High Scalability site (great blog, by the way) recently posted a blog talking about a new kid in town coming out of UC Santa Barbara called Eucalyptus . They are providing an open source implementation an elastic compute infrastructure that is interface-compatible with Amazon's EC2 which you can take and deploy on your own hardware.

This is very encouraging, and I think is a smart approach. Rather than try to build some standard that is lost in committee for years, use the de-facto standard, which in this space is Amazon.

Another piece of the puzzle is Hadoop, an open source implementation of map/reduce. Hadoop also has a distributed file system - one thing that might be worth investigating is building an S3 layer on top of Hadoop's file system.

What about the queuing service? Well, one possibility is to but an SQS API on top of ActiveMQ or OpenJMS.

Throw in CouchDB, and you're starting to get a very interesting stack indeed. I'm not sure about putting a SimpleDB interface on top of this - CouchDB is pretty darn interesting in its own right, and I think the jury is still out on SimpleDB.

[1] I am not sure if he wants me to mention his name, so I won't, but I want to acknowledge that the idea for an open source stack based on Amazon's APIs is not my own, but comes from a colleague at Sun. I think it's a great idea, and may it come to fruition.

The exponential cost of contention

2008-06-05T12:58:00.000-07:00

I enjoy Nati Shalom's blog, although it always has that taste of having the agenda of pushing Gigaspace's solutions. But putting that aside his posts are always well thought-out and well written.

I think his latest blog on the Economies of Non-Scale really drive some points home about scalability, or more to the point, the cost of non-linear scalability.

If 90% of our application is free of contention, and only 10% is spent on a shared resources, we will need to grow our compute resources by a factor of 100 to scale by a factor of 10! Another important thing to note is that 10x, in this case, is the limit of our ability to scale, even if more resources are added.

...

1. The cost of non-linearly scalable applications grows exponentially with the demand for more scale.

2. Non-linearly scalable applications have an absolute limit of scalability. According to Amdhal's Law, with 10% contention, the maximum scaling limit is 10. With 40% contention, our maximum scaling limit is 2.5 - no matter how many hardware resources we will throw at the problem

That's something to chew on. These are real costs, both to your business, to your users, and to the environment. Even if you only have a teeny 10% contention in your system, that 10% will nail you faster than you can say ACID semantics. And as has become very clear to me the final breaking point, the final point of contention in any traditional web application architecture, is the database. Get rid of that and you're home free.

How do you do that? Well, there are a lot of people trying to solve this problem with things like space-based architectures, eventual consistency, distributed map/reduce and Stonebraker's H-Store architecture. Anything to let each instance stand on its own and not have to serialize with the rest of the system at any point, in any form.

Some people argue that scalability is so hard that you shouldn't think about it until you need to. But I really believe that if you do enough to educate yourself and make some wise choices, you will be very glad you did.

What I'd like to see is the industry coalescing around some best practices, an open source "scale stack" ala LAMP, tools, community, and hosting environments like Amazon, that allow developers to easily build applications that will scale from the get go. That's where I want to see things go. That way you don't have to throw your hands up and hope for the best. Because as you can see, the costs can be deadly.

Something to think about...

Greed and vision

2008-05-27T15:01:00.000-07:00

I was listening to NPR this morning to a fascinating analysis by someone who has talked to everyone from homeowners to lenders to Wall Street about the home loan fiasco. He said everybody up and down the chain was taking unreasonable risks, and everyone was making tons of money off of it.

Then he quoted a financial expert, I wish I could remember who, who said

It's amazing how hard it is for someone to see something when they are being paid a small fortune not to see it.

I just had to laugh, that really sums it up.

It isn't because you're dirty, it isn't because you're clean

2008-05-12T10:11:00.001-07:00

My daughter is currently on antibiotics, and she's staying home from school because she has a sniffle.

Why is such an innocent sniffle the cause for keeping her at home, and why is she on antibiotics? Pertussis, or whooping cough.

My daughter goes to a private school, so vaccinations are optional. Many chose not to vaccinate for pertussis (this is actually one we did), and these are the consequences. The Department of Health for the county closed the school for the day on Friday, and is requiring that everyone, regardless of whether they've been vaccinated or not, either take a 5-day protocol of antibiotics or stay home for 21 days. They also have implemented a no-symptoms policy, so anybody with any symptoms, including a sniffle, has to stay home. Sorry, Ariel.

So, swirling around this is the whole debate of vaccinations. As a father, I was stunned at the number and frequency of vaccinations we are supposed to give our children, and which you are required to have before you can enter public schools. I understand, this is all in the name of public health, but as a parent you have to use your intuition a lot, and this just intuitively feels like overkill, literally. It reminds me of the latest discoveries I read about in Science News where there are bacteria in dirt that, when inhaled, create a very strong immunity. Children who are kept away from dirt and kept super-clean are more prone to disease.

I understand vaccinating against horrific and fatal diseases. But many of these vaccinations are for diseases that many of us older folks had as a child: chicken pox, measles, mumps. It was just part of growing up. And there are some real issues here. I am no doctor, but as I understand it, if you are vaccinated against chicken pox and never get it as a kid, then you have a chance of getting it as a teenager, and then it can be *really* bad, whereas if you get it as a kid, then you get a much stronger immunity and generally don't get it when you're older.

I understand then challenge of maintaining public health over trying to protect your child from an army of doctors with needles. I mean, I would be horrified if my child or someone else's child died because of my unwillingness to vaccinate. But at the same time, I wonder what is truly the right thing to do here. We are often so focused on eliminating all and any suffering from this world that we fail to see the ultimate consequences. As an example, part of the reason the plague pounced upon Europe so badly in the middle ages is because humans had been so successful at eradicating the wolf, which, you guessed it, was the primary predator for rats.

I couldn't tell you what the unexpected consequences may be for so much vaccination, but I can tell you that it feels like overkill, and I personally recommend that as a parent you do your research and try to make informed choices about which vaccinations you give your kids, rather than blindly doing none or blindly following the protocol laid out by the powers that be.

MySpace to share data

2008-05-09T09:28:00.000-07:00

This just in:
MySpace is going to allow users to share information from their site.

This is definitely a step in the right direction, but notice that sharing is done either by copying/synching, or through their specific APIs. It doesn't fully extract the content into a separate layer with application-provider-agnostic service APIs.

One can imagine an OpenSocial API on top of a social network data service that is used by numerous providers but owned by none.

Giant Food & Biotech Corporations Make Billions in Profit from Growing Global Food Crisis

2008-05-09T09:27:00.001-07:00

Big Food is serious big business, and thus is inherently evil (profits over social needs, always - the corporate credo).

http://www.organicconsumers.org/articles/article_12088.cfm

Internet content by reference, not by value

2008-05-08T21:41:00.000-07:00

You may have noticed that I am very interested in how data is managed on the Internet as a platform, at a web scale. In that light, I have been having some very illuminating and interesting conversations with an old friend and colleague, Nitin Borwankar. His thoughts on data property rights and DRY data are concepts that if implemented could result in a major shift in how we manage data on the web.

Data property rights is about laying out a "bill of rights" for data that goes far beyond "the right to move". It also includes the right to access, modify, remove and own your data. So often it happens that once you upload your content to a site, you no longer have full rights to that content, as if somehow in the act of uploading it it is no longer yours. It's like living in a serfdom where you do all the work to plow, seed, tend and harvest the land, but the fruit of your labor is not yours, just because you are using the land that someone else owns.

DRY data is about following the principle of Don't Repeat Yourself for web content. Web applications need to start applying this principle, so that rather than you having to load copies of your content across multiple sites (and losing ownership of it in the process), you place it in one location (your "home" on the Web, as it were), and then you refer application providers to that one place. They can focus on providing added value (for instance, referring it to your friends, enabling collaboration, or helping you organize it or present it in useful ways) rather than on the overhead of building and deploying a scalable storage architecture.

Nitin calls this architecture YINAS (YINAS Is Not A Silo).

The value of DRY for the user is obvious - I only have to put my stuff in one place, and I get to really own my stuff, rather than the vendor owning it. DRY is also very valuable for the vendor, as they can save overhead and complexity by delegating the work of scalable storage and indexing to a "data service provider" rather than having to do it themselves. It's even good for the environment, because you need fewer disk farms sucking up power and space. I guess the only folks who would lose out are the storage and power vendors :)

It's funny, it makes so much sense, but nobody is really doing this.

I pulled Tim Bray aside at Java One to talk to him about these ideas after reading his blog about changing his address, and he suggested that concepts are good, but a simple proof of concept is better. Hm... let me think about that ... :)

Meeting Good Friends at CommunityOne

2008-05-05T21:44:00.002-07:00

Lots going on these days, hard to keep up. I have been particularly busy, and that's why I've been quiet. I suspect you probably don't mind too much, dear reader, because if you're like me, you probably experience information overload on a regular basis.

I was at CommunityOne today, and, as seems to be happening these days, I spent most of my time saying Hi to old and new friends catching up and sharing our thoughts and feelings about what is going on in the industry.

I ran into Ted Leung and he shared some of the work he's doing to try and get some of the powerful OpenSolaris features like DTrace working in the various dynamic language environments.

Simon Phipps was hanging out (literally, his feet dangling from the RedMonk UnConference stage during a break) and we ruminated on the fact that Sun seems to be held to a higher standard than many companies in the industry - people see us as a Good Company and so they get very upset when it looks like we're being Corporate and Just Like Everyone Else. There is a sense of betrayal and lost hope. So in a way, it's a positive reflection on Sun - we in general have been good to the community, and on the up and up, so any indication that we're Falling to the Dark Side creates a passionate response.

I bumped into Joe Keller, who was one of my first VPs at Sun when the little company I was at was acquired by Sun. Now he's working for Kapow! (now there's a name), and it's fascinating what they do. He described it, and here's how I interpret it: they're doing data integration at Web Scale. Their customers define a master database schema, look at the web pages that they're interested in (say governmental pages for each state in the USA), and define a mapping between the HTML elements on these pages and the elements in their schema. Then the Kapow! software does its magic, scraping these screens at various semantic levels (screen, HTML, through APIs, etc.), and slurping this data into their database schema. Customers then use their data warehousing solution to analyze the resulting data. As an example, one customer of Kapow! is the company Sun hires to do background checks on potential employees. Yipes!

I saw my old boss Dan Leighton, now an independent consultant, but who was instrumental in setting up the database group and defining our database strategy, working for Rich Green, which ultimately led to our involvement in Java DB, PostgreSQL, and, of course MySQL. So he was happily surprised to hear of Sun's acquisition of MySQL.

I bumped into Rob Stephens, someone else who came from the Clustra acquisition and is now in the Solaris org. He told me I should check out the new release of VirtualBox, the open source virtualization solution whose parent company, which was recently acquird by Sun. Rob's got a very strong sysadmin background, so when he says it's good, I trust him. I had tried out VirtualBox about a year ago, and he says it's really improved. Downloaded, installed, and now pulling down the first GA release of OpenSolaris and will put it in its virtual box and put it through its paces.

I ran into Francois Orsini, who is demonstrating some very interesting stuff with Java DB and Grizzly. I ran into a Sun colleague working on a big Rails project, and we talked about web platforms (Rails vs. PHP vs. Java), scalability, and the place of the relational database in a platform that needs web scale. All very interesting stuff. We'll see how the day goes tomorrow!

OpenSolaris on Amazon EC2

2008-05-05T21:44:00.001-07:00

Wow, great news. OpenSolaris has some very powerful technology, particularly DTrace and ZFS. Having this available as an Amazon Machine Instance (AMI) on EC2 is sweet!

http://www.sun.com/third-party/global/amazon/index.jsp

Windows Live Dev : Introducing Live Mesh

2008-04-24T23:07:00.001-07:00

Very nice, very interesting stuff. This is in line with a lot of what I've been thinking about. Too bad your stuff has to go onto *their* servers and is tied to a very proprietary company. Imagine doing the same thing in an open source model.

http://dev.live.com/blogs/devlive/archive/2008/04/22/279.aspx

Paul Graham's Business Strategy: Be Good

2008-04-22T13:04:00.001-07:00

"Being good is a particularly useful strategy for making decisions in complex situations because it's stateless... it's the only algorithm that works on [a large] scale." Great article.

http://www.paulgraham.com/good.html

Seeped in MySQL

2008-04-16T22:44:00.000-07:00

This is day three of the MySQL conference for me - usually when people hit burn-out, in my experience. And I'm definitely getting there - I left the Sun party early, my tongue and ears tired of talking and listening.

I have met a lot of great folks here - I was on booth duty a lot and people just kept coming by. This includes saying hello to our new colleagues from MySQL, hello to my old buddies in the database group, and also hello to very good old friends from my Sybase days.

Just as an example, I ran into Robert Hodges, who I used to sit near in the B-Treeum (our building next to the Atrium) in Emeryville. He was the lead on the message queue product, and I led the app server team. Now he's the CTO of Continuent.

Hi, Robert!

I also ran into Wen Hu. I worked for him when he was the lead architect for a massively parallel database engine Sybase was working on with Teradata called Navigation Server. Haven't heard of it? Oh well. Then he moved on to implement the partitioning and parallel query support for the Sybase database kernel. I think MySQL is just now implementing something like that. Now Wen is the CTO for ScaleDB. Sorry, no photo of Wen.

Both of these guys are working on scalability solutions for databases. That is the one theme that repeated here over and over: scalability. All these web properties are encountering scalability requirements that actually go way beyond enterprise scale requirements. You basically need to be able to scale, well, infinitely, and linearly. It's amazing to me that this tiny database engine originally built for quick departmental web apps is now having to address requirements that go way beyond anything Oracle or its peers from the 1980s was built for.

And all the folks I talk to agree, standard transactional/relational approaches just ain't gonna cut it. Something has to give. There's even an acronym for this that I just learned at this conference: CAPS - Consistency, Availability, Performance, Scalability[1].

As Robert explained it at his talk, these are four qualities, and you can't have all of them at once. Something has to give. For a lot of folks, what they are most ready to let go of is consistency. For example, if I write something, I want to see it show up right away. But another reader may not see it for a while (we're talking on the order of seconds, maybe a minute). And that's fine. You get this if, for example, you do asynchronous replication to multiple nodes/sites, rather than trying to do consistent, synchronous replication. And as you can imagine, asynchronous replication scales much better than synchronous replication.

This is the model Amazon has taken, to great success, and I believe it's time for this to spread to the mainstream. The important thing (again, thanks to Robert for crystallizing this for me) is that the levels of consistency need to be clearly defined and guaranteed, like the transaction isolation levels.

I talked to some MySQL folks, and they seem to be looking very seriously at this issue of CAPS, and at these alternate storage solutions like Hadoop/BigTable, SimpleDB and CouchDB. So I look forward to seeing if and how the MySQL folks address this growing need.

[1] I have searched for a reference to CAPS on Google, and can find nothing, so it must be something new. If anyone has a link, please let me know.

Phanfare, S3 and Google Apps

2008-04-16T22:37:00.000-07:00

I'm looking very seriously at moving to Phanfare as a replacement for Flickr and YouTube for my photos and videos. Their video quality is much better, and I can use it for video archival as well as display. My friends and family can even download the videos at DVD quality and play them offline or even burn them to DVD. Now that's cool.

The cost: $59 a month. Worth it in my book, and since I'm paying up front I'm pretty sure I don't have to worry about advertising polluting my content. Hey, waddya know, a different business model than free-with-ads, which I am so tired of.

I also like it that they're on S3.

They also have a good blog. I particularly liked their analysis of the benefits of the Amazon Web Services over Google Apps. I think they provide a great testimonial for the value of following open standards - something Sun has always believed in.