BehindLogic: My Ruby World

My Gmail [GTD] Request

2008-07-23T09:54:00.003-04:00

This is the exact text I posted as a suggestion to Gmail:
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

I would like, but have no way currently, to be able to organize a set of labels that are time-sensitive. In the principle of Getting Things Done, I'd like to keep my Inbox full of only "messages I've received but not yet dealt with," not those plus "messages I plan to deal with later." A GTD principle would suggest that I should move some emails into a "deal with tomorrow" or a "deal with next week" label, and that those messages should then automatically reappear in my inbox tomorrow, or next week.

The concept then, again, is this: My inbox is used only for messages that must be dealt with "today," whether they are read or yet unread. The goal is to empty the inbox every day either by dealing with messages directly or by moving them into a "tomorrow" or "next week" label.

The beauty of this is that it can be implemented without ANY forced change to the way people currently organize their messages. The implementation would likely be in the way of making some sort of "timed" filters, which people can use to do their own organization with many times more power.

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Would you like such a feature?

Big G, Little G, what begins with G?

2008-07-22T13:12:00.002-04:00

This little line from Dr. Seuss is a pretty simple kids concept until you use the word Government. Then it's a whole new question. "Big Government? Little Government? What begins with Government?" Wow, big question. I've had numerous things pop into my existence in the past few weeks that gave me little insights into this topic. Why big government works. Why it doesn't work. When small government is needed. When it doesn't work. So what is our balance? Where do we begin our governing, and where do we end it? These words came to me after the concept had been formulating in my head for a couple weeks:

Government should only do what only government can do.

It became so simple and so understandable. It's a complete policy in itself, one that could govern the government, if given a chance.

What I want you to do is memorize it, refer to it in conversation, consider it in your voting, and teach it to your kids. It could go a long way.

ActiveResource vs SimpleMapper

2008-05-28T11:03:00.006-04:00

I wrote SimpleMapper a while back as a replacement for ActiveResource, for two reasons:
1) I needed to work with more than one data source.
2) I needed to use OAuth.

The biggest need was for an ActiveResource with OAuth. I found that it's virtually impossible with the current structure of ActiveResource. (I did get it working, after adding a "callbacks" plugin to ActiveResource and rewriting ARes's core request method.)

So I sat down one morning to write SimpleMapper. The name is rather random -- I like DataMapper, and I wanted to stress that this was just a simple version of a DataMapper / ActiveRecord solution. It took an hour or so to write from scratch the parts of ActiveResource, and I was already happier with my solution than with ARes. The simple structure is what makes it so powerful and extensible.

The structure: I started with a simple Base class, that simply provides methods that bind Connection Adapters with Format Adapters. For example, the Base.get(*args) method simply calls the Connection Adapter's get method with the arguments provided, then extracts individual objects from the returned content by using the Format adapter. This way, the Base is completely independent of the type of Connection and Format being used. I started out by writing an HTTP adapter and an XML adapter, and later I wrote a JSON adapter. Since all the parsing code is in the adapter, it's extremely simple to write new format adapters. You could easily write a yaml, or even csv or plaintext adapter. Or SQL. Write a SQL connection adapter, if you wish (I haven't tackled that yet).

There's one more piece worth mentioning -- ActiveResource does NOT follow the REST way. How? ARes relies on IDs, not URLs. (See Building Web Services the REST Way, down in the section entitled "REST Web Services Characteristics," for authoritative information.) The REST way uses URLs, not IDs. IDs are not reliable. (In thinking about the server side, you should implement a kind of global unique identifier [guid] system for your URLs instead of IDs.)

So SimpleMapper relies only on URLs. The way this works: You call a finder url with the options you need to find some object, a url that returns either a collection or a single object, and go from their. Your API *should* include a URL in the object's properties as the identifier of the object, never an ID. Then to update or delete that resource, it will access it directly via the URL given in the resource's identifier property.

To me, SimpleMapper wins, hands down. If you want to run an ActiveResource-like API, go ahead and clone the http adapter and change things to use an ID to construct-a-URL rather than reading the object's identifier as a URL.

Feed me an API: He who does not work, shall not eat.

2008-05-28T10:00:00.000-04:00

On the road to openness, there is one principle that should not be taken lightly: He who does not work, shall not eat. When you build an application, if you want to feed off someone else's data, the right thing to do is to build an API for your application first.
Take note, Facebook, Plaxo, Google, Microsoft, Yahoo, and every other proprietary system out there! Feed others before you just feed off all their hard work.

A good read for background understanding, try The Value Chain for Information, by Elias Bizannes.

The root of the value chain for information is access to the information in the first place. If you have exclusive access to a piece of information, then you have a monopoly on the entire chain of value for that information. With the dawn of the web, a couple of guys named Larry Page and Sergey Brin (Google) jumped on this fact with a mission to "organize the world's information and make it universally accessible and useful." Good move. They actually took the most logical step. First, we have data. Second, we have to organize it. Third and following are numerous other ways to make use and value out of the data. The reason we like Google is that they [say they aim to] hold the monopoly only on the first step, the step very few of us would want to try to tackle in the first place -- and then they freely open the data to the rest of us to go at adding value in our own innovative ways.

Probably each one of us, when building a web app, have thought about the monopoly of owning the data. But I implore you to study a little closer that value chain of data, and realize that you cannot cover all ways to add value to even the data that you own. You must open it up to others to fill in gaps that you hadn't even thought of.

The ONLY way you can really get by these days is to open up your data. Remember, your app should do one thing and do it well. I've said this before. That one thing should cover a concrete portion of the value chain of information. And you should be providing the data in such a way that it is easy for others to add value to it in whatever way they choose.

Now, if your data is private user data, you'll have to use OAuth to open it up securely only with the user's permission. If all, or even some, of your data is public, then provide it publicly. Its up to you how you want to do it, but remember: Feed others before you feed yourself.

True Data Portability

2008-05-05T12:27:00.004-04:00

I've been thinking about data portability and the phantom "online integrated experience" for a long time now. I know it's possible, and it's within our reach. It's the integration of countless web apps or online services, with each other with no previous knowledge of each other that will make data truly portable, and give us a truly integrated online experience. Sites that are agnostic of each other should be able to connect to allow one to access a user's data on the other and make more sense of it, or do something with it to make it more valuable, and update the user's data on the original site. It would be awesome if I could make a simple web service and integrate it with whichever calendaring site I use (google calendar, other?), whichever to-do list site I use (remember-the-milk, tadalist?), and so on. You could use whichever other sites you use, too, and I wouldn't even need to program any specific API's into my application. This is possible. They have a name for that -- standards. Standards make things work in compliance with one another. It's the same principle that ushered in the industrial revolution.

We have standards for many things -- some of the most recent to become excited about are OpenID and OAuth. These are a form of standards, called specifications. OpenID allows anyone to operate their own authentication -- allow a unique and globally identifiable entity (your openid provider) to agree that you are who you say you are. (The fallacy is that it should often be an entity trusted by both parties, but it ends up being an entity that is proven to be trusted only by the user.) OAuth defines a procedure by which any two sites can talk to each other with a user on hold in order to share data owned by that user, with the user's permission. These are very good and necessary improvements. OpenID is nice, providing a way I can identify myself in the same way to all the services I use; and OAuth is nice because it allows sites to use and modify my data on another site.

But full integration is still a step ahead of us. Every site still has its own language. Each programmer has to make up his own language for his app or service to speak (it's called an API). An API (Application Programming Interface) is "a specification, that defines the means and language of communication with an application." Almost all web services have an API responsive to the same means of communication: HTTP. But the language of communication is still different: each web developer ends up making up his own.

What we need is common terminology, common concepts that define an object (a package of data, a record, etc). REST shows an aspect of this, inferring that any object should respond to (at least) four verbs: GET, POST, PUT, DELETE. In the same way, but going a step further, there are common concepts that we can all apply to common data objects:

Object: Package of custom data. Read, Update, perhaps Append.

Listing: several instances of Object in their abbreviated form. Read (with filtering options).

Collection: several instances of Object in their complete form. Read (with filtering options), Append.

37Signals has debuted a very important piece with ActiveResource -- calling convention to dictate common terminology. Using "include" to mean you want the object an associated object embedded is one example, using parameters the same name as model properties in order to filter your results is another convention. What we need is more of this, but re-thinking it from the concepts at the foundation of data.

I've started a google group for this initiative. I can't say how far it will go, but I'd like to get some key people into it to discuss these things. Especially people who hold some of the key implementations (like ActiveResource) would be wonderful.
See the group: http://groups.google.com/group/cloudapi -- JOIN if you have ideas to share!

Three Web Trends to keep your eye on

2008-04-28T12:35:00.001-04:00

Trend #1: Do one thing and do it well.

In developing web apps, translate: "Replace feature-bloat with KISS and Integration." I learned this lesson in two or more areas of life separately and realized it's a global principle. The same principle was learned in the Industrial Revolution (throughout the 1800s) -- each worker can excel at making one simple part with a high level of quality and consistency, and the product can be put together with these many parts. Of course leaders of people know this same principle -- when each person has their tasks well-defined and knows how to accomplish them efficiently and excellently, there is little limit left as to what can be accomplished.

Why do I call this a trend? You've heard of the "KISS" concept -- Keep It Simple, Stupid. The more I use the internet (and I'm sure it's not just me), the more I find it's the simple services that are the most useful. The ones that simply do one thing for me, but they do it well, keep it succinct, stay out of the way and let me do my thing, not trying to do more (or make me do more) than I want. Gmail -- have you noticed that of all things it could be, it's remained ONLY an email service? All of the energy and effort has gone into making it easier to use, more intuitive to operate, and staying out of your way. Any extra tools (contact groups, etc) are kept in the toolbox, not in your hand.

This is also a psychological thing. People will come to your web app and think, what is this service? Notice: They don't think, "What can it do for me?", they think, "What need do I [already] have that this service can fill?" And then, "Is it worth the trouble?" This last question I don't think I've heard enough in a developer's evaluation of a web app. Some services are killer, the interface beautiful, the monetization simple and invisible, even the solution simple; but if it's not worth the trouble of using the site, the service is worthless. How can we navigate this psychological narrows? Make your app do one thing and do it well. Advertise the one thing it does. Feature-lacking? Open an API and connect with other sites. Add functionality by making separate and distinct web services yourself, and connecting the two. (If you own them both, of course you have advantages for integration, but do keep them separate and allow them to perform their distinct service to the world independently.)

Trend #2: Users are beginning to take Integration for Granted.

In practical terms: "Integration is Key. Do it."

In this wonderful world of online applications, this is something that started out as cool and somewhat helpful and has grown to something that is necessary to forward movement. The world of online applications must start to link arms in order to grow. Many of these simple applications (as following Trend #1) are so good on their own, but we all know they'd be even better together. Thus the word "Mashup" was birthed. Mashups combine two or more services into one to provide a more valuable service. That's great, but it's not the integration I'm talking about. I'm talking about in-app integration -- like access to your gmail contacts from gcalendar and gdocs. There's a really useful tool called "Presdo" that recently was publicized, that can export to several calendars. Real integration would mean any user can voluntarily "add" presdo data to their calendar app -- no matter what calendaring app they use.

But there's a problem. Integration is hard. Yes, most of my work in programming has been focused on integration of systems, migrating or communicating data, etc. I still say integration is hard. Why? Because 1) everybody manages a different type of dataset (one manages a calendar, the other manages a todo list), and 2) everybody organizes their data differently. It would be nice if everyone could structure their data the same way, follow global naming conventions, and provide the same API; but people also use their data differently, and some structures are more efficient for certain purposes. What I'm saying is, everyone's underlying data structures will always be different, and integrating systems on this level is hard; BUT the last idea, providing the same API -- or at least API language -- is not impossible. We already have standards such as XML and JSON that allow us to at least speak the same language. The same language for data. In the database world, we have SQL as a sort of standard -- it's a language that has been devised that very elegantly handles requesting specific data. But in the web world we have no standard. Communicating app-to-app is a new thing, and this new "language" will soon become necessary. It becomes evident with the somewhat recent birthing of "DataPortability" that we are on this path -- the prerequisite step of making our data usable to others in a secure manner is being finalized, and the step of standardizing how we communicate about this data is the next. Everyone who creates an API has to make up their own language, or set of request url's and query-strings, to communicate what data is being requested. We need a standard.

I'll post about this point further soon, and I think I'll entitle it, "He who will not work shall not eat."

Trend #3: Integration Bloat.

Wait a minute, you say. Isn't that contradictory to what you just ranted long and hard about? No, what I mean by "Integration Bloat" is what you get if you misuse the above principle. Users are beginning to take Integration for Granted. Think about this: What if Subway tried to convince you that they were better because they have mayonnaise? That would be ridiculous -- of course they have mayo. It's the same for web apps. We're coming to the point where I really shouldn't have to choose my favorite application based on who they connect with. I should be able to expect that they will integrate with other sites. Plaxo is an example of a failure here -- I don't know why they exist anymore (as in, what their focus is) because they accent so heavily on all their integrations. If they showed me one good reason to use their site, only ONE THING they do better than anybody else, I'd be likely to use them. But their integration bloat has gotten proud of itself and starts to get in the way.

(These trends are all related.)

Just a note on the topic of trends in general: these trends are all related. One gives birth to another, and the existence of one causes others to exist as well. Think just a little further and you will see connections between these trends, their parents, and the postmodern way of thinking that we live in. The good lessons from our culture we can apply to programming. The bad lessons we can learn from and fix. And if you haven't noticed it already, the postmodern mind is probably the first whose way of thinking has been primarily influenced by the paths of technological development. Things like wikipedia have both come from and contributed to the postmodernist's belief that the majority is right (and not because they choose the right thing but because they are the majority). The blog has both resulted from and fed back into the idea that we all have the right to express ourselves to the public at-large.

Please feel free to post comments, longer comments, or link to a response post on your own blog.

HashMagic: a utility hash extensions gem

2008-03-31T19:55:00.003-04:00

Rubyforge: http://rubyforge.org/projects/hash-magic/
Documentation: http://hash-magic.rubyforge.org/
Install:

gem install hash_magic

HashMagic, first of all, doesn't touch Hash except for two methods. Like the FormattedString gem, it just provides a way to get a "different kind" of the original object.

HashMagic provides two special kinds of hashes: OrderedHash and SlashedHash. OrderedHash is just what it says - a hash that keeps its order. You can specify a static order, if you want. SlashedHash provides a way to expand and collapse a multi-level hash structure into a flat single-level hash, with keys joined by a slash ('/') character.

There are times when this becomes very handy -- like when I want to weed out certain keys at various levels in the hash structure, only if they're there, without having to make a very complex method to walk around the structure and not complain when things don't exist; or when I want a hash structure of ordered hashes -- yes, they play together nicely. You can order a SlashedHash, even setting the order of keys in deep levels of the structure.

This was necessary for generating xml for quickbooks qbxml -- the elements must be ordered correctly, in a multi-level structure. This was also necessary in the SimpleSync gem, for mapping attributes -- the top-level attribute 'email' in one source (quickbooks) maps to a second-level attribute on the other side, inside an association: "<emails><personal>...email...</personal></emails>". All I have to do to accomplish that is 'email' => 'emails/personal'.

You wanted to see code? Look at the HashMagic Specs!

SimpleSync: a synchronizer for ANYTHING

2008-03-28T16:26:00.005-04:00

Step One: Set up a few models -- use SimpleMapper, DataMapper, ActiveRecord, ActiveResource, my Quickbooks gem, or any other ORM you can get to respond correctly to the few essential methods: obj = Model.get(finder_options={}); obj = Model.new(attributes={}); obj.save; obj.delete.

Step Two: Follow the five simple steps to synchronize your data source:

1) Create a syncer object

@sync = SimpleSync::Syncer.new( <last_sync_time> )

2) Add some sources

@bug_source = @sync.add_source(<source_model>, <finder_options>, <common_id_attribute_name (example: id)>)
@juice_source = @sync.add_source(<source_model>, <finder_options>, <common_id_attribute_name (example: bug_id)>)

3) Define for each source how to grab updated records to sync

@bug_source.new_records {|start_time| <block code that returns record objects> }
@bug_source.changed_records {|start_time| <code that returns record objects> }
@bug_source.deleted_records {|start_time| <code that returns record objects> }
@juice_source.new_records {|start_time| <code that returns record objects> }
@juice_source.changed_records {|start_time| <code that returns record objects> }
@juice_source.deleted_records {|start_time| <code that returns record objects> }

4) Map the attribute-conversions for all attributes you want to synchronize

@sync.mappings[@bug_source => @juice_source] = {
  'first_name' => 'FirstName',
  'last_name' => 'LastName',
  'company_name' => 'JuiceCompanyName',
  'birthdate' => 'Birthday',
}
# And usually just an inversion of the first hash, but sometimes not:
@sync.mappings[@juice_source => @bug_source] = @sync.mappings[@bug_source => @juice_source].invert

5) Sync!!

@sync.sync!

6) Wait for a while, then sync again!

@sync.sync!

The purpose of SimpleSync is to handle all of the logic of synchronizing and leaving all the specifics of the data to your custom models and attribute mappings. The above code was taken from a working script and slightly modified just to better illustrate the purposes of each piece.

Install the gem:

gem install simplesync

Now get this: I have a fully-functional synchronizer between Quickbooks Customers and my addressbook website (not yet public), using 3 of my gems: SimpleSync, SimpleMapper, and Quickbooks -- in a script of only 150 lines. Beat that, I dare you. :)

OpenID, OAuth, and DataPortability

2008-03-28T13:57:00.003-04:00

This is just the slides that went along with my presentation for two of the ruby groups I attend (SouthBend and Detroit). This is posted primarily for the people who heard the talk, just for review and reference purposes.

Share, please!

2008-03-14T16:25:00.002-04:00

Just a little note for you: If you follow my blog and you like it, please share it with your buddies, coworkers, fellow Rubyists, etc. The past several public releases of things I've been making are just a leading-up to some things to come. I guess I'm saying, if you've found this blog valuable, it's only going to get better!

P.S. Upcoming: Ruby: the Object, Class, Module, and Metaclass explained

SimpleMapper, the new lightweight and versatile ORM!

2008-03-11T01:12:00.012-04:00

I want to point you to a "new ActiveResource" -- SimpleMapper. SimpleMapper is my complete rewrite-from-scratch of ActiveResource. Kindof like Ezra Zygmuntowicz's rewrite of rails -- more a rewrite of the entire idea itself.

SimpleMapper's basic assumption is that you have a ruby Class that you want to make Persistent. Keep your data wherever you want, all you have to do with SimpleMapper is specify how to communicate with it and in what data format. So there are only three parts to SimpleMapper apart from the Base class: Connection Adapters (communication), Format Adapters (data format), and Plugins (extra functionality).

Connection Adapters : Adapters are easy to create. My first adapter was the Http adapter so I could use it in place of ActiveResource.

Format Adapters: The data is filtered both ways through a format library of your choice - whether xml for http work or sql for a database adapter. Any other "format," even SQL, or encrypted text could be written into a format adapter.

Plugins: I now have 4 basic/default plugins. 1) 'properties' which helps to define properties and the identifier property of your objects, 2) a 'callbacks' plugin which manages callbacks for the http adapter and any other adapters that want to use callbacks (like oauth), 3) an 'oauth' plugin, which gives the SimpleMapper Http adapter oauth functionality, 4) 'association' -- experimental, to see how associations work, look at the associations_spec in the spec directory of the gem. I purposed to make them able to work across data sources, and dynamically too.

Usage

This example demonstrates using the simple_model and OAuth plugins. The session method is provided to emulate the controller's part in the game. OAuth needs to access the session -- it will run the session method on the object given to set_oauth, as you see in "Person.set_oauth(self)".

CONFIG = {
  :consumer_key => "lSwfFZ4n4MB7k457ZZMiGnsRIwL2O4j9oIGN42n3",
  :consumer_secret => "p64RuLYJox1C9nrhXGmUlC0h38ruS5GHVlVGsgNZ"
}

def session
  Hash.new {|h,k| h[k] = {}}
end

require 'simple_mapper'

class Person < SimpleMapper::Base
  # Use the properties plugin, so we can use properties parsed from the RESTful entity.
  has :properties

  uses :oauth

  # Load the parsing engine: JSON
  set_format :json

  # Load the connection adapter we want: HTTP
  add_connection_adapter(:default, :http) {
    # This block is run in the context of the adapter, so here we set up the adapter.
    set_base_url CONFIG['people_url']

    # Set up the OAuth within the adapter, where it belongs.
    requires_oauth(
      CONFIG['consumer_key'],
      CONFIG['consumer_secret'],
      { :site => base_url.to_s,
        :request_token_path => "/request_token",
        :access_token_path => "/access_token",
        :authorize_path => "/authorize",
        :auth_method => :query,
        :authorization_method => :scriptable,
        :session => lambda {session['people_oauth_session']}
      }
    )
  }

  # Now we can define the properties for the model.
  properties  :self,
              :first_name,
              :last_name,
              :created_by,
              :created_at,
              :updated_at,
              :birthdate,
              :phone_number,
              :email
  # And the identifier. The properties plugin assumes there are several properties, and one of them is the identifier.
  identifier :self
end

# Run the following line in the controller. If your OAuth is scriptable, i.e. it has keys that are already valid, it will simply set up the OAuth from the session. If you need authorization from the user, you'll need to define the "begin_pathway" method in your controller. (Should call 'redirect' for the user and store the state of things and where the user is heading, so when they get back you know what to do with them.)

Person.connection(:default).set_oauth(self)

# - - - In the console... - - -

>> @person = Person.new(:first_name => 'Bobby')
=> #<Person:0x1157740 @first_name="Bobby">
>> @person.save
=> #<Person:0x1157740 @first_name="Bobby", @self="http://localhost:3000/people/20">
>> @people = Person.get
=> [#<Person:0x110de9c @first_name="Daniel", @persisted=true, @self="http://localhost:3000/people/1">, #<Person:0x110dbcc @first_name="Jon", @persisted=true, @self="http://localhost:3000/people/2">, . . . ]
>> @people[-1].delete
=> nil
>> @people[2].first_name = 'Jo'
=> "Jo"
>> @people[2].save
=> #<Person:0x110d8fc @first_name="Jo", @persisted=true, @self="http://localhost:3000/people/16">
>> @people[2].delete
=> true
>> @people[2].save
=> #<Person:0x110d8fc @first_name="Jo", @persisted=true, @self="http://localhost:3000/people/21">

The Person.connection(:default).set_oauth(self) is, of course, only important for OAuth Authentication. Dig into the source for that method if you want to see what it actually does. It's just a macro to begin or complete the OAuth Authentication sequence, which includes redirecting to the provider site to allow the user to grant permission to his/her data. If you don't know anything about this sequence, I strongly recommend reading about and becoming familiar with OAuth.

Summary

In my usage so far, it's much easier to use than ActiveResource. It just does things right. It's lightweight, so I don't have to try to figure out what's going on. It doesn't rely on the bloated ActiveSupport package that messes with so many core classes. I can explicitly define how my model should be persistent, to the T, yet without too much configuration.

To install, just type:

gem install simplemapper

in your console.

ActiveResource OAuth plugin

2008-03-10T14:03:00.002-04:00

It's not the best, but it's working. I've found that ActiveResource is not a polished product. It's very flaky indeed. For one, it's picky about the way the XML looks, unless you write a new format parser for it, and the way those should work is not documented. Anyway, I won't rant about that. OAuth is why you're here. You need an ActiveResource client app to use OAuth to access your (or someone else's) API. No problem.

Before I go into specifics, let me explain a little something about OAuth. OAuth is meant to be user-centric authorization. A user authorizes you to access their data on another site (not necessarily ALL their data, but whatever part you are granted access to). Therefore, it is important that you be careful that if you use OAuth with an ActiveResource model, you have thought through how to keep each user's data safe. If you set oauth tokens straight into a model, they may be available to everyone using your app at the time. In other words, don't let Joe authorize your site to his data only to be followed by Sally browsing at the same time who now has access to Joe's data because the ARes model is now authorized to his data.

Let's get into the meat. Your model will need to include the configuration:

class Person < ActiveResource::Base
  requires_oauth(
    APP[:consumer_key],
    APP[:consumer_secret],
    { :site => connection.base_url.to_s,
      :request_token_path => "/request_token",
      :access_token_path => "/access_token",
      :authorize_path => "/authorize",
      :auth_method => :query, # http://term.ie/oauth/example/index.php doesn't like the :authorize header method.
      :authorization_method => :scriptable,
      :session => lambda {session['person_oauth_session']}
    }
  )
end

Then in your controller, you can do a couple different things:

@Person = Person.with_oauth(self)
people = @Person.find(:all)
# - - - - -
Person.with_oauth(self) do |person|
  people = person.find(:all)
  puts people.to_xml
end
# - - - - -
after_oauth_authenticates(Person) do |person|
  people = person.find(:all)
  render :xml => people.to_xml
end

Notice the differences: The first two I use primarily for what I call "Scriptable OAuth" -- in other terms it could be known as no-user-interaction OAuth. This is assuming that the consumer key and secret have been provided by the user himself from the site being accessed, and that consumer token is already tied to said user on the site. Using this method, authorization is automatically granted anytime. (Recommended only for automated scripts, and if you write the server code, make sure you let your user know that these are essentially keys to their data and should not be shared.)

The last one is notable: Use it to wrap all the code in a controller action that requires outside OAuth data. This will return a redirect to the authorize_url when it needs authorization, and it will run the block when it's already been authorized. You'll have to manage sending the user back to this action after they come back to your registered callback_url.

You can download the files at http://svn.behindlogic.com/public/ruby/lib/active_resource/plugins/.

# - - - - - - - - - - - - - - - - - - -

All that and I have to point you to a "new ActiveResource" -- SimpleMapper. SimpleMapper is cooler, lighter, better. Check it out!

Ruby's Code Blocks, Procs, Lambda, and Methods explained!

2008-02-28T09:58:00.008-05:00

You've seen code like these before:


before do |a,b|
  puts "something"
end
after {|a,b| puts "something else" }
think = {
  'about a' => proc {|this| "returns this text"},
  'a' => lambda {|is| "pretty much the same thing"}
}

I've had trouble many times understanding the contexts these will run in. I recently found a very good resource: http://en.wikibooks.org/wiki/Ruby_Programming/Syntax/Method_Calls.

Here are the core differences between code blocks, procs, and lambda's:
    A Code Block, on its own, is not usable. Think of it as a block of code that needs a runtime environment to be defined for it. Think of a person without a world to live in. A Code Block is the starting point which will be made into a Proc or a Method. When you pass a Code Block to a method, it is always automatically turned into a Proc object. Therefore, wherever you define a Proc/Lambda, it will hold on to the local context of where you defined it.

    A Proc is a Code Block that has been bound to a set of local variables. Once bound, the Code Block may be called in different contexts and still access those variables.
    A Method is a Code Block that has not been bound to a set of local variables like a Proc or lambda, but rather is bound to an object and has access to its instance variables.

    A Lambda is a Proc, being bound to local variables instead of an object, but is more like a method in the following respects: 1) when called, the arguments will actually be checked like a normal ruby method, and 2) calling "return" returns to its caller, while from a Proc, calling "return" will return from the code block the Proc was executed from. Generally for reasons of more intuitive behavior, lambda should be used by default, and Proc.new when its "looser" behavior is required. (These are all explained perhaps better with examples at the reference site above.)

One problem I've come across is that sometimes I don't want a proc/lambda to be bound when I define it, but rather later on when I run it. That is, I may want to define a proc/lambda in one context that will actually be executed in a different context or even in the context of an object. Following is the fix I've conjured for that:

class Proc
  # Changes the context of a proc so that 'self' is the klass_or_obj passed.
  def in_context(klass_or_obj)
    klass_or_obj.send(:eval, self.to_ruby)
  end
end

This allows you to "re-bind" the proc to a new context, the context of the current object ("self").

Furthermore, this allows for the following interesting core extensions:

class Object
  # Executes the block in the context of self, then returns self
  def extended(&block)
    block.in_context(self).call
    self
  end
end

module Kernel
  # Executes a block/proc in the context of the klass_or_obj passed, returning the return value of that block/proc.
  def as(klass_or_obj, &block)
    block.in_context(klass_or_obj).call
  end
end

"extended" allows you to run the code given it in the context of the object, but it returns the object when it's done, so you can safely use it while defining or assigning an object. "as" allows you to simply run a block of code in the context of the object given. Simple examples:

j = 'something'.extended do
  def self.backwards
    self.reverse
  end
end
=> "something"
j.backwards
=> "gnihtemos"
as(j) do
  self+' else'
end
=> "something else"

Happy methodology!

Creating Usable Quality

2008-02-18T16:25:00.004-05:00

From http://www.paulgraham.com/newthings.html:

I like to find (a) simple solutions (b) to overlooked problems (c) that actually need to be solved, and (d) deliver them as informally as possible, (e) starting with a very crude version 1, then (f) iterating rapidly.

This is a very important group of concepts. All respects to Paul Graham. I'd come very close to these principles myself, but hadn't actually narrowed them down yet. Let me outline that a little better:

Simple Solutions: Do one thing and do it well. I don't want a vacuum that also washes windows. I don't want a means of connecting and communicating with my friends to also include a lot of frivolous demands (facebook).
Overlooked Problems: These may be hard to find, unless, just like research paper topics, you narrow down enough.
Actually need to be solved: If it needs to be solved for you, it's likely it needs to be solved for others too. But if the problem is simple, make sure your solution is just as simple. Or simpler. Nobody wants a complex website doing a task a single dynamic page could do.
Deliver them as informally as possible: This one made me think - Why? One reason that may be critical for any idea is that the informal release (among your friends?) may let you know early on that your idea isn't marketable....
Starting with a very crude version 1: ....Or you may fix a critical function or focus before you get too many users turned off. Same idea as a beta period, but perhaps even less exciting.
Iterating Rapidly: Also known as "Agile Development" - rollout small updates every day. Develop with long-term goals in mind, but break those down into small pieces and achieve a small goal every day. Is that a Getting Things Done principle? Put in your monetization later.

Sliced my hand with an ice-skate...

2008-01-28T16:04:00.000-05:00

I was playing hockey yesterday and my left middle finger got run over by an ice-skate. It cut down to the tendon, which was pretty cool because I could see the tendon move back and forth when I moved my finger. It hardly bled, and didn't hurt at all. My finger was very much still alive and it wasn't numb anywhere. Off to the hospital I went, because it was deep and because it hit the tendon, it probably cut the tendon some. At the hospital, we waited a little (4 hours, 20 minutes), then walked right in to the "Trauma 1" room. I didn't feel at all like a trauma patient, but then again maybe they were just going to work on me in someone else's room. The nurse came in, she was nice and almost as happy as I, and the doctor said he'd have to call in the bone doctor to stitch the tendon. Another half hour went by where I was trying not to gross my sister out with my wonderful musings about fingers and bones and tendons and hospitals. My patient information sheet even had check-marks that said I had warm, dry and pink skin. Also alert; it didn't say anything about cheerful.

The "surgery" went well. Of course I watched as much as I could without being in the way. It's a strange feeling when part of your body is, while numbed, being pulled in different directions (a few times with the express purpose of rending flesh). It's even stranger to catch a whiff of raw flesh and realize that it's your own. And then it's pretty sweet to know that that much tearing and tying should end up healing -- just as good as new, I hope. After 7 hours out of my intended schedule, the lesson has been learned: keep your hands close should you fall with other people around.

ActiveResource and OAuth: we're working on it

2008-01-16T16:20:00.000-05:00

I struggled most of today to decide how I should represent OAuth-accessible data from another of my sites, in ruby. But then it became obvious: well of course -- ActiveResource! Now I don't like ActiveResource all that much, but it does work for what it's made for. But I poked around in ActiveResource's code to see how I could use OAuth along with it, and found it wouldn't be practical, at least without adding a wee little bit to ActiveResource: the concept of generated_headers. Why, wouldn't we sometimes need to have a header or two dynamically created on each request? So I submitted a ticket and patch to rails core to get this thing added.

Beyond this addition to ActiveResource, I'm thinking of writing a plugin that will extend ActiveResource to include a few key methods that will make OAuth easy.

- - - Update Jan 17: useful OAuth links - - -

1) A VERY simple everyday-language introduction to WHAT IS OAuth, followed by a step-by-step walkthrough of implementing it in Rails -- A MUST SEE for Rails people: http://www.slideshare.net/choonkeat/oauth4-and-oauth4r

2) RubyForge OAuth gem: http://rubyforge.org/projects/oauth/ and http://oauth.rubyforge.org/

3) OAuth4r -- an OAuth plugin for Rails, includes the same walkthrough (pdf) mentioned in #1: http://code.google.com/p/oauth4r/

4) This Google Project, (which includes OAuth Ruby code ...but not a gem? http://oauth.googlecode.com/svn/code/ruby/)

5) These google groups: OAuth, OAuth-Ruby
- This google groups thread might give you some info: http://groups.google.com/group/oauth/browse_thread/thread/cd6544d8f96ed34c

6) Very useful. Very. When writing a consumer for the first time, use this Sandbox OAuth Provider. When writing a provider, use this Sandbox OAuth Consumer.

7) A guide to using OAuthConsumer in a Mac application

Quickbooks Ruby Gem

2008-01-16T10:34:00.000-05:00

Quickbooks Rubyforge project page:
http://rubyforge.org/projects/quickbooks/
Quickbooks gem documentation:
http://quickbooks.rubyforge.org/

EZ Install:

gem install quickbooks

Quickbooks is a powerful small-business accounting software, probably the most popular (and most capable) out there affordable for small businesses. Quickbooks has an SDK, detailing a tight XML API accessible in a few different ways. Quickbooks XML (qbXML) is the request and response language that is used in all types of connections alike: Quickbooks Online Edition (QBOE) is accessible via a kind of home-brewed OAuth-flavored API. Quickbooks on your computer is accessible via OLE. Quickbooks on a different computer is accessible over the network via a SOAP interface. And lastly, Quickbooks provides a "connector" program that allows for a reversed request-response technique to allow quickbooks inside your company to connect to a website service equipped for working with Quickbooks.

That said, the point is that quickbooks is accessible and fairly friendly to programmers. They've done something right in opening this up. They need this capability in order to survive the next wave or two of software/internet trends: Openness, Seamless Integration, Accessibility.

After that long preamble, I should get right to the code! You all know ActiveRecord, DataMapper, and other ORM's? Let's access Quickbooks the same way! Consider the following code:

tom = Quickbooks::Customer.first(:first_name => 'Tom')
tom.phone = '(123) 456-7890'
tom.save

Ok, stop thinking that will be possible. The quickbooks gem, as of version 0.0.3 (recent release), can do that. Its inner workings model that of DataMapper predominantly, so things like .all, .first, and the concept of "dirty" and dirty_attributes are in full force. Currently it connects only via OLE, and the capabilities haven't been tested beyond the Customer model, but I'll be working toward that and the other connection options. But if you know qbXML, you can do whatever you want with the gem (using the OLE connection method). Requests and Responses are handled smartly with a Qbxml library included in the gem, so you can stop fretting about all this qbXML parsing and generating you'd otherwise have to do.

Version 0.0.4 will extract the OLE connectivity into an "adapter" of sorts, and then I will welcome any contributions of code toward other connection types, like SOAP or QBOE (since I will be focusing more in the models and workings within quickbooks). It will also better understand the required ordering of properties when it constructs qbXML requests.

I hope by this post to not only push a little "know that this exists!" into your ear, but also to gain contributors, as I'm guessing the demand for this gem will increase faster than I can make it stable and comprehensive. Thanks and as always, Happy coding!

A BLISSfully simple guide to OpenId on Merb

2007-12-28T10:01:00.002-05:00

OpenId can seem like a daunting task to implement when you first look at it -- the specification is so long (ok, well it needs to be, it covers a lot of "specifics"), it can be hard to test, there isn't a nice little "playground" to try out each step, etc. If you have one thing wrong, it can be difficult to find out what is wrong, but if everything's right, it just works. Hopefully this how-to will help you painlessly get it done right, in Merb. I followed this tutorial, for Rails, but converted several things to apply to Merb instead. With merb_datamapper-0.4.4 and datamapper-0.2.5 I had to fix a couple things, but merb_datamapper-0.4.5 is in working order.

You will need the following to get your app openid-enabled:
1) The ruby-openid gem
2) 2 Routes
3) Sessions
4) 1 Controller with 2 Actions
5) 1 View

1) ruby-openid
This step is pretty simple. In the root of your application, just run the command:

gem install ruby-openid -i gems

Run that, then include it as a dependency in config/dependencies.rb:

dependency 'ruby-openid'

2) 2 Routes
One route will receive a POST request (or get, if you prefer) from your login form, requiring only one field that (according to the openid specs) SHOULD be named 'openid_url'.
The other route will receive a redirect from the user's openid provider, with a bunch of query parameters.
Here are my routes -- I added in there a /login and /logout route: login will show a login form, and logout will remove the login information from the session.


r.to(:controller => 'openid') do |openid|
  openid.match("/login").to(:action => 'login').name(:login)
  openid.match('/start_login').to(:action => 'start_login').name(:start_login)
  openid.match('/complete_login').to(:action => 'complete_login').name(:complete_login)
  openid.match("/logout").to(:action => 'logout').name(:logout)
end

3) Sessions
Sessions must be turned on in Merb, if you want to remember that someone's logged in (who they're logged in as). Here's the easiest way to do it with DataMapper: Open up config/merb.yml, and find the "session_store:" line. Uncomment it, and make it say "session_store: datamapper". merb_datamapper makes it so simple, the next time you start up your application, if there is no 'sessions' table, merb_datamapper will create one, and you'll be ready to go.

4) 1 Controller and 2 Actions
You need 2 functional actions -- I recommend creating a new controller, but you can just as well put those actions into an already-existing controller if you wish.
Controller Action #1: This action will receive one parameter, 'openid_url', and from that, it is to 1) determine the user's openid endpoint server, 2) request a request token (shared secret) from that server, and 3) redirect the user to verify/authorize that request token. The ruby-openid gem takes care of all of the magic, all you have to do is tell it to start. Here is my (very minimal, not quite fault-tolerant) openid start_login action:


def start_login(openid_url)
  trust_root = 'http://localhost:4000'
  checkid_request = openid_consumer.begin(openid_url)
  redirect checkid_request.redirect_url(
      trust_root,
      trust_root + url(:complete_login) # using the named route from the above routes
  )
  # Pick up various failures via rescue here
end

As you can see, there are only two lines of action there: 1) openid_consumer.begin(openid_url), which does the discovery of the user's openid server and gets a request token, and 2) redirect. The checkid_request.redirect_url method generates the url with the proper parameters -- the root url of your website you wish to ask authorization for, and then the url you wish the user to be returned to.

Controller Action #2: This action will receive the user back from his/her openid server, with an answer to your authorization request. Its purpose is to check whether or not your login succeeded. I may not be doing this the best or most correct way, but it works:


def complete_login
  response = openid_consumer.complete(request.send(:query_params), 'http://'+request.host+request.path)
  if response.status.to_s == 'success'
    # The user is now logged in: Do what you will.
    session[:openid_url] = response.identity_url
    redirect(session.data.delete(:after_login) || '/')
   else
    # Failed: The openid_server returned unauthorized, or other error.
    # Be nice to the user and tell them something went wrong. (not in those words)
  end
end

Now of course you will want to fill in the failure code, and probably change what it does after the user is successfully logged in. As you can see I put in there to redirect to the url stored in session[:after_login]. I had set that value just before forcing the user to log in, saving the url the user was trying to access before they logged in.

One more thing in the controller: You've noticed I'm using a method "openid_consumer". That doesn't just come from nowhere, I have a protected method in the controller that sets up the OpenID::Consumer object for both methods to use:


protected
  def openid_consumer
    @openid_consumer ||= OpenID::Consumer.new(session, OpenID::Store::Filesystem.new("#{Merb.root}/tmp/openid"))
  end

(use MERB_ROOT instead for merb < 0.5.3)

And one more thing: you need to "require 'openid/store/filesystem'" at the top of the controller these actions are in, and you may need to create the tmp/openid directory.

5) 1 View
This one view is just the login form. You could include this anywhere on your page, or on every page of your app, if you like. Craft that session[:after_login] (saving the url) smartly so that the user can be returned to whatever page they started at before login.

Did I miss anything?

- - - update January 17 - - -
in merb_datamapper 1.4.5, use

:session_store: datamapper

instead of data_mapper with the underscore.

Installing do_mysql on Leopard (DataMapper mysql DataObjects adapter)

2007-12-27T11:58:00.001-05:00

After a little searching and putting a couple of things together, I got mysql working with Datamapper on Leopard. I'd imagine most people have already got theirs installed, but here's a suggested how-to for any who still have yet to figure it out. Just type each command one at a time as they're shown here.

sudo -s
ARCHFLAGS="-arch i386" gem install mysql -- --with-mysql-dir=/usr/local/mysql
exit
PATH=$PATH:/usr/local/mysql
sudo gem install do_mysql

OAuth is a key to the future: The longings of the average user and how they can be addressed

2007-12-21T11:50:00.000-05:00

My previous post talked a bit about the needs for web apps that interconnect seamlessly, yet securely and without requiring the user to constantly log in to this site and that. Eran Hammer-Lahav, a contributor to OAuth, provided his comment, and later a follow-up blogging that pretty rightly hits a point. Since then, I've realized that OAuth is indeed sufficient for these purposes. The key is in how an app decides when and to whom to allow access to a user's data.

What are the needs of the average user? Since I consider my self an average user (on my off days, that is), here's what I want:
1) Web apps that do one thing, and one thing alone.
2) In doing that one thing, they smartly bring in other relevant data when I need it. (Example: Google documents, sending a link to someone, pulls up my contacts from Gmail.)
3) I don't want to have to log in to each separate site just to get data from it. (This is partially achieved by OpenId -- what about ajax? What about allowing an app to log in behind-the-scenes to another app as me, using OpenId?)
4) I don't want to have to grant permission any more than is really necessary. I should be able to specify that my photos are allowed to be shared with any site as long as I am the one wanting to access them from that site. And that anyone else can access them through other sites as well, but they aren't allowed to modify them.

I want my documents to reside on Google Docs, my calendar on Google Calendar, my Photos on Flickr, my videos on YouTube or Google Video (why aren't they the same if Google bought YouTube??), and my todos on Tadalist. Each of those sites are specific-purpose-driven sites. They each do one thing. And for the most part, they do it well. But they aren't connected. I can't email a photo on flickr to friend straight from Gmail. I can't see my todos in Google Calendar. I can't choose someone from my gmail contacts on tadalist when I want to email a todo to a friend. You get the idea. Through a unified login (OpenId?) and a standard communication protocol (OAuth), Tadalist should be able to access my gmail contacts for me, in a fully transparent way.

OAuth, which I previously thought was insufficient for the task, may be just the thing for the task after all. The main thing is that if you use OAuth, please, please, make your permissions generic. Make it simple for the user, don't make them grant every single application that asks for data -- instead, provide a few options I can choose from once and forget about it. That way, most of the time, a user won't even have to see your site to log in or grant permission - they just get redirected to your site, to their openid, back to your site, and back to the original site with all the permission they need. (It works, but isn't it a little overkill?)

OpenId, OAuth, and more: A New Specification is needed!

2007-12-18T13:23:00.000-05:00

My next post partially recants some conclusions of this one. By all means read this, and when you're done, please continue over to the follow-up post. Thanks for being my guest!

The development and use of web applications is growing rapidly. Several tools, such as Ruby on Rails, have made it increasingly easier and more natural to use the web for purposes of tracking, managing, administering, creating, and getting things done. Using this market platform, the big companies (Google, Yahoo, Microsoft, etc) have been emerging with a success superior to the segmented community as a whole, because they have enough resources to create a number of quality web apps and connect them together in intuitive ways. We at Sabretech Consulting, LLC have determined that the key element that is needed for the community (as well as the big guys) to reach this emerging market platform effectively, is a standardized and secure way to connect apps to apps, for a user to use common identities and offloaded permissions management. OpenId and OAuth have taken steps toward this seamless and secure integration, but on the scale I am thinking of, they (and OAuth particularly) become inadequate. Let me explain:

OAuth allows one web app to communicate with another web app to access and modify a user's data. Though the protocol is good for a one-to-one purpose, it becomes inefficient at the next stage, where several applications are involved. With OAuth, every application involved must keep track of permissions for its own data in respect to every other connected application. This is where the inefficiency lies -- I in fact hold this to be a fault of philosophy: OAuth assumes that the hosting application is the owner and authorizer of the data it holds, where in fact it should be the User who owns and controls that data.

In keeping with the philosophy where the User controls their data, I propose that a new specification is made, similar in workings to both OpenId and OAuth, but separate and distinct: a specification for the decentralized control of permissions. Just as OpenId has achieved decentralization of authentication, we need a decentralized control of permissions. When this is used alongside OpenId, it would complete our thirst for privatization AND integration!

Imagine a group of web apps (I would call it an "app cloud"), each created by different developers, using these new standards. This is how I see it happening: A User could navigate to app1, and log in with his/her openid_url. The OpenId server verifies the user's Identity to app1. It also gives app1 a 'permission-to-act-as' token, which includes the openid_url and a permission key. However, app1 needs to use some data that is stored on app2. app1 can ask the user's Permission server for permission to use data from app2. To to so, app1 sends its 'permission-to-act-as' token to the User's Permission server, which then verifies the token with the OpenId server, and then grants permission via a 'permission-to-use' token, which includes the permission server url and a permission key. App1 sends this token along with its request for data to app2, which then needs to verify nothing but the 'permission-to-use' with the User's Permission server. Though not necessary, it would make good sense for the Permissions server to be one and the same as the OpenId server, so it would be good if they can co-exist at the same url.

Follow-up: Basically stated, perhaps a new specification is not needed. OAuth may be sufficient. My point, hopefully, is that there are needs that must be addressed, and should be done right. Read more about this in the next post.

Rails Plugin: is_searchable

2007-11-20T12:01:00.000-05:00

Announcing is_searchable, provides search functionality for models with ridiculously easy setup.

Here is a model that uses is_searchable:

class Person < ActiveRecord::Base
is_searchable :by_query => 'CONCAT(people.first_name, people.last_name) LIKE :like_query OR people.phone_number LIKE :like_query',
:and_filters => {
'age' => 'people.age = ?',
'over_age' => 'people.age > ?',
'under_age' => 'people.age < ?',
'first_name' => 'people.first_name = ?',
'last_name' => 'people.last_name = ?'
}
end

This code, in the simple form shown above, gives you two functions: Person.search and Person.search_count. Both of these should receive the same arguments (query, options), except that only Person.search will honor limit and offset. Here are some search examples:

Person.search('pa', :filters => {:over_age => 21, :under_age => 40}) #=> finds all middle-aged people with 'pa' in their name.
Person.search('tom') #=> finds all people named Tom or Tommy, etc.
Person.search('', :filters => {:under_age => 21}) #=> finds all people not yet old enough to drink (in the USA).
Person.search('', :limit => 10, :offset => 20) #=> finds the 3rd page of 10 people.

This plugin has been THE search functionality for all my models for a long time now, and it does the job well. It also plays well with my restful_api plugin (hmm, I haven't blogged about that one yet).

Install like any usual rails plugin:
ruby script/plugin install http://svn.behindlogic.com/public/rails/plugins/is_searchable/

And, if you have any questions or problems, please comment!

Unveiling... ez.tocontact.us!

2007-10-26T04:41:00.000-04:00

... a simple free ContactUs service! Point your form there, include the captcha; try it, put in where you want the email to go -- and you're all set!

http://ez.tocontact.us/ is the first of my Bliss services (Behind Logic Internet Simple Services). Have you ever needed a simple "contact us" page to go along with all the rest of your otherwise static content? Now you can have a contact-us page without programming ANYTHING. Simply point a form to submit to http://ez.tocontact.us/ and it will email the contents of the form for you. The first time you submit a form there, it'll take you to a page to put in the email address you (as the developer) want associated with the form you just submitted. After 1 day your email address will be activated (gotta wait the day, for spam reasons), and you can start submitting your form. Only one more thing is required: the captcha. Oh but this is really simple too. Just include the image at http://ez.tocontact.us/captcha.jpg, and a 'captcha_phrase' text field (<input type="text" name="captcha_phrase" />). When you submit the form, ez.tocontact.us will validate the captcha and send the email on its way.

More options can be included in these special form fields: on_success, on_failure, or on_complete (if you don't use the other two).

JSONP: you can submit your form with cross-domain AJAX, if you wish. Just do an ajax JSONP call (essentially it's including a new 'script' tag with a url and query string) to http://ez.tocontact.us/jsonp with all your form values, and the captcha text in the query string. Include a callback, on_success and on_failure, and it'll satisfy your every longing.

For a better explanation of how it all works SO SIMPLY, go to http://how.tocontact.us/.

OpenId by nickname

2007-10-15T12:32:00.000-04:00

By the way, while I'm on the topic of OpenId...

Why do I have to type in my whole OpenId url every time I log in using it?

I'd like to see some websites with a form that says, "Who are you?" I just want to say I'm so-and-so, and if I am, it should believe me. I want to type in my nickname for that site, like 'yanno'. I'm sure the site can remember my openid from when I set up my account and logged in for the first time. They can still remember how to authenticate me. It'd be cool if I could just tell the website who I am every time, instead of how to authenticate me.

Cross-Domain Ajax OpenID Authentication

2007-10-15T11:15:00.000-04:00

What is the biggest disadvantage of OpenId? The length of process.

You have to be redirected to your openid page, login, allow the site, and redirect back. If you've already allowed the site, at least you only have to be redirected twice, and it's transparent. But you still leave the site you were at and a page reload is necessary. With the rising tide of Ajax web applications, I want as few page reloads as possible.

My proposal is to extend OpenId standards, or at least create a service which supports, fully-ajax cross-domain authentication. How can this be done?

Currently the OpenId process is sorta like this (meaning, this is the general idea, I might be off on a couple specifics):
1) I type in my openid url in a site.
2) My openid server generates a secret and shares it with the site.
3) The site redirects me to my openid server for authentication.
4) I may need to log in, and/or "allow" my openid server to introduce me to the site.
5) My openid server shares with me (or actually my browser) the secret it shared with the site, and redirects me back to the site.
6) I'm back at the site and I've told it the same thing my openid server did, so it believes I'm who I say I am, because my openid server says so.

This could all be done with Ajax. BUT why isn't it done with Ajax?
Two reasons:
1) With Ajax, how do I know if I've been routed through and authenticated by my openid server?
2) If we don't leave the requesting site, it could access my openid authentication information.

Problem #1. Already been addressed in more security-minded applications like bank websites. What is it? Avatars. The showing of your avatar proves that you have indeed logged in with your openid site, because the openid site only provides it to an authenticated user-agent.

Problem #2. Technically that's still a problem. Javascript in one page can access information loaded into an iframe or another window it opens. (Of course we'd be okay if everyone had a little security key like Paypal provides for $1 on request, because with a changing password we can share the password with anybody as long as we don't share how the password changes with time.) Perhaps browsers should come up with a security model that allows Ajax on site A to load content from site B, with site B requesting a restricted environment so site A can't access it. But until then, can we possibly work with trust? On my OpenId server, if it supports this Ajax authentication, it should support it only on request, per site. Then I as an openid person can say "sure, let this site authenticate me without leaving the site!" Or if I don't trust them, I'll stick with the usual more secure redirect method.

In all, perhaps you can expect an OpenId server to come out in a couple months that supports cross-domain Ajax OpenId authentication. Register for an openid name on my site and we'll hope to have sites here and there popping up with the option for in-site authentication!