Pieces Of Rakesh

Downloading JavaScript Files in Parallel

2009-03-20T16:12:00.006+05:30

Steve Souders, creator of YSlow, author of the book High Performance Web Sites, is back in action writing a book titled Even Faster Websites. In it he details what one can do after the 14 YSlow rules (laws?) have been implemented, and you still want better better performance.

6 days ago, Steve gave a talk at SXSW Interactive, the slides of which are available on Slideshare. In it, he goes on to suggest (slide 27), that we should load scripts without blocking. To me, this seems like a very interesting idea, and based on his hints in the next few slides, I started experimenting.

Let me back up a little. When browsers download scripts, they block all other resources from downloading until such time that the script has finished downloading. When I mean all other resources, it includes even other CSS files and images. There are several ways to reduce the impact of serial script download, and the High Performance Websites book makes a couple of good suggestions.

Rule 1 - Make Fewer HTTP Requests
Rule 5 - Put Stylesheets at the Top
Rule 6 - Put Scripts at the Bottom
Rule 8 - Make JavaScript and CSS External
Rule 10 - Minify JavaScript

There are several others, mostly around caching strategies and reducing HTTP overhead, but these alone reduce the impact of serial script download significantly.

Now, if you haven't already taken care of these rules of optimization, my suggestion will be hard to implement, or will reap no benefits. You can stop reading now, and go back to your site to optimize your code.

Regular `<script>` tags:

See that waterfall pattern of file downloads? Nothing else is downloaded when a script is being downloaded, not even other scripts. The time gap between successive downloads is the time taken to evaluate the script, roughly proportional to the size of the script. Notice also that this takes in excess of 600ms even when the files are being served from my local machine, effectively having near-zero network latency.

With script parallelization

Now, this looks much better, doesn't it? Not only has everything downloaded in parallel, it has taken less than half the time for the same size of files. In fact, you will notice that some noise when recording this data made the files take longer to download in this case, yet the total time was less than half of the pervious case.

So, how did I do this? Read on...

One script tag to rule them all

Reading into Souders' suggested techniques (If you still have that slideshow open, go to slide 26), I decided that what I wanted was:

I want the browser busy indicators to show,
I want a script that doesn't care if I'm downloading from the same domain or not, and
I wanted to preserve the evaluation order in case of interdependent scripts.

So, I came up with the following little script:

<script type="text/javascript">
    (function() {
        var s = [
            "/javascripts/script1.js",
            "/javascripts/script2.js",
            "/javascripts/script3.js",
            "/javascripts/script4.js",
            "/javascripts/script5.js"
        ];

        for(var i=0, l=s.length; i<l; ++i) {
            var sc = "script", tp = "text/javascript";
            if(window.navigator.userAgent.indexOf("MSIE")!==-1 || window.navigator.userAgent.indexOf("WebKit")!==-1) {
                document.writeln("<" + sc + " type=\"" + tp + "\" src=\"" + s[i] + "\" defer></" + sc + ">");
            } else {
                var t=document.createElement(sc), sa = "setAttribute";
                t[sa]("src", s[i]);
                t[sa]("type", tp);
                document.getElementsByTagName("head")[0].appendChild(t);
            }
        }
    })();
</script>

It's a little ugly: There's a user-agent sniff happening in there and I've written it so as to use the least number of bytes possible (save for the indentation), but it works, and has no dependencies on any library. Just modify the array on top to place your list of scripts you wish to include, and let the rest of the script do the dirty work. If you need to include scripts from another domain, just use an absolute path.

This code was written keeping in mind that this should be the only piece of on-page inline JavaScript. This can then go on and fetch the script required in parallel, and those in turn can unobtrusively do their thing.

This works as advertised in IE and FF. I haven't tested the parallelization in other browsers, but at least it fails gracefully and preserves script execution order.

Find this interesting? Follow me on twitter at @rakesh314 for more.

JsMemcached.js - A Jaxer client for memcached

2009-01-21T19:20:00.003+05:30

Two days ago, on a lazy evening, I started work on my first solid Jaxer project — building a memcached client. As a testimony to the elegance of JavaScript, it took me less than 2 hours to get a basic memcached client up and running.

What is memcached, you ask? From the memcached website:

memcached is a high-performance, distributed memory object caching system, generic in nature, but intended for use in speeding up dynamic web applications by alleviating database load.

What it does is, lets you use excess RAM on your server machines as temporary stores of data, so that you can avoid going to your DB when you want frequently accessed data. Needless to say, accessing data from RAM is much faster. Since the creation of memcached for LiveJournal.com, memcached has become the cheapest and fastest way to scale a website.

Now, my implementation is not feature complete, nor at peak performance, but I aim to take it there. Even so, I decided to get my code out and available for everyone to see, so that people can tell me what they think of it.

You can go to the Google code page for the JsMemcached-client project, or download the code right away. Though limited in features and with a possible bug in an edge case in the get call, you can start using this already.

This is my first open-source project, so please don't be too harsh in your feedback.

Help needed: I need to figure out how to performance-optimize this code. Either I need to get rid of that loop in the get call, or I need to write a wrapper around libmemcached. I would prefer the latter, since I have to add more features for feature-completeness. Any idea on how I can get Jaxer to talk with libmemcached (or any other such out-of-process thing)?

Function declaration vs. function assignment in JavaScript

2008-12-29T19:51:00.004+05:30

I'm surprised how much I learn about JavaScript every single day.

It all started with a post that's wrongly titled to indicate a bug in Internet Explorer. The comments are an interesting read, where people go ahead to suggest that this "bug" exists in all browsers except Firefox. Then, they discover that it's actually a bug in Firefox, and not the other way round. Then a bug report is filed with Firefox. Brendan Eich comes along and resolves it as an invalid bug, with a brief explanation.

To explain, here's what's happening:

Ned Batchelder (the poster of that blog post) came across this piece of code that worked differently in IE and Firefox.

function really() { alert("Original"); }
if (0) {
    alert("No");
    function really() { alert("Yes, really"); }
}
really();

The explanation is pretty simple. A function really is defined which alerts "Original". Inside an if block which should never get called, the really function is redefined to alert "Yes, really". Since the code block is inside an if that should never get called, the assumption would be that the function shouldn't get redefined. This is how Firefox behaves. However, not so with IE, which redefines the function. Weirdly, the alert("No"); is never executed, even though the function gets redefined.

Turns out, IE is not the only one doing this. All webkit browsers and Opera also exhibit this behavior. Kris Kowal was the first to suggest that this is a bug in Firefox for the right reasons. So, a bug report was created and BE closed it. Here's his explanation:

ECMA-262 Edition does not specify functions in sub-statements, only at top level in another function or a program (see the normative grammar). Therefore this bug is invalid as summarized.

Moreover, we have extended ES3 (as allowed by its chapter 16) for ~ten years to support "function statements" whose bindings depend on control flow. So this again is invalid. There's a bug to dup against, if someone finds it feel free to change resolution accordingly.

/be

For those who didn't understand, here's a simpler version of what BE said.

function really() { ... } is a function declaration.
Function declarations can only exist at the top level, or inside other functions. They cannot be inside the if statement.
If the browser finds that it is inside the if block, the browser is not allowed to throw an error. Instead, it should handle it gracefully.
There's no spec that defines how such situations should be handled, so browsers are free to do what they want to.
All browsers except Firefox first scan the code at load time to find function declarations. When doing so, they look at the function first, and then it's redefinition. In this case, the original function should be discarded, and the new function should be used.
Firefox does this at runtime, evaluating the functions when the code execution sees it. Hence Firefox doesn't redefine the function at all.

So, BE's explanation that this is not a bug is valid. This is a gray area in the spec, and the browser is free to implement it the way it wants.

Also, and more interestingly, BE brings up function statements which do not follow this rule. A function statement would look like this:

var really = function() { ... };

Now, the spec very clearly says that function statements should be evaluated at runtime based on control flow - stuff like that if statement. Ned wouldn't have run into this problem if he was using function statements.

I learn new things about JavaScript every day!

HTTP cache optimization

2008-11-21T22:22:00.004+05:30

You know what else Twitter is good for? Quotations. As good friend and colleague, Piyush Ranjan said:

Recession is [a] good time for innovation. Best innovation happens in crunch situations. Otherwise people just throw money at the problem!

At Cleartrip, as with any other company, we're always looking for ways to reduce our costs. Now with the travel industry in India hit hard, and the general economic slowdown, it's very interesting for us techies to innovate in ways that will help us cut costs.

One the recent things we've been working on, and the stuff I want to talk about here, is the area of bandwidth savings. The benefits of bandwidth savings are usually twofold. Firstly and obviously, it reduces our bills. Secondly, it usually manifests itself as a better experience for the user, since he doesn't have to wait as long for stuff to transfer from our servers to his browser every time he moves between pages.

Now, our site is a fairly dynamic site. Almost every page in the site (well, the significant ones at least), are so dynamic, they cannot even be cached for a couple of minutes. So, caching HTML doesn't suit us. What we can cache is everything else - images, CSS and JavaScripts. Also, since we try to adhere to web standards as much as possible while being as Web 2.0 as possible (whatever that means), our use of CSS and JavaScripts is pretty heavy.

Phase 1

So, a couple of months ago, we made our first set of optimizations. We noticed that our images change very rarely - some of them hadn't changed since we first made them. There was no point sending down copies of the images to our users frequently. These could be cached pretty aggressively. We decided on an arbitrary time of 1 month for the caching of images.

Next came the JavaScript and the CSS. This posed a considerable problem. We keep doing JS and CSS changes very frequently on our site - as frequently as a couple of times a day. We couldn't afford to have them aggressively cached at the client end. We needed the agility to update our users' cache with the new versions of our code. Again, arbitrarily, we decided that the cache time for JS and CSS files would be 2 hours. That way, it wouldn't be in users' cache very aggressively, while remaining in his cache for about the length of his usage session. At the same time, it would take at most 2 hours for our changes to propagate to all users.

So, with this configuration change rolled out (it's not at all hard to do if you understand caches well) we sat back and monitored our savings. Turns out, the savings were pretty amazing. Hrush, our founder, even blogged about how our data center providers were surprised to the extent that they said that it was "just not possible" that we have such low bandwidth consumption for the traffic we get!

The problems

How ironic our data center guys said that, because just around that time, we were looking at how we can optimize bandwidth usage even more. And why did we decide to optimize further?

Because you can
Some of our files never changed at all - core JS libraries, for example. Why bother downloading them again after 2 hours?
Because this setup was hard to work with. Remember I said we patch stuff to production very frequently? That it takes two hours to propagate meant that if it was a bug fix, it might be broken for two hours even after we've put up the patch.
Because it gets very hard to make changes that have dependencies with other resources. For example, if the JS needs certain CSS changes, and certain markup changes, in what order do you put the changes up on the server?

This usually meant that we had to plan in advance and we would still have some backwards-compatibility cruft in our JS to manage this 2 hour transition. Sometimes it would take an entire day to make even a minor patch since it had to be batched with breaks of two hours. One quick-fix would be to rename the file and all references to it, but it's easy to see how painful that is to do on a file-by-file basis, not to mention that we'd run out of file names pretty soon!

Phase 2

One thing for sure, though. If we could somehow manage the file name issue, we can solve all the problems I listed out. That's because every time we made a patch with a different file name, it would instantly be available to all users, irrespective of their cache status. This gives rise to another interesting possibility: we didn't have to restrict ourselves to the 2 hour window anymore. We could potentially increase our cache expires time to 20 years for all that matters. That should give us MUCH more saving than we get currently, and our users will download as little as necessary.

Except, we didn't know yet how to solve the file name problem gracefully. Ideally, there shouldn't be a human being deciding which files have changed, what the file names are, and going all over the place and changing them. That would make it tedious and error prone, not to mention boring. Then of course you have to think about how these file name modifications would work with source control. You don't want to mess up your clean source code management history.

The solution to the source control mess thing was rather easy. We fake the file name. Here's what we do. We take a file name and append a random number to it. This will make the client believe that the file name has changed, and will negotiate with the server for the file. Meanwhile, at the server, we could have a rewrite rule that transforms the file name to something that maps to a real file on our server. Sounds simple enough. Tried it, and it worked like a charm.

Now, to crack the real problem - how do we generate these numbers in a sensible way. The number essentially had to be such that it would never repeat (at least for any given file), it would be global in that two pages shouldn't be using different numbers for the same resource, and when the number changes it should instantly reflect site wide. Now that we knew how the number should behave, the quest was on to come up with a mechanism to generate these numbers.

The solution

After a lot of thinking, we had a shameful duh moment, and it suddenly all made sense. We didn't need to invent these numbers at all. We just needed to use our source-control revision numbers! The revision numbers match all the characteristics of the number we want. Why bother with complex systems to generate and track numbers when it was already available, even if very disguised.

I'll save you the implementation details about how we made this available site wide, and how we made it possible to have instant global changes to this number. That definitely wasn't the tough part, and I'm sure you'll figure out the details. Who knows, maybe Piyush might just release a plugin for Rails to do it automatically for you. However what surprises me is that it's very hard to find such gems of knowledge on the net. I'm now beginning to think that maybe this design pattern should be used for distribution of all static resources on the web. We're definitely not the first to invent this pattern - why is no one else talking about it?

We've only just rolled this out on cleartrip.com and are still to get a decent sample of data to see how it has impacted our bandwidth consumption. But any fool could guess that our bills should reduce significantly with this change.

Joel on Project Managers

2008-11-12T14:48:00.003+05:30

This quote is long, in typical Joel On Software style. He said this in his latest podcast over at Stack Overflow with Jeff Atwood. I've ripped it off the transcript wiki. Definitely worth a read.

Here's my feeling about project managers:
One of the things that is interesting is that project managers, traditionally, are brought on because you have a team of yahoos - and this is just as true in construction, or in building an oil rig, or in any kind of project as it is in the making of anything - making a new car at general motors, or designing the new Boeing 787 dream liner - as it is in the software industry. Project managers are brought in because management says: "Hey, you yahoos! You're just working and working and working and never get the thing done and nobody knows how long it's going to take." If you don't know how long something's going to take and you can't control that a little bit then this really sucks from a business perspective. I mean; if you think of a typical business project - you invest some money and then you make some money back. The money you make back - the return on investment - might be double the amount of money you invest and then it's a good investment. But if the investment doubles because it took you twice as long to do this thing as you thought it would then you've lost all your profit on the thing. So this is bad for businesses to make decisions in the face of poor information about how long the project is going to take and so keeping a project on track and on schedule is really important.
It's so important that they started hiring people to do this and they said: "OK, you're the project manager - make sure that we're on track." These project managers were just bright college kids with spreadsheets and Microsoft project and clipboards. They pretty much had to go around with no authority what so ever and walk around the project and talk to the people and find out where things were up to and they spent all their time creating and maintaining these gigantic gantt charts - which everybody else ignored. So the gantt charts, and the Microsoft project, and all those project schedules, and all that kind of stuff, was an artifact created by a kind of low level person. Although it might be accurate depending on how good that low level person was, but it was still an output only thing from the current project: Where are we up to? What have we done? How much time have we spent? What's left? Who is working on what?
Then, for some reason, these relatively low level people, who were not actually domain experts, (if they were at Boeing they don't know anything about designing planes, if they were on the software team they're not programmers - they're project managers, and they don't know anything about writing code) they start getting blame when things went wrong and they started clamoring for more responsibility, more authority to actually make changes and to actually influence things and say: "Hey, Joe's taking too long here - we should get Mary to do this task, she's not busy." The truth is that they started getting frustrated because they were low level secretarial-like members of their teams and they wanted to move their profession up the scale so they created the project management institute - or whatever it's called - and they created this thing called... ah, I don't even know! But they created a whole professional way to learn to be a professional project manager and they decided to try to make it something a little bit fancier than just the kid with the clipboard that has to maintain these gantt charts all day long. You can tell this is what happened because the first thing project managers will tell you about their profession is that the most important thing is that they have the authority to actually change things and that they are the ones that actually have all the skills that can get a project back on track, or to keep a project on track, and therefore they need to have the authority to exercise these skills otherwise they'll never get anything done, they'll never be able to keep the project on track, they don't just want to be stenographers writing things down.
The trouble is, they don't actually have the domain skills - that's why they are project managers. If you are working on a software project you know how to bring it in on time and you've got to cut features, and you know which features to cut, becuase you understand software intrinsically and you know what things are slow and what things are fast and where you might be able to combine two features into one feature, where you might be able to take a shortcut. That's the stuff a good developer knows, that's not the stuff a project manager knows. In a construction project it's the architects and the head contractors who know where shortcuts can be taken and how to bring a project in on time not the project manager. The project managers don't have any of the right skills to affect the project and so they inevitably get really frustrated and everybody treats them like secretaries, or treats them like 'annoying boy with clipboard', when they really don't have a leadership role in the project - and they're not going to be able to because they don't have the domain expertise. No matter how much they learn about project management, no matter how many books they read, or how many certificates they get, no matter how long they've been doing project management: if they don't know about software, and software development, if they don't have that experience, they are always going to be second class citizens and they're never going to be able to fix a broken project.

Understanding eval scope. Spoiler: It's unreliable!

2008-10-08T21:46:00.010+05:30

Today, I ran some tests to help me understand the scope in which an eval runs. Turns out, like so many things in the browser world, it's very unpredictable and exhibit different behaviors in different browsers.

Let's start with the following snippet of code. I've added comments to demarcate areas in the code, which I will be changing with each iteration.


var foo = 123;
var bar = {
    changeFoo: function() {
        // We'll keep changing the following snippet
        alert(this);
        eval("var foo = 456");
        // Changing snippet ends
    }
};

bar.changeFoo();
alert(foo);

A little explanation of the code above. foo is a variable in the global scope, and it's value is set to 123. An object bar is created with a single method changeFoo which does an eval. The eval creates a local variable (thanks to the var) foo, and sets it's value to 456. bar.changeFoo is called, and the value of the global foo is alerted.

The aim is to test the scope in which eval runs. If eval is in the global scope, the global variable foo should change it's value. If eval is in the local scope, the global foo should be unaffected. Then there are various things we can do inside the changeFoo method which should keep altering the scope of this, so we are also alerting this to see what happens.

The findings are listed below:



alert(this);eval("var foo=456");
alert(this);window.eval("var foo=456");
alert(this);this.eval("var foo=456");
alert(this);eval("var foo=456", window);
(function() {    alert(this);    eval("var foo=456");})();
(function() {    alert(this);    window.eval("var foo=456");})();
with(window) {    alert(this);    eval("var foo=456");}
with(window) {    alert(this);    window.eval("var foo=456");}

	Changed snippet	Internet Explorer	Safari 3.x	Firefox	Google Chrome	Safari Nightlies
1	`alert(this); eval("var foo=456");`	123	object	123	object	123	object	123	object	123	object
2	`alert(this); window.eval("var foo=456");`	123	object	123	object	456	object	123	object	456	object
3	`alert(this); this.eval("var foo=456");`	error	object	error	object	error	object	error	object	error	object
4	`alert(this); eval("var foo=456", window);`	123	object	123	object	456	object	123	object	123	object
5	`(function() { alert(this); eval("var foo=456"); })();`	123	object	123	window	123	window	123	object	123	window
6	`(function() { alert(this); window.eval("var foo=456"); })();`	123	object	123	window	456	window	123	object	456	window
7	`with(window) { alert(this); eval("var foo=456"); }`	456	object	456	object	456	object	456	object	456	object
8	`with(window) { alert(this); window.eval("var foo=456"); }`	456	object	456	object	456	object	456	object	456	object

What I think of these results:

I don't know what Firefox is doing in case 2, and for some reason Safari Nightlies seem to be following it. Maybe it's just beyond my understanding, but case 2 is not supposed to be different from case 1. Why does case 2 operate in global scope? If window.eval is different from eval, case 3 shouldn't all have given errors. Someone please help me understand that $hit.
Case 4 makes sense, but that's a non-standard behavior in Firefox. Understandable that no one else exhibits it.
IE baffles me in case 5, and Chrome seems to ape it. In this scenario, the anonymous function is supposed to have the global scope - so, in this case, this should point to the window. WTF is happening here!
Consistent with case 2 above, Firefox and Safari Nightlies continue to display weird behavior in case 6. For some reason, in these two cases, the eval operates in the global scope.
Now, I have no idea why, but only cases 8 and 9 seem to really work at all. This is despite Doug Crockford going on and on about not using with constructs. It's also despite being beyond (my) understanding about why the with should make any difference to the eval, since eval is part of the window object.

All in all, if you are going to be evaling JavaScript (not JSON), and you want the eval'd code to run in the global scope, you should use the with block around the JavaScript snippet. Or else, you can lose a lot of hair handling cross-browser issues.

Hope you don't lose as much hair as me.

Housekeeping

2008-09-17T23:28:00.002+05:30

I've done a couple of minor changes to this blog's UI. Nothing dramatic, but I thought I should draw your attention to it, especially if you are one reading only from my Atom/RSS feed.

The first change came from realizing that my posts are longer than they should be. Somehow, I can't seem to compress them beyond what I already do. Long pages are a pain the wrong spots to read, so I decided to expand Douglas Bowman's original layout to a fluid width page, so that my posts consume lesser vertical space. (I just noticed that Steve Yegge has done similar fixes - I'm flattered.)

Secondly, and sort of to compensate for my lowered rate of posting on this blog, I've included two new feeds you can subscribe to on this blog. You can find latest updates to these feeds in the right sidebar. First is my feed from del.icio.us, which happens to be my bookmarking service of preference. Secondly, is my list of shared RSS feeds from my Google Reader. Both of these feeds get updated more frequently than my blog itself, so you might find these interesting.

This way, not only do you keep in touch with what I'm writing, you can also keep in touch with the stuff I'm reading. Expect to find some tech humor in these feeds too ;). Here's the rule of the thumb: The two new feeds reflect what I'm thinking strongly about, but I do not necessarily have an opinion about. My blog is my list of opinions. That's the difference.

I've been testing these thingies on my blog for some time now, so if you've been around here recently, you might have already noticed these changes. I just thought I'd wait a bit before I can announce these new things - turns out Google is doing a reasonable job of ensuring these things work, after all. ;)

JavaScript for Linux Hackers

2008-09-03T14:25:00.004+05:30

Two weekends ago (24 August), I gave a talk about JavaScript at the Indian GNU/Linux Users Group of Mumbai (ILUG-BOM) - A small gathering of Linux hackers from around the city. The talk was held at the Homi Bhabha Centre for Science Education (HBSCE), TIFR, Mumbai.

It was very weird having to explain JavaScript to kernel hackers and sysadmins. It entails a different approach - one where you have to assume that the audience knows a lot already, probably more than you in some respects. They are not one to get wowed by browser effects and visual fanciness. Also I know very little about Linux systems, so we had very little in common. It's very challenging preparing for such an audience.

I spoke about the language, it's history, it's expressiveness, the type system, variable casting, objects, marshaling objects, it's lambda nature, and several language constructs, especially functions. What I didn't cover was things like the DOM, inheritance patterns and constructor functions, but there has to be something for next time, right? ;)

I think it went pretty well. Well enough for them to invite me for another session where we could cover the left-out topics. I'm sorry - I would have put my slides on slideshare or something, but honestly, when I was running through my slideshow the day before the presentation, I nearly dozed off. So I decided that I'd just do the presentation without the slides - just me talking, and a JavaScript console on the screen.

element.hasFocus? document.activeElement!

2008-08-29T20:48:00.005+05:30

Today at work, in some code I was writing, I wanted to know if a given input box has focus or not. Turned out, this is surprisingly difficult. The input element doesn't have any hasFocus or similar property! No wonder working with the DOM keeps tripping people up!

Turns out, from quite some time now, Internet Exploder has been supporting a proprietary property - document.activeElement - which tells you which element is current focussed. Exactly what I needed. Except this is proprietary - I tried in Firefox 2, and it didn't work as expected. However, fortunately, turns out the HTML 5 spec has now incorporated this new property of the document object, and all future browsers should support it. Firefox 3 already supports document.activeElement. I've read that Opera supports it, but haven't tried. Safari does not, but the latest nightlies support it as well. Of course IE6 and IE7 support it perfectly well - IE invented it after all. So, of the big browsers, only Firefox 2 and Safari are problematic.

Since my code was not super critical, I've decided to skip support for just this bit for Firefox 2 and Safari. In any case, I'm hoping (against hope?) that both these browsers have a faster upgrade cycle than others, so they'll be outdated pretty soon.

Just in case you were thinking that the focused element can easily be 'discovered' by using the onblur and onfocus events, think again. Firstly, according to the specs, the focus and blur events don't bubble, so you can't use event delegation to capture all focus/blur events on the document. In any case, if you could use event delegation, putting these event handlers in an external script would mean that the script will kick in after a little delay - either after the script has loaded if the script tag is placed at the bottom, or after the page load happens if you are waiting for the DOM to be ready before you can use it, which in my case wasn't acceptable. The only other solution is to have a script include or a inline script tag that declares a function before the DOM is loaded, and then have inline onblur and onfocus event handlers DOM Level 0 style. That's just plain ugly. None of these solutions are either workable or graceful.

Brendan Eich on code translation to JavaScript

2008-08-13T21:28:00.003+05:30

I've always hated the idea of some server-side language (like ASP.Net, GWT, RoR, what-have-you) generating JS code to run on the client. I'm glad at least Eich agrees with me, as is published in this interview. (Jump to page 3 for this excerpt.)

I did not intend JS to be a "target" language for compilers such as Google Web Toolkit (GWT) or (before GWT) HaXe and similar such code generators, which take a different source language and produce JS as the "object" or "target" executable language.
The code generator approach uses JS as a "safe" mid-level intermediate language between a high-level source language written on the server side, and the optimized C or C++ code in the browser that implements JS. This stresses different performance paths in the JS engine code, and potentially causes people to push for features in the Ecma standard that are not appropriate for most human coders.
JS code generation by compilers and runtimes that use a different source language does seem to be "working", in the sense that JS performance is good enough and getting better, and everyone wants to maximize "reach" by targeting JS in the browser. But most JS is hand-coded, and I expect it will remain so for a long time.

That said, I think Eich doesn't highlight some of the other problems with server-generated JavaScript: ability to debug, potentially unoptimized output, and potentially inefficient code. I've worked with server-generated JavaScript in ASP.Net and RoR, and I know what a pain it can be.

IE Congratulates Firefox

2008-06-20T14:59:00.004+05:30

... on shipping Firefox 3.0.

The comments from around the web are hilarious:

Don't eat it.
Is it not poisoned?
That cake will give you CSS rendering errors, in your colon.
Moments later the cake hit a knife/plate standard it wasn't compliant with.
It must have been hard for them to put "love" on the cake
As you eat away the icing, you'll see that the cake is blue... little by little, it's becoming clear... it's the blue screen of death!
Congratulations on shitting the IE7 Team
...did it come with an End User License Agreement...?
There should be some bugs inside the cake.
...it's probably half baked too!
I guess that box was rendered in quirks mode.
Sheesh, even their cake has box model bugs.
I was wondering what the IE team had been doing for the past 3 years...
There's actually an IE team?
[Apple] snuck their cake through during the last Quicktime update.
And if you didn't like the candles, you had to replace the whole cake.
...that the cake tasted pretty good, but as they started to dig in, they sadly realized that instead of a whole cake, they had actually gotten a thin layer of cake on top of a cake-shaped support structure made of toothpicks and glue.
Turns out they forgot to add sugar to the cake. That will be added on Patch Tuesday.
Good luck getting the recipe for that cake.

Simply use HTML. Not XHTML.

2008-06-10T19:24:00.003+05:30

The good 'ol argument about HTML vs. XHTML seems to have resurfaced on the Internets. I have been firmly in the XHTML camp at one time, but I had made a mention in a previous post about why I think that there was no point in fighting this battle anymore. There is a clear winner - HTML.

Of the lot of recent posts out there about which is better than the other, this little thought experiment echoes my sentiments about XHTML vs. HTML.

You pore through the raw source code of the page and find what you think is the problem, but it’s not in your content. In fact, it’s in an auto-generated part of the page that you have no control over. What happened was, someone linked to you, and when they linked to you they sent a trackback with some illegal characters (illegal for you, not for them, since they declare a different character set than you do). But your publishing tool had a bug, and it automatically inserted their illegal characters into your carefully and validly authored page, and now all hell has broken loose.

The emails are really pouring in now. You desperately jump to your administration page to delete the offending trackback, but oh no! The administration page itself tries to display the trackbacks you’ve received, and you get an XML processing error. The same bug that was preventing your readers from reading your published page is now preventing you from fixing it!

The fact is today's web is one where content might pour in from various locations, many of which you might not have control over. It is important to inter-operate with these kinds of content sources. Expecting strictness from an external source is not only overkill, it's folly to do so.

I had faced this problem when I had worked on the Sacramento Kings website. The content was coming from various sources, some even as trustworthy (for them at least) as the NBA. However, content encoding and ill formed markup issues were huge enough to get the JavaScript all crazy. I can't even imagine the amount of problems I'd have faced if we'd have decided to use a XHTML strict, or even transitional, doctype for this job. How can we force a content author to ensure that his content validates, and that the reference validation for your site and the reference validation for the content author is the same?

Simply use HTML. Let the onus of making sense of the content lie with the browser. It's not a human's job to make content appealing to a computer. If a computer cannot understand it, it should work hard. Not the human.

Just for the humor, check out this page that a friend happened to hit when pulling up W3.org the other day. I know I'm being harsh when I say this, but the guys who made the standard can't seem to respect it.

Goa

2008-06-03T15:48:00.004+05:30

Just got back from a trip to Goa with colleagues. Here are some of the pics:

Head over to Flickr for all the pics.

Getting iSync to work with the Nokia N91 8GB

2008-04-21T15:50:00.002+05:30

My phone is a Nokia N91 8GB, and my primary OS these days is the Mac OSX 10.4.11. When I followed instructions I could find anywhere on the web - the simple connect, launch, sync instructions - it simply didn't work. iSync kept complaining that it couldn't sync with my device.

That just meant it's time to get under the hood and do some hacking! If you ever face this problem, here's how you solve this issue.

Quit iSync if you already have it open.
Go to your applications folder, right-click on the iSync application icon, and select "Show Package Contents".
Keep clicking through to the following path: Contents/PlugIns/ApplePhoneConduit.syncdevice/Contents/Plugins/Nokia-N91.phoneplugin/Contents/Resources.
Once there, duplicate the MetaClasses.plist file just to have a backup. You can do that by right-clicking the file, and selecting "Duplicate".
Now, open the file with a text editor - I used TextWrangler, and search for the line that says <key>com.nokia.n91</key>. A little lower, you should find a <key>Identification</key>. Right below that should be a <dict> XML node.

Edit it to look as follows:

<dict>
    <key>com.apple.usb.vendorid-modelid</key>
    <string>0x0421/0x042F</string>
    <key>com.apple.gmi+gmm</key>
    <array>
        <string>Nokia+Nokia N91-1</string>
        <string>Nokia+Nokia N91-2</string>
    </array>
</dict>

That's it - you are set. Now, launch iSync and it should now be able to work with your N91 8GB as advertised.

The key here is that the N91 8GB model is identified as Nokia+Nokia N91-2, which isn't available in the supported devices list by default - the regular N91 is identified as Nokia+Nokia N91-1 or simply Nokia+Nokia N91, which is supported. The hack just adds support for the N91 8GB by telling iSync to treat it as the Nokia+Nokia N91-1 for all practical purposes.

Web 2.0 Professionals

2008-02-28T14:08:00.002+05:30

Got this job offer in my mail today:

COMPANY: [Company Name] ...with a reputation of being a leading supplier of networking equipment and network management for the Internet.

LOCATION: Bangalore

JOB OVERVIEW: We are looking for Senior Web.20 professionals on a very Urgent basis. Person with good relavant experience is most preferable.

EXPERIENCE: Min. 4 yrs to 12 yrs

POSITION: Permanent

If you a are looking for a change then plz fwd your updated word formatted (*.doc) CV to me ASAP with following details:

[snip]

5. Experience in Web 2.0

[snip]

Typos and poor language apart, can someone please explain what a Web 2.0 Professional is? What does he do? How in the world is he supposed to have 4 to 12 years experience? And how can he give details of his Experience in Web 2.0? Not to mention, I am very curious about how Web 2.0 is relevant to a leading supplier of networking equipment and network management for the Internet.

I guess this just shows that we are at the peak of the hype.

Some where in a board room:
How do we boost sales of our NICs and routers?
What we really need is some Web 2.0 action going for us.
What is that?
Don't you know? Everyone's talking about it. It's the greatest thing to have happend!
Ok. Get HR to find a Web 2.0 Professional. Make sure he has a decent amount of experience.

On X-UA-Compatible

2008-01-29T13:01:00.000+05:30

There's been so much said about this...

Chris Wilson spells out the need for it.
Aaron Gustafson explains the details on A List Apart (!).
Eric Meyer welcomes it.
So does Jonathan Snook.
Anne Van Kesteren thinks it's an IE lock-in all over again.
Mozilla devs have questions.
PPK observes quitely.
WaSP guys clarify that they were not necessarily involved, though MS claims so.
Jeremy Keith thinks MS's got it backwards.
Andy Budd thinks IE just shot itself in the foot.
Jeffery Zeldman defends the idea anyway.
John Resig thinks the trouble is worthless.
Dean Edwards sits on the fence and snickers.
Roger Johansson is not convinced that it's a good idea.
Rachel Andrew thinks this is a step backwards.
Safari says they are not going to implement it.
Ian Hickson thinks MS is going to be in trouble.
Mike Davies says that it's the end of the line for IE.

... that if I write an opinion piece, it will go unnoticed.

So, what do I think anyway? If this is MS's only chance at fixing the web, I love the idea. However, this is a drastic step, and MS cannot botch this up. If they do, no one will want to work for their browser anymore. If no sites are written for their browser, users won't use their browser anymore. As a front-end developer, having to cater to three different browser types (IE6, IE7 and good browsers) with HUGE differences between them, is already a pain in the wrong spots. Adding one more to the mix will only worsen the situation. But if IE8 starts actually behaving like the good browsers, we can finally hope that all our problems will vanish with IE6 and IE7 - whenever that happens.

So, if MS thinks that this is the solution to all their problems, so be it. The world will comply this one last time. This is a lot of trouble. It better be worth it. If this ends up having a less-than-desirable result, MS is doomed. IE is doomed. And the web will be a better place anyway, IE or otherwise.

Bill G Has Left The Building

2008-01-24T21:27:00.000+05:30

A Movie About Coffee

2007-12-11T15:34:00.000+05:30

How To Build A Read/Write JavaScript API

2007-11-20T01:30:00.000+05:30

I've learnt most of this primarily by reading through Google Calendar's JavaScript Client Library code. I've also picked up clues from lots of other material around the Internet. There are also some minor improvements I've added.

So, here's the use-case: You probably already have a REST (or similar) API for server to server communication. Having a JavaScript API would be a great idea (after all, JavaScript is the most deployed programming language available on almost every platform in the form of a browser runtime). This poses many problems. Most significantly, browsers are very strict about the same origin policy. You are aware of certain hacks out there to use JavaScript across domains, but at best they give you read access or rely on browser plugins. You can do writes using query string parameters, but you know that that's just plain wrong.

Whatever be your solution to this problem, you want to play within the browser's security model, and not depend on any browser-specific security loopholes. Another very important thing you want to achieve is to ensure that your API users do not have to do any setup at their end – be it in terms of installing a server-side proxy, or jumping through hoops of any other kind. If a setup is unavoidable, it should be very simple to do, requiring little or no effort. You might also add additional requirements of user authentication (after all, you are letting them do writes), preferably at your domain – OpenID style – and have access to your cookies even when your application is being used from another domain entirely.

People might point out solutions like CrossSafe and Subspace. From what I gather of both these ideas, their goal is to secure your site from any third-party script snippet. That is not a necessary goal in our case. Also, both these techniques rely very heavily on some form of setup at the API consumer's end (which aren't very easy to do either – may even be impossible for say shared hosting environments), which we don't want to have. The technique I'm suggesting here is very similar in it's operation to both Subspace and CrossSafe, but eliminates (or reduces drastically) the need for any setup at the user's end.

The JSONRequest specification also needs mentioning. Unfortunately, the spec itself is rather new. Needless to say, there's no native working implementation of it as of this writing. CrossSafe comes rather close as an implementation, but it's not complete. (To make matters worse, completing the implementation will require even more server-side co-operation at the API consumer's end.) That said, I don't know why Doug Crockford has decided to keep PUT and DELETE methods out of the spec, among others. I guess it might be for simplicity. However, I think in today's RESTful days not having those methods supported is not a good idea. If Crockford's spec ever becomes the standard, I will be a little unhappy that the additional methods are not supported. The API creation technique I'm mentioning here supports all the HTTP methods that the browser supports for HTML forms (which is only GET and POST for all major browsers to the best of my knowledge), but at least it's a browser limitation – not one imposed by this technique.¹

So, let's get started. Here's what you require to get cross-domain read write JavaScript APIs to work.

The "setup" required at the client's end is that he should have at least one static cacheable resource embedded in the page where he's consuming the API, which is loaded from the same domain as his page. This could be in the form of a static CSS file, or an image. If the page doesn't have either, it will be required to insert one – maybe in the form of a 1px image hidden away by using inline style attributes. This is usually not too much to ask for, considering that pages are either made up of spacer GIFs or CSS documents, usually loaded from within the same domain. The static resources I mentioned could even be from a different sub-domain within the same domain, but it might complicate scripts slightly to have it set up that way. If this setup is not possible at all (oh, come on!), you could still find a work around², but I think that this is the easiest way to get things up and running.
You will need to do some setup at your end, if you are the creator of the API. In particular, you will need to setup a "proxy" page that intercepts the requests from the JavaScript client API, conditions the data, and passes it along to the REST API. This proxy page also reads the response from the REST API, conditions the data to suit the client, and flushes it down to the JavaScript.

Now, let's go over the process of actually orchestrating the communication.

The API client library is included on the page by means of a script tag pointing to your domain (your domain being the host of the client library). This is similar to including the Google Maps API on the page.
Once included, the script scans the page for the static resource mentioned above. This is done by walking the DOM looking for link or img tags, and checking the value of the href/src attribute to ensure it lies within the same domain as the calling page. The URL of this resource is stored for use later. At this point, if required, the client library can signal to the developer that it is ready for communication with the server. If the resource is not found, the client-library should throw an error and terminate.
When a request requires to be made, the client library takes the request parameters and prepares the markup for a form. This form can have any method attribute value, and should have it's action attribute set to the proxy page on your domain. The parameters to be sent to the server should be enumerated as hidden fields within the form. The client library also specifies the resource (in a RESTful sense) that needs to be acted upon. Also, the name of the static resource we had hunted down earlier is passed on to the server. This form is not appended to the document yet. This markup is then wrapped into <html> and <body> tags. The body tag should have onload=”document.forms[0].submit();”.
The client library then creates a 0px x 0px iframe, without setting the src attribute, and appends it to the page's DOM. This makes the browser think that the iframe exists in the same domain as the calling page. Then, by using the iframe document object's open(), write() and close() methods the markup created in the previous step is dumped into the iframe. As soon as the close method is called, the form gets submitted to the proxy page on your domain because of the onload in the body tag. Also note that this gives the server access to any cookies it might have created from within it's domain, letting you do things like authentication. In this way one part of the communication is complete, and the data has been sent to the server across domains. However, the iframe's document.domain has now switched to point to your domain. The browser's security model now prevents any script access to most parts of the iframe.
The proxy page sitting on your server now queries your REST API – basically doing it's thing – and gets the response. Response in hand, the proxy is now ready to flush the response to the client.
If the response is rather large in size, as might be the case with a huge GET call for instance, the proxy breaks it up into chunks of not more than say 1.5k characters³.
The proxy is now ready to flush the response. The response consists of iframes – one iframe for each of these 1.5k chunks. The iframe's src attribute is set to the static resource we had discovered earlier. It is for exactly this purpose that we had hunted the resource down and passed on the URL to the server. At the end of each of these URLs, the proxy appends one of the chunks of the response, after a “#” symbol, so that it works as a URL fragment identifier. Also, the iframe tags are each given a name attribute, so that the client script can locate them.
Meanwhile, the client-side code is where it had left off at the end of step 4 above. The script then starts polling the iframe it created to check for the existance of child iframes. This check of iframes will need to based on the iframe name the server will be sending down. It will look something like this: window.frames[0].frames[“grandChildIframeName”]. Since the static resource we have loaded into the grandchild iframe is of the same domain as the parent page, the parent page now has access to it, even the intermediate iframe is of a different domain.
The client script now reads the src attributes of the iframe, isolates the URL fragments (iframe.location.hash), and reassembles the data. This data would typically be some JSON string. This JSON can then be eval'd and passed on to a success handler. This completes the down-stream communication from the server to the client, again across domains.
With the entire process complete, the client-library can now perform some cleanup actions, and destroy the child iframe it created. Though leaving the iframe around is not a problem, it is not necessary and simply adds to junk lying around in the DOM. It's best to get rid of it.

This was simply the outline of the process, and there are several additions/improvements that can be done. For example, better control on reading/writing HTTP headers, having a reliable readyState value, error handling in case of HTTP errors (4xx, 5xx errors), handling of HTTP timeouts, etc. are all desired. However, this should be enough to get you started.

If you haven't already realized the significance of this, we should now be able to build much more sophisticated mashups that do much more than the current breed of mashups on the web. It opens up the floodgates to entirely new kind of applications on the Internet – applications we haven't seen as yet.

Let's enable better mashups! Nothing should now stop you from being able to give open secure access to your site's functionality in JavaScript.

A little creative thinking will let you circumvent the problem of browser-restricted HTTP methods when querying your REST API. Send an extra parameter to the proxy page when you are creating the form to specify which method to use. Let the proxy page then hit your REST API with the specified method.
The work around to not having any same-domain static resource would be to ask the API user to have a blank HTML page on his domain, the URL for which should be manually provided by the user to the client script. I don't think this is a great idea since it is an extra step that the API user has to do. However this can be used for one of those if-all-else-fails situations.
This 1.5k restriction is to overcome a URL length restriction in Internet Explorer, though most other browsers allow much more. Note, HTTP itself does not impose any restriction on the URL length.

Pics from Kerala

2007-11-09T18:57:00.000+05:30

Finally managed to get some time today to upload my pics from my trip to Kerala to Flickr. Here are some of my favorites from that list.

Head over to Flickr to see the complete photo set.

Dojo, Flash, SWFObject, Packer, IE's "Click here to activate and use this control" and Eval Contexts

2007-10-18T21:30:00.000+05:30

Boy, that's a long title!

Solving this problem I had at hand took a good day and a half. What I discovered seems obvious in hindsight, but it wasn't obvious to me until I had solved it. So that no one ever goes through the pain, here's what happened.

There is this thing I'm working on at work that required the use of SWFObject class simply to get around the IE "Click here to activate and use this control" nuisance when inserting flash files into the page.

Now, the reason why that message comes up is pretty shitty, but the problem and solution is very well documented. SWFObject is supposed to solve this problem. So, rather than re-inventing the proverbial wheel, we decided to use SWFObject. However, needless to say, IE would still keep throwing the message.

Now, a bit about the setup we have. We are using Dojo for this particular thing. I have written a wrapper around SWFObject so that I don't have to bother passing it all the stuff it needs every time I need to use it. Also, on a test machine, we host built versions of the code, which used Packer to compress the JavaScript. We were using Packer more as a compresser than as an obfuscator, since Packer gave us a considerable improvement in the compression ratios over anything else. On my local machine, I use the unbuilt version of the code.

Irrespective of where the code ran from, we used to get the nasty message in IE.

For those who don't know yet, here's how you get around the message - it's pretty simple. Create the object/embed tag from an external JavaScript file, and insert it into the HTML document. It's that simple.

Now, going by unobtrusive principles, our JavaScript always lies in external files (unless there's very good reason to not do so). So, why was this message showing up? There were two scenarios here:

My dev machine: Because of the way I had layered the files in Dojo for ease of maintenance, Dojo would dynamically pull in the references on the fly. This is very convenient and makes for easy to organize code. The way Dojo does this is by making an XHR call to the server to fetch the .js file, and then evaling the code. Now, these evals are run in the context of the window, effectively making them inline scripts. IE thus thought the code was inline, and showed the error message.
The test machine: So, even if it doesn't run on my machine, that's fine if we can get it to run on a test machine correctly. On this test machine we always house the built version of the Dojo file. With a properly written build profile, Dojo might not have to go around and pull in references as required since they would already be baked into the built files. So, that would make the SWFObject code external, and IE shouldn't have a problem with it, right? Almost. Packer requires an eval of the code it has obfuscated to make sense of it. Again, the eval is run in the context of the browser, and IE treats the resultant code as inline, again showing the error.

So, in both the cases, the problem was the context of the eval, though in entirely different scenarios. It could also be concluded that SWFObject cannot ever be compressed using Packer. It was just painful to track down and fix this bug. I hope no one ever has to go through this pain again.

If you need to know, we had to switch to ShrinkSafe to avoid the eval that Packer does. There are definitely better ways to compress, but this should do for now. The code-size increase wasn't too significant, so we were fine with using ShrinkSafe. Ideally, I would use ShrinkSafe for SWFObject and packer for everything else, giving the best of compression and avoiding the implicit eval. This doesn't solve the problem on my local dev machine (since the code is unbuilt), but that doesn't really matter.

Client-Side Performance Optimization of Ajax Applications

2007-10-16T11:52:00.000+05:30

There has a lot been said about server-side performance optimization. But a recent report from Yahoo concluded that the server accounted for less than 5% of the time for a user to view a web page. Here's how you can performance optimize the client-side code. Note that this article is targeted towards pretty advanced JavaScript programmers, working with pretty client-heavy applications.

Network performance

This feels just stupid to say: Reduce the amount of source-code you've written. There are several ways of doing this. The first is to simply not write any JavaScript at all. But that might not be an option for you. Another way is to lazy load code - don't download code unnecessarily on the client. This is especially true of single page applications. Another thing you simply must do is pass your code through a good compressor - like Dojo's Shrinksafe, Crockford's JSMin, Edwards' Packer or the YUI Compressor (or even a combination of those).
Another thing that I've heard most people recommend is that gzipping of JavaScript files helps reduce network latency. While this is entirely true, a bug in a rather prevelent version of IE makes me wonder if I should do that. If anyone can prove me wrong, I'll only be glad.
In all popular browsers today, JavaScript files are always sequentially downloaded. It doesn't matter if HTTP says you can have 2 connections open. It doesn't matter that CDNs on different domains can serve files in parallel. It doesn't matter that Firefox always disregards the HTTP restriction and downloads multiple files all the time. When it comes to JavaScript, all files are always downloaded sequentially.
This only makes sense, since script should be executed in the order in which they appeared in the markup. Reduce the number of files to be downloaded. This can be done by using a build process, something similar to what the Dojo build system does - combining the JavaScript files into one file.
Cache Ajax responses. The network is undoubtedly the weakest link in the chain. Avoid making requests. A request once made should never be made again. Implement cache invalidation if you need. Even then don't fetch data just because the cache is now invalid. Wait till it is actually required. If you never end up needing it, you've saved another hit.

Perceived performance

Reduce the amount of source-code you've written. I know this sounds like a repeat of a point above, but I had to bring this up from a perception point of view too. It seems that the more JavaScript that's downloaded, the more time it requires for the browser to interpret it, increasing at an exponential rate (not linear). Which means that even after your code has been downloaded, the browser will just sit there doing (apparently) nothing for some time. Usually, this is a problem above say 500 kb of code.
In any case, if you are downloading 500k of JavaScript on the load of the page, there better be a very good reason for it. You should be able to have much faster download times by splitting up these files into modules, which you can download at a later time - maybe on demand.
Get something downloaded and displayed as soon as possible. This might be something as simple as markup with the UI skeleton for the application, and simple "Loading..." indicators. It helps a great deal in reducing the frustration in working with an application.
If you can help it, put your JavaScript includes at the bottom of the page. This gives the browser enough time to download and render most of the page before even starting to mess with your scripts. Considering that JavaScript downloads sequentially, and doesn't let any other resource be downloaded at that time, you should only download JavaScript once you already have something to show the user.

JavaScript performance

There's a lot that can be said here. I've started getting a lot of kicks lately in trying to milk every millisecond of performance from a browser. So here's what I've learnt so far.

If you are writing a function that returns an array, you usually want to pass in a callback as a parameter of that function. This will improve the performance by at least a 100%.

Instead of:


    var anArrayOfData = getAnArrayOfData();
    for(var i=0; i<anArrayOfData.length; i++) {
        // do something with anArrayOfData[i]
    }

Do the following:


    getAnArrayOfData(function(item) {
        // do something with item
    });

This is better because you usually loop inside the function anyway to build the array. Having to loop through the returned array again is a waste of processing time.

Instead of:


    function getAnArrayOfData() {
        var returnData = [];
  
        for(var i=0; i<largeSetOfData.length; i++) {
            // Some code...
            if(condition === true) {
                returnData.push(largeSetOfData[i]);
            }
        }
  
        return returnData;
    }

Do:


    function getAnArrayOfData(callback) {
        var returnData = [];
        
        for(var i=0; i<largeSetOfData.length; i++) {
            // Some code...
            if(condition === true) {
                returnData.push(largeSetOfData[i]);
                if(callback) {
                    callback(largeSetOfData[i]);
                }
            }
        }
  
        return returnData;
    }

This way, the callback parameter is optional, and you still return the returnData, but you could also provide the callback function and avoid another external loop to iterate through the return data. I've changed all the getElementsBySelector methods in my libraries to use this approach, for example. It only seems logical - if I get an array, I will usually need to iterate through it.

Use native functionality whenever possible. Case in point: forEach iterators. This is very helpful, and part of the 1.6 standard, but the most popular browser in the world can't do forEach loops. Most people either live with it, write their own forEach iterator using simple for statements, or use a library that already has this built in. If you are the second type of person, you aren't achieving much except code readability, which is not a bad thing. Most frameworks' forEach loops also take much the same approach. However, there's a better way.
```
    function myForEach(array, callback) {
        if(array.forEach) {
            array.forEach(callback);
        } else {
            for(var i=0; i<array.length; i++) {
                callback(array[i]);
            }
        }
    }
```
In almost all browsers, the block in the if statement will be executed, giving the best possible performance, since you are using native functionality. However in less modern browsers that happen to be more popular and which you have to support but can't do much more about optimization, the else block will still work. Think of it as being the graceful degradation principles of CSS being applied to performance.
Now, I haven't done it above, but I strongly recommend that you stick to the JavaScript standards when deciding the function signatures of both the myForEach and the callback functions. This is because if the world does become a better place one day, and the most popular browser in the world actually learns how to be modern, you code will work such that it uses the most optimum features in the browser, without you having to change a single line of code.
Don't build too much of scaffolding code to make JavaScript behave like classical object oriented programming languages. Usually, you will not end up with much more than helper functions. A lot of the paradigms of classical OO don't apply to JavaScript. Learn to use JavaScript for what it can do. Don't make it what you want it to be.
Use threads. Ok, JavaScript doesn't really do threads at all. However, you can kinda simulate the effect of threads. What you achieve by doing this is that you hand over control back to the browser for a brief instant before proceeding with your code. This gives the browser time to react to any other user action that might have happened, make any updates to the DOM that you had asked for, bypass that nasty The script on this page is unresponsive warning, etc. So, how do you do this?
```
    // some code
    setTimeout(function() {
        // some more code
    }, 0);
```
If you don't understand how exactly this works, this could be a source for a lot of bugs. Use this with caution. However, I've used to get a very high apparent performance very successfully.
Cache function results. If for a given set of parameters a function will always return the same results, you really only need to calculate it the first time. Once calculated, save the data in a variable, and read from that variable hence forth. For example:
```
    var squaresCache = {};
    function getSquare(number) {
        if(!squaresCache[number]) {
            squaresCache[number] = number * number;
        }
 
        return squaresCache[number];
    }
```
The example above isn't very good for at least two reasons. Firstly, using this pattern for computing squares is just plain stupid. Secondly, it seems (though it need not be) that squaresCache is a global variable, which is plain evil in any programming language. However, I hope it illustrates the idea of populating the cache the first time the function executes and subsequently reading from the cache instead of re-calculating the data.
Strings in JavaScript, like in many other languages are immutable. So, for lots of string concatenation operations, you need to use the string builder pattern in JavaScript too. The simplest way to do that is to declare an array instead of a string, push strings into that array instead of concatenating, and finally call array.join(""); to get the concatenated string.
Do not use eval. Eval is painfully slow. But why do you need to use eval? Other than converting a JSON string to an object, I never write any code that needs to be eval'ed. Remember the other cousins of eval - new Function(someString) and setTimeout(someString, ms). You don't need the Function constructor at all, and you don't need to pass strings into setTimeout at all. Instead, in both cases, you can use anonymous functions. Thus, the implicit eval is avoided. Using anonymous functions give the added benefit of retaining variable scope through the creation of the closure. The eval is always carried out in the global scope.

Unfold your if statements. This is particularly useful for code that checks for browser features. For example, instead of:


    function addEvent(element, eventName, callback) {
        if(element.addEventListner) {
            // add the event one way
        } else {
            // add the event another way
        }
    }

Do the following:


    if(element.addEventListner) {
        function(element, eventName, callback) {
            // add the event one way
        }
    } else {
        function(element, eventName, callback) {
            // add the event another way
        }
    }

This unfolding of ifs applies even to loops, for example. So, if you can keep if statements outside a loop, do that. It doesn't make for readable code, but it's significantly faster. Bonus points to you if you just thought to yourself that my forEach example can be improved using this technique.

DOM Performance

Of all the parts making up client script, DOM manipulations are the slowest. So, you have to take the most care here.

Use innerHTML. Don't be too much of a purist. Being a purist won't make your application faster. You wouldn't believe how much faster your code is if you use innerHTML.
Never update the DOM. Ok, if that's not possible, at least do it as infrequently as possible. Bunch up your updates to the DOM and save them for a later time. Realize that it is not the size of the update but the high frequency of updates that's slow. Doing appendChild in a loop is updating the DOM frequently. Caching the markup in a string, and then setting the innerHTML in the end is batching and updating infrequently. The second is much faster.
However, the technique above is mostly only useful if you are adding new stuff to the DOM. What if you are updating existing elements on the DOM? How do you keep updates to a minimum when you want to change style, class names, content and children of a node that already exists? Simple. Clone the node you want to work with. Now you will be working with a clone of the real node, and the cloned node doesn't exist in the DOM. Updating the cloned node doesn't affect the DOM. When you are done with your manipulations, replace the original node with the cloned node. However, note that the performance problems here are because of the content and rendering reflow that the browser has to do. You might get similar benefits by simply hiding the element first, making the changes, and then showing it. Though I haven't tried this, it should work in theory.
Keep track of events. For me, this is the worst part of working with the DOM. This is important because when your application (or any DOM nodes) are being unloaded or destroyed, you will have to manually unregister the events from the nodes BEFORE you destroy the elements. Yes, this is the garbage collector's job, and that's supposed to be the job of the environment your code runs in, but guess which browser is the offender here. Internet Explorer doesn't free all the references even when the user leaves your web page. Unless you want your web app to earn the reputation of being responsible for many a crashed browser, and a horrid browsing experience for other websites too, count your references.
If you are going to iterate through a node list to attach event handlers, you are probably wasting processor time. Instead, simply attach the event handler to some parent of the node list and read from the event object to know what was clicked on. You save the cycles required to iterate over the nodes this way.
Avoid calls to functions like getElementsBySelector, where there's lot of DOM walking involved. If you cannot, then make sure you work on as small an area of the DOM as possible. If your favourite version of getElementsBySelector lets you send in a root node under which to search, do that. Otherwise, provide a very high specificity, starting with a "#someId" so that the function can narrow down the search. Also, understand how these functions work internally. For example, you could use a getElementsByClassName to find divs with the class "foo", and the implementation of getElementsByClassName will probably be just three lines, However, using getElementsBySelector("div.foo") will be faster in almost all frameworks, even though it might have a hundred lines of code in it's implementation, since it has less DOM walking to do.

Sorry for the kinda horrible organization of this post. I should also say that not all of these ideas are my original - I found a lot of these by reading many sites scattered across the web. However, I hope you found this post useful.

Much Deserved Update

2007-09-21T10:55:00.000+05:30

It's been an eternity since I wrote to this blog, so an update is in order. I fear this update will seem very similar to a pervious post. Also, please bear with me if this post seems too long.

Much in the vein of that previous post, I've quit my job again. Iventa was a great place to work at (Notice, I didn't say that about my employer before Iventa), and it has let me learn, explore and part with (and thus expand) my knowledge more than any other place I've worked at before.
I'm now with Cleartrip.com. One of the top travel portals today in India, Cleartrip could easily be mistaken to be just a travel thing out there. Truth is, outside of the IITs - an organization I used to interact with regularly - this is one place where I've seen more innovations happen more frequently than any other place. All the people you meet at Cleartrip are gurus in their areas of work. I can only consider myself to be lucky to be in the company of such a congregation of sheer genius.
I've joined Cleartrip as a JavaScript developer guy. If you look at that from a organizational chart perspective, this is a slight demotion in rank for me. But that was completely intentional. I was starting to move towards managing people at Iventa, and I think I had still not had enough of playing with technology. I guess that only means that I want to grow some more in the area of technology, and not in the area of management. Tech is where I get my kicks from, and I don't see that changing at least for some time.
The management at Cleartrip was kind enough to allow me a short vacation between jobs. I took the opportunity to go to Kerala - this time not because it's my home town, but because I had never been to a place of such scenic beauty as a tourist. I've got very interesting stories (including one where I'm chased by a wild elephant trying to protect her kid from me, since I was armed with a camera), but I've got even more interesting photographs. I shall be uploading them soon. Which brings me on nicely to the next two points.
The reason I've not uploaded any pics so far is because my beloved Hangy is dead. Again. This time, it seems that the moisture from the Mumbai rains got to her chipset on the motherboard, and AFAIK there's no one who can fix that. So, my only option is to replace the board, but I'm reconsidering that in favor of a huge upgrade. I've meanwhile asked my computer-fixit-guy to come up with an alternative - probably a second-hand motherboard I can buy from him. Let's see what he comes up with.
So, assuming I get ready to post pics online to share, which service is really the better? I like Google, so Picassa seems to be an interesting option, but I've not found that to be reason enough to change from Flickr yet. Any opinions for or against either of these? Please enlighten me.

This post would be incomplete without some more important information.

This blog is NOT dead. I really mean to post much more. In fact, since Hangy's dead, I got my MacBook Pro from work home today so that I can compose this post. (Which incidentally makes this the first post on this blog written on a Mac. I wouldn't be surprised if the next one is written in Emacs on Ubuntu, or something!)
That said, this blog is going to deviate a bit in its direction. So far this blog has largely been about markup and style. However, IMHO, all that had to be said about markup and style has been said, either here or elsewhere. The subject is so beaten now, that a new acronym had to be invented in hopes of reviving interest. (POSH, for God's sake!) Meanwhile, I've decided to move on. I need to give a better explanation of why I'm moving on, and I think that's a topic for another post. I really want to share my thoughts about why I think all the fuss about markup and style is not worth it - at least in the connotations it started to carry. That would be an ironic post, since the most popular post on my blog so far has vehemently promoted XHTML! I guess I had to go through all of it to learn how unimportant it is.
I've moved on to JavaScript, if you haven't guessed already from my last few posts. I've been hungrily learning so much about JavaScript these days, that I can easily claim to know much more than some of the good JS coders out there. I know that's a tall claim, but on more than one occasion I've realized that some of these gurus' arguments don't make sense. I guess that's a good sign.
Which nicely brings me on to the last point I want to share. This blog is going to turn from a primarily web standards promotion blog to a primarily JavaScript hacking blog. I shall now discuss lessons I've learned in JavaScript, application architectures from a JavaScript thick-client perspective, interesting hacks and tricks in JavaScript that I've picked up along the way, and occasionally the stupid browser nuances (you know which browser I'm talking about) I come across. Expect posts talking about JavaScript as a language (one of the most modern - old as it is - and among the richest most expressive languages in the entire computer industry IMHO, and already available on your desktop!) for all the cool things it can do, and JavaScript from a more serious application design point of view, being an important part of application architecture design sitting at the top-most layer, ensuring that users have a great experience. This particular topic about JavaScript being a serious thick-client programming language has been of particular interest to me lately, and the few forays I've had are very promising. So, expect me to talk a lot about these kinds of topics now on.

URI fragment identifiers and HTTP

2007-06-18T16:48:00.000+05:30

I came across an interesting problem today with how HTTP handles URI fragment identifiers. Here's the spoiler - it does not!

Here's the long story. If you are on a page which has a URL that looks like this:
http://www.domain.com/page.html#fragment

Now, #fragment is known as the URI fragment identifier. This particular thing has got very popular lately with it's potential uses in Ajax applications, since it can be easily (mis)used to let client-side code perform many actions.

This also helps take care of some back-button problems with Ajax. For example, let's say you are on page.html. Then, you perform some client-side operation that appends #fragment to the URL. Now, let's say you navigate to page2.html and press the back button, you would land up on page.html#fragment. Then, the JavaScript on the page could read the #fragment and perform an Ajax action to restore the page to the state you had left it at.

However, I just discovered when working on this one project that the #fragment is never sent to the server as part of the request headers. The HTTP specification says nothing about handling URI fragment identifiers, and sure enough most browsers do nothing with them.

So, if you ever have to read the #fragment on a server, remember that you can't! That's just one more of the problems that we have to deal with as RIA developers.

Safari for Windows

2007-06-12T20:59:00.000+05:30

Yes, that's right. It has finally happened. Safari for Windows is now available as a free download.

There are still some questions unanswered - my number one question is if it uses the same rendering engine as the Mac version. I hope it does.

On a side note, I wonder what will happen to Swift.

	Changed snippet	Internet Explorer		Safari 3.x		Firefox		Google Chrome		Safari Nightlies
		foo	this	foo	this	foo	this	foo	this	foo	this
1	`alert(this); eval("var foo=456");`	123	object	123	object	123	object	123	object	123	object
2	`alert(this); window.eval("var foo=456");`	123	object	123	object	456	object	123	object	456	object
3	`alert(this); this.eval("var foo=456");`	error	object	error	object	error	object	error	object	error	object
4	`alert(this); eval("var foo=456", window);`	123	object	123	object	456	object	123	object	123	object
5	`(function() { alert(this); eval("var foo=456"); })();`	123	object	123	window	123	window	123	object	123	window
6	`(function() { alert(this); window.eval("var foo=456"); })();`	123	object	123	window	456	window	123	object	456	window
7	`with(window) { alert(this); eval("var foo=456"); }`	456	object	456	object	456	object	456	object	456	object
8	`with(window) { alert(this); window.eval("var foo=456"); }`	456	object	456	object	456	object	456	object	456	object

Pieces Of Rakesh

Downloading JavaScript Files in Parallel

Regular <script> tags:

With script parallelization

One script tag to rule them all

JsMemcached.js - A Jaxer client for memcached

Function declaration vs. function assignment in JavaScript

HTTP cache optimization

Phase 1

The problems

Phase 2

The solution

Joel on Project Managers

Understanding eval scope. Spoiler: It's unreliable!

Housekeeping

JavaScript for Linux Hackers

element.hasFocus? document.activeElement!

Brendan Eich on code translation to JavaScript

IE Congratulates Firefox

Simply use HTML. Not XHTML.

Goa

Getting iSync to work with the Nokia N91 8GB

Web 2.0 Professionals

On X-UA-Compatible

Bill G Has Left The Building

A Movie About Coffee

How To Build A Read/Write JavaScript API

Pics from Kerala

Dojo, Flash, SWFObject, Packer, IE's "Click here to activate and use this control" and Eval Contexts

Client-Side Performance Optimization of Ajax Applications

Network performance

Perceived performance

JavaScript performance

DOM Performance

Much Deserved Update

URI fragment identifiers and HTTP

Safari for Windows

Regular `<script>` tags: