IUnknown.QueryRandomThoughts()

Should you move on?

2009-03-29T22:09:00.000+05:30

In the current economic gloom, it's probably preposterous to even broach the subject of leaving a job but I was wondering if there was a methodical system to decide if the time had come to switch jobs.

All of us have an inbuilt compass that decides whether the time has come to look for opportunities outside the realm of the current company. It could be for various reasons - lack of role definition, lack of challenging opportunities, unsatisfactory work conditions/compensation or a manager that we just don't get along with etc.

Apart from the "gut system", is there an objective way to decide to look outside? It turns out there is. One can figure out the answer if was one was to earnestly answer the below 12 questions -

Do I know what is expected of me at work?
Do I have the materials and equipment I need to do my work right?
At work, do I have the opportunity to do what I do best everyday?
In the last seven days, have I received recognition or praise for good work?
Does my supervisor, or someone at work, seem to care about me as a person?
Is there someone at work who encourages my development?
At work, do my opinions seem to count?
Does the mission/purpose of my company make me feel like my work is important?
Are my co-workers committed to doing quality work?
Do I have a best friend at work?
In the last six months, have I talked with someone about my progress?
At work, have I had opportunities to learn and grow?

I wish I had come up with these questions but I just happen to stumble upon them while reading "First, break all the rules" by Marcus Buckingham and Curt Coffman.

The questionnaire is put in such an articulate and simple manner that it helps you bring out the truth out of the closet regarding whether you belong in a specific place, are in a position to contribute and grow professionally. You immediately know the path you have to take if the answers are predominantly towards "Disagree" or "Strongly Disagree". By the way, they suggest that all the questions should be answered on the scale of "Strongly Disagree", "Disagree", "Neutral", "Agree" and "Strongly Agree".

To digress a little, the book was about the fact that individuals leave managers and not organizations. I'm sure you can clearly see that all the areas in the questionnaire can be directly addressed by an individual's manager thus I'm not surprised about their findings. After reading the book, I'm thoroughly convinced that individuals leave because of their managers. I hate to admit but I've had some smart individuals leave when they were reporting to me and now I can clearly see their reasons. I could pass the buck on to the organizational operative context but at the end of day I didn't do enough to realize their potential.

I hope this questionnaire helps everyone who have started to feel iffy about their current position but don't know what should they really do.

Faux Pas

2009-03-25T09:14:00.000+05:30

I didn't realize my blog was completely broken until late evening yesterday. I was using a borrowed blogger template from another site and the lazy me completely forgot to download the images, css and javascript from their site. The ignorant me didn't have a clue that they had bandwidth restrictions for their photobucket account where all the images were hosted.

It was awful to look at the blog and see all the nasty photobucket images asking to upgrade the account as the download bandwidth had been exceeded. Eeeww! I really apologize to all my readers(okay, sorry, I really meant 3 people who visit my blog everyday) who had to put up with this. With my credible intelligence level, I'm glad that on one thought that this was my new experiment in blog aesthetics. Hehee...

Crisis Driven Development

2009-03-19T23:07:00.001+05:30

Warning: Satire at work!

I'm one of the blessed souls to have been part of some software teams that leveraged Crisis Driven Development methodology. There has to be certain magnetic force that draws me in to these kind of software engagements. Um...Did I mention I was blessed? Yes, I did.

On second thoughts, it really has to be -

the sheer charm of working late hours and weekends so that I get to know the value of family. It truly works on the principle of distance makes heart grow fonder. It tests the strength of all the personal bonds and see whether they are worth keeping.

the lure of being randomised by always working on some burning issue that needs immediate attention as it affects 0.02% of the users. It truly tests whether everyone in the development team is customer centric and whether they are connected to every single user in the product ecosystem.

The high of being used just like any other hardware resource till I last my usefulness. It is a test of endurance, how long can one survive a burnout. I would like all the athletes vying for the Olympic gold to be part of such teams. Heck, Roger, if you are listening for ways to beat Nadal, this is it. Vamos, Roger!

The thought of never getting appreciated for all the hard work that I put in for all the "so-called" crisis. It gets you ready for the real world and makes you thick skinned.

Having all the crisis attributed to the community of Developers Shavelopers. It doesn't matter whether requirements were changed only 100 times in the process but it should have worked flawlessly once the final requirements were handed over just in time - I meant a day before the final delivery. It teaches the principles of accountability and how to build highly flexible systems that can read the minds of people running the business and calibrate accordingly.

Okay, enough satire but the fact of the matter is that it is extremely disheartening to see projects function like this. What does anyone get by randomising their team all the time? Only a bad product.

I'm not sure if these teams get anything done the right way. The management only stands to lose if the team is always in the crisis mode but they don't see through it. They think the team is fixing issues and rolling out functionality to fix the crisis whereas they are oblivious that it always results in creating more bugs and unmaintainable code base as no one really had the time to think through and do a proper triage.

One needs to learn that building high quality software takes time and the developers need to be given the right tools, the right frame of mind and just enough space to unleash their creativity to create the best of the best.

Have you ever been part of such teams? Would you like to share your experience?

Analysis Paralysis

2009-03-19T21:39:00.000+05:30

You know what is worse than taking a bad decision? Yes, you guessed it, not taking one. I've come across some very intelligent individuals who just couldn't make a decision and vacillated between alternatives, even when they did make a decision.

The thing which baffles me the most is that some of them are highly experienced, worldly wise and extremely intelligent individuals. I could spend hours with them talking about anything and everything under the sun and would come out of the discussion learning new things. But, I'm not sure what happens the moment they have to make a decision. They would either tend to consult lot of people before making a decision lest they might hurt someone's feelings/pride or would have their mental tectonic plates gyrate passionately on both the sides.

Something just takes over the cerebral faculties and clouds their judgement. Sometimes, these individuals take so much time that the issue becomes irrelevant. On second thoughts, it might not be such a bad thing at all as the issue does get solved :) I'm not sure if it is the case of not feeling empowered enough, fear of failure, over cautiousness, or a mere passion for over analysis but they need to tell themselves that they are in a leadership position because someone thought they deserved it(most of the times, they truly are deserving) and they ought to get things going.

The thing that might work in these cases is constant follow up regarding the pending decision after a resolution deadline has been setup. I know it could be hard and there is a chance that you might offend your peer or senior by the regular follow-ups but I've seen it work sometimes.

What are some of the strategies/suggestions you would recommend to tackle this behaviour?

Related Post

In State Of Denial

The Future of Management

2009-03-18T22:52:00.000+05:30

Have you ever felt that modern management practices in mid to large size organizations sometimes fall short on fostering creativity? Do you feel that your organization is not doing enough to harness the talent of all its employees? Is innovation localized to a select group in your company and you think the management is not doing enough to make innovation systemic? Have you ever had a nagging feeling why HR practices try to bring consistency and homogeneity in a heterogeneous environment? Do you think the employees are smart enough to manage themselves and organizations don’t need deep rooted hierarchies to watch over them? Do you think your organization is not nimble enough to adapt to the changing business environment?

Well, the book is for you if you answered in affirmative for any of the above. It is a very insightful book, which is divided in four sections - Why Management Innovation Matters, Management Innovation in Action, Imagining the Future of Management and Building the Future of Management.

The book starts with outlining the reasons why should we adopt new set of management principles and how current management practices are not suitable enough for the new knowledge economy. The second section outlines examples of companies which have adopted the new innovative management styles and are extremely successful - Whole Foods, W.L. Gore and Google. The third section outlines the need to first understand why we believe what we believe and then embrace new principles that foster innovation within an organization. The last section shares certain lessons to become a management innovator.

It's a book which would inspire you to challenge certain norms within your organization. I really enjoyed reading the book.

Social Search: AardVark

2009-03-12T16:04:00.000+05:30

How do we naturally search for information without using google? No, the answer is not Yahoo. I mean to ask without getting help from a search engine. Unfathomable, isn't it? Well, may be not.

Wouldn't you rather rely on someone from your personal network to seek advice on great places to see, the best restaurants to have dinner in, the most engaging movie to watch etc. than getting the information through google? Well, I generally get these kinds of information through like minded folks or "so-called" experts in my network and that's exactly the premise of AardVark.

AardVark is a social search engine designed to get your questions answered from your network though instant messaging products. I never thought there could be another way of searching after google but this definitely seems very promising.

Designing for Performance

2009-03-06T15:53:00.000+05:30

Do you think performance is tuned after the code has been written or is it one of those obvious quality attributes that has to be considered as part of the functional requirements of the system? Considering performance as an afterthought and deciding to tune your software towards the end of the development can only help so much. You really need to consider performance at every stage of software development life cycle. The system's responsiveness needs to be measured and calibrated at every iteration of your product design and development.

One can definitely design systems with that performance as an afterthought notion but I'm not sure whether they would be widely used. Would Google or Amazon have been that popular if they had taken minutes to search for content or books? I bet not. Would Porsche, Ferrari or Stealth Bomber have been possible with the performance-as- afterthought process? I bet not. They were built ground up for performance and speed and they live up to those standards. Why should you believe me? I've driven every single one of them. In my dreams. Every night. Seriously.

So, how does one approach thinking about performance at the design phase? Does one need to employ different design principles for different delivery channels? Though there are specialized patterns which cater to a specific delivery channel like Web, Mobile or desktop based solution but there are some which are overarching and can be applied regardless the delivery channel. IMHO, one needs to consider the following -

Caching - Caching of data can dramatically increase the responsiveness of any application though certain considerations need to be made before finalizing the strategy. One could follow W3 of Caching - What, When and Where.

What to Cache - It really depends on the application context but anything that would take inordinate amount of time to retrieve and doesn't change that frequently is a good candidate for caching. I also always think about the memory footprint of the cache. I would look at the size of the every object that needs to be cached by enumerating over its properties and computing the actual memory it is going to consume. This helps in the Where part of the puzzle.
When to Cache - One really has two options around it - Proactive or Reactive. Proactive Caching is generally employed for datasets(Not the ADO.NET DataSet) that get used in the application in most of the scenarios. It is a technique by which one loads the dataset at the start of the application. Reactive Caching is used when one is not sure when a dataset would get used thus it gets cached it after its first retrieval.
Where to Cache - It depends upon how fast the cached dataset needs to be accessed and how big it is? If the cached data doesn't need to be loaded in fractions of milliseconds and is huge then it makes sense to use an Out of Process Cache. But, if the size of the dataset is not that big then one can think about caching the data in the same process. In-process caching gets a little tricky for the web delivery channel when one has a cluster of web servers. As the cache needs to be identical in all the webservers, one has to either develop something in house or use sophisticated products like NCache to replicate the cache state among the clusters.

Data Structures - Poor selection of data structures can lead to lot of memory wastage and denegenration of performance. Would you really use a LinkedList for storing all your Customers? Would you use an Array for dataset that is always changing? Probably not. I think one needs to decide early the data structures which would get used in the domain model of the product.

Algorithms - One not only needs to use the right data structures to store the data but also use the right algorithms to insert/retrieve the data from them. They are tightly coupled and both of them have to be selected in tandem. If you were to sort your dataset, would you use BubbleSort or QuickSort? You wouldn't care if the dataset is too small but using BubbleSort in large datasets could be an extreme wastage of CPU cycles. The selection of the right algorithm plays a huge part in the responsiveness of your application and it makes sense to give a lot of thought to it.

Asynchronous Behavior - I'm not sure whether asynchronous behavior can increase the response times of your application but they can tremendously increase the responsivess of your application. In the world of short attention spans and even smaller patience levels, responsiveness means performance. One can use variety of techniques to break the long running transactions and execute them in a step manner while engaging the user. Do you have order processing which runs through a myriad business instructions? Do you have your users see a fascinating marvel called rotating hourglass and twiddle their thumbs after they submit an order? Wouldn't it make sense to break the transaction, put it on a order processing queue and let users know they would be informed when their order is processed? One can easily use some sort of queuing mechanism like MSMQ to decouple the component that submits the order from the service that actually processes the order. In the web scenario, one could use AJAX rather than having the user reload the entire web page again.

Interface Design - It makes sense to design coarse grained interfaces a.k.a Chunky Interfaces to reduce the chatter among the software layers. It's best to have calls to retrieve and insert data in chunks rathen than invoking multiple method calls to achieve the same logical unit of work. For example, let's say you had an Order class, which had details about the Order, its LineItems and the details about the Customer. Would you have 3 separate calls to create Order, OrderLineItems and Customer or just have one call to create Order, which would execute a transaction to create the Customer if it doesn't exist and then save the Order and its LineItems. It would be prudent to have only one call rather than making 3 independent service calls to achieve this logical unit of work as it reduces the chatter between service layers and help boost performance.

Data Partitioning - As the slowest moving piece of any application is I/O, it makes sense to give a lot of thought to the database design and how the data would be partitioned in it, if needed. There are times when certain datasets get used a lot, think of million of hits per day. In those scenarios, it makes sense to partition these datasets with the help of a partition key. The partition key decides how the data would be split physcially. For example, if you are building a social network and you get equal number of english, french and german speaking users then should you divide the users in different databases? It depends upon the context but one should think about the layout of the data persistence. In MS SQL Server 2005, one could even partition a table without having the application ever to know about that the data is distributed in different filegroups.

Are there any special design considerations that you know of, which can help boost the performance?

Why Technology Startups fail?

2009-03-04T09:19:00.000+05:30

I was having coffee yesterday with one of the founders of my company and somehow the discussion came around to technology startups. Why do so many of them fail? He said they fail because most of them don't have a strategy for acquiring customers. The idea could be brilliant but it is bound to fail or struggle if the startup doesn't know the go-to market strategy, how would they have the customers buy their products, and the cost of acquisition of these customers. Simple. Insightful. Profound.

The thing which can kill a startup is the cost of customer acquisition. A startup could be raking in millions of dollars but not be making a single penny in profits because of the high cost of acquiring and servicing customers. In the formative months/years, the startups don't mind paying more to acquire customers when compared to customers average spending on their site as it is all about growth and about getting the market to know of their product. But, it has to be backed up by a sound startegy of bringing these costs down over a period of years so that they can start making profits.

Thus, invest in creating a go-to market startegy so that you can understand the potential and limitations of your idea.

In State Of Denial

2009-02-26T23:22:00.001+05:30

I was ruminating over the perils for a business entity and figured the thing that can hurt an entity the most is having its management in the state of denial about its health. Heck, this could be true for everything around us - our projects, professional growth, personal relationships etc. Most of us run our businesses/groups with a perception that it is succeeding until we discover that we have landed over on the dark side. It's interesting to see that most of us make believe that it was an overnight phenomenon and it was never meant to happen whereas the reality is that tectonic plates of adversity had started gathering momentum long ago.

Have you ever driven in India? If not, well, first lie in the prostrate position and thank your stars that destiny has chosen you as the lucky one. Well, it's an absolute pandemonium! There are no rules on the road, people cut you off, they are always driving at your tail and the worst, they don't hesitate to unleash an honking avalanche and bury you under the cacophonic sound if you were not to give them way. Crazy, isn’t it? They say one can drive anywhere in the world if one has driven in India. Don't believe me? Wanna try it out? Yeah, I didn't think so :)

Now, the interesting part - One has to look at everyone's animated expression every time there is a fender bender (BTW, which happens every few seconds). People are ALWAYS shocked that they got into such an accident. Well, was it not expected in these driving conditions? One should be thanking the heavenly forces that it didn't happen few days back. But, they don't. Why? They are in the perpetual state of denial about the horrid driving conditions.

In both the above scenarios, there is a state of denial which leads to unpleasant ramifications. The logical choice is to look within and seek bold new initiatives which can challenge the status quo. But, the sad part is that sometimes we are too late in discovering the disconcerting facts about the health of our environment. By that time, the playing field has already changed and one has to work diligently to be part of the race again. Thus, the idea is to observe the business context intently and be nimble enough to make proactive changes.

Thus, always periodically review the progress against the plan, be inquisitive about the plans of your competitors, invest in innovation and nimble enough to adapt to the changing environments.

The Most Important Management Skill

Sunset at Kirkland

2009-02-26T14:07:00.000+05:30

I've been to Kirkland many a times and the sunset at this point never ceases to stupefy me. I always experience tranquility while sitting on the pier, hearing the waves crash at the beach and watching the sun rays disappear over the horizon. The opulent hues in the sky over the lake paint the world in such a surreal manner that it dwarfs any of the prolific works of painters of our times.

Related Post -

Dawn in Kirkland

The Most Important Management Skill

2009-02-25T23:34:00.000+05:30

Listening. Period. It has to be by far the most important trait that any manager needs to have. By the way, I'm not only talking about listening as in hearing but also interpreting their vernacular through their expression of speech and body language. We sometimes forget that verbal expressing is just one miniscule part of the overall communication and a major part has to do with the body language, posture, and facial expressions through which a person expresses their thought process.

I'm not sure why some of us are not good at it but the reality is we most of the times are caught up in our web of thoughts or already have preconceived notions about the individual conveying the message. Thus, we tune out somewhere in the middle of the message and don't give a fair chance to that specific message. It could be in a hurry to end the conversation because of some prejudice against that person/topic or our mind has already come up with some smart retort.

I wonder how one could manage effectively with a listening impediment. My sense is that one can't. A manager's job is, well, to manage the team. In my field, the success of the project is also governed by the cohesiveness of the team i.e. the social engineering aspect of it. Also, how motivated and aligned every team member is towards the success of the project?

Wouldn't someone with listening impediment miss the core issues team members are grappling with? As a manager, one has to weed out all the team issues to make sure the team is aligned with one thing only - success of the project. And, one can only really do it if someone can listen, understand, judge and respond effectively.

The Matrix Runs on Windows

2009-02-24T22:41:00.000+05:30

Yes, for real :) Don't believe me? See it for yourself.

ASP.NET MVC vs. WCSF

2008-03-12T17:21:00.000+05:30

I've been thinking about the best way of building enterprise web applications after creating few sample applications with both ASP.NET MVC framework and WCSF and if I was to look at from a holistic perspective, both the frameworks provide excellent separation of concerns thus enhancing the modularity and testability of the application yet there are differences in their approaches.

ASP.NET MVC provides a new ground-up framework for building web applications that doesn't uses the core ASP.NET framework concepts like PostBacks, ViewState, Code Behind etc. It really enforces the discipline of creating simiplistic views, which are dependent on a controller to handle user events and pass the model to it. It forces one to think of Views as a pure rendering mechanism and doesn't allow the developer to pollute the .aspx files with all the gobbledygook of interactions with services to build a page. It provides a very clean way of handling all the HTTP verbs and provides a pluggable architecture to inject components to handle those requests. All the HTTP calls are routed to a controller which decides the action i.e. method to invoke, gets the right model and passes it to View to render the data. One can build very clean REST based solutions by using ASP.NET MVC framework. To learn more about the framework, I would recommend reading Scott Guthrie's blog and viewing these videos.

WCSF on the other hand builds uses the existing ASP.NET Forms based framework and provides a pattern based approach of building applications. It has guidance bundles which provides the right solution and project structure, code generation recipes, out of the box integration with various application blocks like Security, Exception Handling and Logging. It also provides good separation of concerns by following the MVP design pattern. Thus, a developer doesn't have to learn new patterns of building web applications and can utilize all the core concepts of PostBack,ViewState etc. and on top of it is given pre-configured access to the Logging, Exception Handling and Authorization features.

IMHO, WCSF is a better option if one was looking at capitalizing on ASP.NET strengths and, standardizing web development practices. ASP.NET MVC framework provides utmost flexbiity in creating web solutions but it's a 180 degree change in the way we have been building ASP.NET web applications and I'm not even sure how would it pan out while building enteprise applications. In the coming days, I would publish a detailed comparison of both the technologies.

Starting out with Web Client Software Factory a.k.a WCSF

2008-03-10T19:32:00.001+05:30

Lately, I've been completely submerged in Software Factories - WCSF and WSSF(Web Service Software Factory) and have come to like one of them, WCSF. I'm bit surprised about the fact that it hasn't caught on to lot of people's imagination albeit having lots of features to create ASP.NET web applications in a standardized way.

What it is?

It is designed to help one quickly and consistently create web applications that adhere to well known architecture and design principles and patterns. It is an integrated collection of VS.NET solution and project templates, various code recipes, design patterns and prescriptive guidance about creating large scale modular web sites. It synthesizes ASP.NET, ASP.NET AJAX, MVP Design Pattern, Application Blocks of Enterprise Library, Windows Workflow Foundation and VS.NET Solution templates and wizards to create these modular web sites.

Why should one care about it?

There are various reasons why one should care about the Client Software Factory but there are few that stand out -

It provides a sound baseline for creating web applications which provides a mature web solution structure, out of the box integration with various application blocks like Security, Exception Handling and Logging and, good separation of concerns between layers(through Dependency Injection) thus improving the testability of the application.
The Factory allows an organization to customize and extend the code recipes and templates to suit their architectural style.

What is the best way to get started?

The best way to get started on WCSF is to install it and go through the hands-on lab on codeplex. One needs the following before installing the client factory -

Guidance Automation Extensions - This would enable the custom recipes of WCSF. Please install this before the GAT.
Guidance Automation Toolkit (for VS 2005) - This would facilitate customizations. Please follow this link if you have VS 2008 installed
Finally, install WCSF from here

Though there are lot of how-to articles and help on codeplex for WCSF but I've found David Hayden's Screencasts an invaluable resource in understanding WCSF. He has brief presentations covering various aspects of WCSF and I would highly recommend viewing all his screencasts before starting out.

Life is an Experiment

2008-02-12T19:16:00.001+05:30

In the last few days, I've been doing lot of reading around how to blog effectively and have come across lot of interesting articles. Umm..., wouldn't it have been little better if I was doing more writing than reading? Anyways, I've read articles about blogging, making blogging a habit, how to attract visitors on your blog etc but I don't know how I landed on this J.D. Meier's article about Life. But, I'm thankful I did.

I had heard lot of interesting sayings like that "Life is a journey", "Life is a circle" etc. but never that "Life is an Experiment". What a profound statement! It truly defines us and accurately depicts the frailty of human nature. It lets us take our mistakes and shortcomings in our stride, allows us to be fallible and fills hope that someday we would be much closer to the person we want to be.

Infinite choices i.e. The Long Tail

2008-02-03T22:53:00.001+05:30

What an insightful book! I just finished reading it cover to cover in a day! It enlightened and helped me see the great success stories of our digital age with a new perspective. I always knew the technical reasons behind the successes of eBay, Amazon, iTunes, Rhapsody and the likes of them but never in a way Chris explained it in this book.

It would help one understand the explosion and successes of the new internet businesses and the reasons behind them. It compares the online business with the traditional retail business and explains their edge because of the following forces -

Current abundance and affordability of the tools of production.
The dramatic reduction in the cost of distribution in this digital age.
The prevelance of search filters to connect the suppliers and the actual consumers.

Take for an example, Why Wikipedia is more successful(or, will be) when compared to MS Encarta and Birtannica or how Rhapsody/iTunes are more popular when compared to the traditional retail stores for music like Tower Records.

I'm sanguine that the pattern of Long Tail is ubiquitous in the modern times and I could even relate it with our own business model. I work for an Outsourced Product Development company,GlobalLogic, and I think we fit in the Long Tail of our market segment. Furthermore, there is a Long Tail within our own revenue model - majority of our revenue is driven by minorty of our partners i.e. The 80/20 rule.

I just feel that it could have been written a tad well as lot of examples have been repeated in the book. I think it could be have been wrapped in 150-170 odd pages rather than its current size of 226 pages. Nonetheless, a must read!

Thank You, Jacky!

2008-02-03T21:51:00.001+05:30

I just updated my site with a blogger template from Jacky Suit. It has given my blog a professional and clean look(at least one person in this world thinks so..ahem..me). It was easy to download, the instructions were extremely straightforward and on top of that, it didn't cost me a dime!

The ironic thing is that his own blog didn't have that professional look and it looked like an information clutter house. Hopefully, someday he would become cognizant of his own creativity.

Thanx mucho Jacky!

Alien Abductions

2008-01-30T16:20:00.001+05:30

I've not been able to read/write anything lately. I wish I could simply put it on some sort of alien abduction or a self-discovery Himalayan trip. Well, not that I don't need a long self-discovery Himalayan trip but that's a matter of another blog entry. Anyways, the fact of the matter is that I've hit some sort of reader/writer block. Hey, is it me or it sounds uncannily like a ReaderWriterLock?

Note to Self: To do something about my sense of humor as it is getting restricted to technology.

My reading list is growing bigger by every passing week but I just can't seem to muster any strength to read. Trust me, I do have enough interesting books to read but they are just gathering dust on my bookshelf.

Note to all the readers from Rest of the World: When I say dust, I really mean it and I'm not using it as a figure of speech. One of the benefits of living in New Delhi is that one not only gets to live in a true cosmopolitan city but also gets to experience living in a desert like Sahara. Yes, it is that dusty!

The same thing applies to writing. I get restricted by my own thought process every time I begin to write anything about software, technology or life in general. Could it be that I'm completely burnt out by working hard over the last few years and I really need that Himalayan trip.

Burnt Out, Working Hard. C'mon whom are you kidding? Why don't you come out of the land of the exaggerations, Now!

On a serious note, I just think it would be futile to write as some other person would have already expressed something similar thus what is the use of me expressing it again i.e. why to get into something called Blog Echo? But, I think I have to get over that notion as I would giving *that* thought a new life by expressing it in my own words. There is a chance that expression of my thought process might be able to showcase a new perspective to someone even though they might have heard the same thing before.

I heard a wise saying about always making one's goals public. It doesn't mean that one might not fail but it reduces the probability of failure. So here it is, I plan to read and write more.

Here it is to the possible winds of change! Let's see what happens.

Marriage of WCF and ASMX

2007-11-07T23:48:00.001+05:30

I never imagined that writing a simple POX based application with correct interface semantics in the current release of WCF would be such a pain. Wait. It is impossible.

The intent was to expose a WCF service that can return POX when invoked through a regular well-formed URL with Query strings or submit HTML form data to it while doing a HTTP-POST -- Just like we used to do in the good ol' ASMX days. Guess what, this feature is not natively built into the current release of WCF.

Well, the WCF geek would be shrugging his head right now, itching to show me the way to build WCF services that can be consumed by HTTP-GET/POST verbs and can return POX. Not so fast, boy. I'm cognizant that one can build services which can return POX but they are loosely typed i.e. they have to declare methods which can only take Message type as an input. What does one do when one has to define operation contracts that need to define native data types as input parameters in the method signature?

Okay, I lied.

I know there is yet a better way -- To build a router, basically a message inspector, that can route the messages to the right operation contracts by inspecting the kind of action that is desired by the user. But, can you imagine how much code one would have to write to build this router in the context of a gigantic enterprise class application? I think you already know the answer. At least, I don't want to. As a matter of fact, I cringe imagining someone writing that kind of code as it would not be maintainable in the long run.

The good news is that WCF in .NET 3.5 would support the desired feature set out of the box but the bad news is that the upcoming framework is still in beta and to make matters worst, there isn't any confirmed release date. We have to roll out the system in production by January thus couldn't bet on the beta release. Thus, I ended up choosing ASMX to build our service delivery platform. But, I really wanted to harness the power of WCF to build the service delivery platform as it provides options galore to configure and host services.

So, what did I end up doing? Married both the technologies. Our service interfaces have both the WebService and ServiceContract Attributes, the methods have both the WebMethod and OperationContract Attributes. We ended up created ASP.NET Web Services project and decorated the WCF tags on the services which would allow us to switch to the new framework when it comes out without compiling the code. Of course, we would have to provide the right configuration to host it as a WCF service but still we wouldn't have to touch the code. Here is how a service interface looks like --

This way we de-risk ourselves by not adopting a beta product but still be ready to embrace the new framework. Have you grappled with this problem and have a better solution? I would appreciate if you can take the time out to share your thought process.

Agile Crazile

2007-07-18T22:21:00.001+05:30

Blasphemy! You would say.It might be, especially when the whole software world treats agile methodologies as a panacea to all its problems. I have to admit that I've been in a perpetual state of skepticism about the benefits of agility and the fact of the matter is that I haven't found any valid reason to move out of my comfort zone of skepticism.

I think it tries hard to solve the problem of showing the ROI faster to the client but in the process ends up creating an even bigger problem -- Chaos. Chaos has ensued in almost all the engagements that we have executed using agile methodology. Some of it definitely comes from our lack of understanding of the agile practices but most of it comes from the way the methodology evangelizes the projects to be executed. IMHO, agility has got benefits but it hasn't matured yet(again, could either be lack of our understanding or dearth of good case studies). Most of the material out there has been written by very gifted writers which ends up seducing everybody to espouse it but I think most of it is bit pedantic and lacks pragmatism. Furthermore, almost all of it has been written by comparing it with the wooly mammoth of Waterfall -- Waterfall is bad thus Agile is good, hence proved. Whoa!

What leads to this chaos? The culprit, more or less, in all the engagements has been the flux in requirements and the monstrosity involved in managing frequent iterations. I believe the whole notion of succinct user story is somewhat misleading. It doesn't give good enough idea to think about the design holistically and provide enough opportunity to put hooks in place so that the system can evolve over a period of time. Think of products like Operating Systems, Office based applications, your favorite IDE, Amazon.com, eBay, iTunes etc. -- do you really honestly believe that the engineers of these systems would have just gotten user stories, planned them in iterations, did few design spikes, few whiteboard sessions, created "just enough" documentation, planned enough for the first few months and starting churning out code without putting emphasis on rock solid design and what the future holds. Well, I don't think so.

I don't think one can adopt the agile philosophy to create systems which target millions of users. I believe all the lofty notions of brief user stories, design evolution and "design is code" can be put on a decorative mantle in these cases. While constructing such systems, wouldn't you want to detail out the vision of the system? Wouldn't you want to create detailed use cases from the perspectives of all the users? Wouldn't you want to take your time to design? Wouldn't you want to get submerged in creating comprehensive documentation to convey the thought process to the army of developers rather than doing the "just enough" documentation which can lead to ambiguity? I think the cost of refactoring the design and code and deploying them again is astronomical in this context thus it's better to follow a thorough approach rather than an agile one.

The other thing which contributes to the crazility is frequent releases. Somehow, the development team always gets bogged down by fixing the critical bugs from the previous iteration's feedback loop. One would think that the choice comes down to working on new set of requirements or bugs from the previous cycle but that generally isn't the case. Why? The thing is the development team has to roll out the agreed upon user stories and the customer also expects the team to fix the bugs thus the team dynamics goes out for a spin. It seems to me they get tied in a vicious cycle where they are always constantly refactoring the design and code to accommodate the new set of changes. It is like a dog chasing its tail.

The problem worsens when one doesn't have strong program management in place which can't push back and educate the customer about its fallout on the team dynamics. Ultimately the developer team has to work longer hours at work. This defeats one of the core objectives of agile which mandate the developers work only 40 hours week in regular iteration followed by a sprint towards the roll out of the product.

Don't get me wrong, I absolutely love the idea of frequent iterations as it provides real value to the customers as they get to be part of the growth of their software but detest it for its repercussions on team dynamics. I love it as it gives the customer an opportunity to provide early feedback to the team but abhor how the development team has to work long hours to incorporate that feedback. I think the core of both the above problems is that agile methodology believes there would be enough time to build the software the right way but the fact of the matter is that software is always conceived by a strong time bound market need and, the software would gather dust if it was to miss that timeline. In my experience, I've not once seen a piece of software being constructed without a time bound market need.

I believe most of the agile practices are best suited for a very ideal software construction setting where the developer works in tandem with the customer and both of them somehow magically agree on all timelines and what features have to be constructed. I think that kind of framework would let developers easily manage the flux in requirements and incorporate the changes from the feedback loop without any added pressure.

Nonetheless, as I said in the beginning, I'm a skeptic, but still working passionately towards to become a believer as there are lot of merits in following agility. Thus, I'm currently reading two of the famous works on Agile which have been recommended to me by people I respect a lot --

1. Agile Principles, Patterns, and Practices in C# by Robert C. Martin

2. Agile Management for Software Engineering: Applying the Theory of Constraints for Business Results by David Anderson

I would love to hear any thoughts on how to execute projects on agile methodology, especially some good case studies of executing large scale enterprise projects.

Got Thoughts?

PS - Thanks to my friend Rajeev Singh for coming up with a very creative name for the problems we are facing on daily basis i.e. Crazy + Agile = Crazile

6th Sense

2007-07-12T15:40:00.001+05:30

Starring: Malcolm Crowe (The Developer), Cole (The Program Manager)

"I see dead people!"

Probably one of the most famous movie lines of all time, isn't it? The movie takes one through the journey of a child psychologist Malcolm Crowe who dedicates himself to heal Cole, a young boy who sees dead people, but in the end Cole ends up healing Malcolm by making him realize that he is dead! – This is something which sets Malcolm free from all the pain he was bringing on himself. Sorry to break the plot to all those who have been living in a cave since the last 8 years and didn't get a chance to see this movie in a hot 'n' happening Caveatre(Theatre in a Cave).

You ask what on earth this movie has to do with ".NET Development + Architecture + Perspectives"? Well, it has everything to do with the all-knowing, self-acclaimed superhero we know as software engineer a.k.a Malcolm Crowe. I think the reality that sun is about to set on him is going to hit him hard, the sudden pang of realization that he wouldn't be the reigning king of Softwareopotamia will be a severe blow to his ego. But, the ultimate question is, would this realization set him free or would he imprison himself in the cage of depression? Would he able to overcome that Cole, the Program Manager, is the one who will help him bring his suffering to end?

Malcolm, who thinks that a software can’t be conceived without him (rightfully so) and he should decide the timelines of constructing one, is in for a surprise. He who elevates himself to the status of a Geek God by spending long hours at work is in for rude awakening. He who thinks that he is know-all and can help Cole is in for shock as the tables are about to be turned on him.

So, what is going to happen at climax? Well, enters 6th Sense in the next scene. Now, wait a minute, how can a movie itself enter in its own scene? Isn’t it recursive? Betcha! But, I’m not talking about the movie. Please, give me some credit, will you? I’m talking about 6th Sense, the Analytics product --

“6th Sense Analytics™ is a hosted software solution that automatically and unobtrusively collects the metrics that organizations need to understand the execution of software development projects. Whether the work is onshore, offshore, in-house or outsourced, 6th Sense Analytics brings the software development process into focus, ensuring your projects are on time, on budget, and on target with business goals.”

What does this mean for Malcolm? Short answer, this means the big brother a.k.a the Program Manager a.k.a Mr. Cole is going to be watching you!! Long answer, it means that data sensors will be installed on your machine which will monitor your favorite IDE to collect metrics about how you program, when you program, when are you best at programming a.k.a Flow Time and what is your overall velocity?

Malcolm: Say What?
Cole: Sorry, I told you that you were in for a rude awakening.

Malcolm (panting): How can they?
Cole: Well, I can understand but the fact of the matter is that it’s not your hobby project that someone is monitoring. People are investing real money for you to construct software for them.

Malcolm (furious now): Software is like an art not a science!! How do you expect me to program when someone is watching over my shoulder?
Cole: [Pause], Well, you are right! It does add some extra pressure for one to constantly produce which might affect the overall quality of the software as one might be in the race of being the most productive person. But, in the end, this is being done just to monitor the overall velocity of a developer like you. You can choose not to indulge in the rat race of producing X number lines of code per day. This product would just give you a metric of how much work can you get accomplished per day. This in turn would help you estimate future projects better. Not to mention, it provides lot of transparency to the Program Manager and the customer about all your hard work.

Malcolm: hmm….May be it is for good, I think I would have to try it for myself. So, how does this work again?
Cole: Here it is --

I'm betting that this realization would set Malcolm free and would make him an informed developer if not a better one. I think this in turn will help the entire ecosystem of software engineering companies like ours. So, all the Malcolms in Globallogic, please be ready to embrace the new productivity analytic tool as we have forged an alliance with 6th Sense analytics company to integrate this product in our home grown Velocity Platform -- http://opensource.sys-con.com/read/399503.htm

So, What is Velocity? --
"The Velocity™ Method is a blueprint of processes, artifacts and role definitions that have been refined over 1,000 product releases to quickly bring high-quality products to market."

Here is our ideal Velocity configuration --

I think this alliance would go a long way in helping us become a leader not only in the quality of Software construction but transparency as well. Now, how much impact does this "Mr. Big Brother Watching You" product has on developer's productivity is need to be seen.

Thoughts?

Managers are from Mars, Developers are from Venus

2007-07-12T15:36:00.001+05:30

Really?

It turns out to be true. If you don’t trust me, please check it out in your team, in the team next to your cubicle, in your company or at a local theatre near you.

Be warned that this has been written from a perspective of a developer thus certain language might be unsuitable for managers especially newly conceived managers – This gibberish has been rated MMG. Thus, Manager of Managers Guidance might be required while reading it.

What really makes us different? Don’t our hearts beat in similar fashion, don’t we go through the similar rhythms of life, doesn’t the thought of a chilled Corona with lime on a sunny afternoon evoke a similar response from us, don’t we perceive reality the same way (ya kiddin’ me?), and don’t we communicate and express our thought process alike (you really pushin’ it now)?

PS – I guess the similarity stopped at our hearts beating in similar fashion.

But, most importantly, we have the same lowest common denominator!! Managers were also developers at one point in their lives but, unfortunately, they mutate once they are beamed out of the bit world. Nonetheless, can peaceful bilateral relationships exist between Mars and Venus, Managers and Developers? Is there hope?

Short Answer, Yes.

Long Answer, Y e s :)

If you are a manager and think your developers are on a totally different plane, please read on.So, what kind of attributes can bring you closer to your group of developers? Here is my simple list, please feel free to add/remove or critique it all together --

Technical Competence – Show me a manager without technical competence and I will show you a manager who is not very well respected among the developers. Well, you may get lot of brownie points for being a great human being or having a charismatic personality but you won’t ever be able to be one of the guys if you don’t have a technical bone in your body. I’m not saying one has to have a degree in computer science but decent understanding about what your team does on a daily basis is definitely a must. Furthermore, how can you really act as a conduit to the success of your kick-ass developers or mouth off their accomplishments in front of your customers and senior management if you don’t understand technology? I believe this attribute also helps you to kick ass when it really comes down to marrying business needs with the technology roadmap.

~~Autocracy~~ Democracy – How hard it is to get rounded decisions by taking a holistic perspective of the team? I think if you are one of the managers who micro manage things or takes decision in autocratic way then you really have to figure out a way to change yourself. There are times when autocratic decisions are required but they should be few and far between. IMHO, one only stands to gain by listening from the people who are in touch with ground reality a.k.a code a.k.a the software.

Ability to manage ambiguity – I think majority of the developers are generally much contended with creating bits and lack the overall business perspective somewhat. I think a manager would score really high on the developer’s respect-o-meter if he can bring in some level of clarity to ever changing business needs and articulate that in technical context. Yes, articulation in technical context. Not sure, how is this possible? Please look at point number 1.

Playing to their Strengths – I think one of the reasons project fail or are behind schedule is that developers don’t play according to their skills. I think one can only have a highly productive team and developers with a healthy swagger if one plays according to their strengths. In my experience, most of the managers generally have this attribute imbibed in their DNA as they are born with the natural instinct of mitigating risk. But, if you think you are not one of the lucky ones, then you would have to collaborate with your developers at a deeper level to understand what moves them. In case, you are not in a position to figure out the strengths then I would suggest following Scott Berkun’s advice of asking a developer how you can help them shine or kick-ass in the project.

Growing Talent – Playing to your team’s strength alone is not enough you really have to help them grow. I don’t mean helping them climb the corporate ladder faster but helping them realize their true potential. It means either mentoring them or helping them find the right mentor. It means helping them to take on lateral tasks which might not be part of their core skill set. I think developers are generally more productive and committed if they think they are following a well laid out plan.

Technical Competence $30
Playing to their Strengths $25
Helping your team grow $25
Ability to manage ambiguity $20
Standing up to defend the rights of your team, Priceless!

Before you go on a rant complaining about this, please bear in mind that this has been written by a New Manager On The Block who is still learning the ways of being an effective manager.

Designing Windows Folder Structure

2007-07-12T15:17:00.001+05:30

For the first time, I was let down by my esteemed GSP(Google Search Professional) credentials when I couldn't search for a single white paper or article about data segmentation techniques for storing large number of files on a disk. Thus, I had to spend couple of hours to create a test harness and gather results to figure out the optimal number of files in a folder.

The Context

We have tens of millions of files to be served from our media server and we were looking for optimally arranging them in a folder structure so that it doesn't impact performance. The options were either stuffing files on a single folder or creating a hierarchial structure of folders to store these files. If we were to opt for the latter then we had to solve another problem -- the optimal number of folders in a given folder? (It turns out that this really is not a problem as the performance level for files and directories are very comparable)

The Approach

I decided to write a simple program which would create N number of files in a folder and would then try to locate a particular file by its name. It turned out to be very simple program to write but I had lot of "doh" moments while running the program. For creating the files, I just created a simple method to copy an image file and paste it with a unique name --

   private static void CreateFiles(int numberOfFiles)
   {
        Stopwatch stopWatch = Stopwatch.StartNew();
        for (int i = 0; i < numberOfFiles; i++)
           File.Copy(sourceFile, path + "thermometer" + i + ".jpg");
        stopWatch.Stop(); 

        Console.WriteLine("It took {0} seconds to create {1} files", stopWatch.ElapsedMilliseconds/1000, numberOfFiles);
        Console.Read();
   }

And, to locate a file --

    private static void LocateFile(string filePath)
    {
         Stopwatch stopper = Stopwatch.StartNew();
         bool isFound = File.Exists(filePath);
         Console.WriteLine(isFound);
         stopper.Stop(); 

         Console.WriteLine("It took {0} milliseconds to find the file", stopper.ElapsedMilliseconds);
         Console.Read();
    }

I ran the program for the following configurations --

1,000 files in a folder
10,000 files in a folder
15,000 files in a folder
18,000 files in a folder
1,000 files residing in their own unique folders. Thus, it had 1,000 folders.
10,000 files residing in their own unique folders. Thus, it had 10,000 folders.

The Results

I ran the tests in a single user mode on WindowsXP with 1.6 GHz single core, 1 Gb of RAM laptop having a 5400 RPM disk. I've taken out the results for configuration #5 and #6 as they were identical to #1 and #2 respectively.

Our overall SLA to serve an image is 500ms thus we want the image retrieval cost to be within 40-50ms as the application has to fire business rules before serving an image. According to the test results, I've come to the conclusion that the ideal size for our case is really 10K files in folder as the OS was able to serve an image in 37ms at the first invocation and 6ms in the subsequent invocations. The overall strategy that has been devised is --

Create a Hash function which can take the name of the image file and output the folder name. The folder name will not be more than 4 digits so that it can support maximum of 10k entries.
Create folders underneath the hashed folder with the unique ids(For our case, it would never be more than 10K at this level)

Next Steps

Create an ASP.NET site which would serve the images.
Run the ASP.NET site in a program like JMeter to check the results in a multi user mode.

Please feel free to comment on the above approach and share your experiences in case you have designed for such a scenario.

Attribute Driven Architecture

2007-07-12T15:01:00.001+05:30

I think it's an Architect's primary responsibility to come up with the most bizzare and ambiguous terms to define very simple constructs - We somehow have to show that we exist for a reason :). I was recently asked to take a session on Arch. 101 to share the things that we consider before creating architecture. Thus sprung the idea of discussing all the "abilities" and how they act as a conduit to come up with the complete technology roadmap. Here is my perspective on all the design principles that can be followed to promote one ability over another --

Thou shalt not make architecture without having an ability in mind

What are these so-called abilities and how do they come in play while crafting the blueprint of an application?

Maintainability
Performability(is that even a word?) a.k.a Performance
Scalability
Reliability
Flexibility

By the way, there are plethora of abilities but I will cover the important ones which come in play often enough.

Maintainability -- It is defined as an ability of a system to undergo repair and evolution. Following principles can help one achieve this ability –

Modularization
Single Responsibility Principle – One should design classes in such a way that there is only a single focus for change.
Open-Closed Principle – One should design frameworks which are open for extension but closed for modification
Liskov Substitution Principle – One should design sub-types which are substitutable for their base types.
Guidelines for Writing Good and Readable Code -- I'm cognizant that this is really not a design principle but helps tremendously in creating maintainable applications.

Performance – It is defined as the responsiveness of the system. Keeping the following principles in mind can help build a high performance application --

Caching – One can follow the W3 principle of Caching – What to cache, When to Cache and Where to Cache.
Asynchronous Behavior
Coarse Grained Interfaces – Chunky interfaces always perform better than the Chatty ones.
Data Partitioning
Efficient Resource Management

Scalability – It is defined as the ability to maintain performance while system demand increases. Following design principles can help build a scalable application --

Layering
Loose Coupling
Data Partitioning

Reliability – It is defined as the ability of the system to keep operating over a period of time. Designing the system while keeping the following things in mind promote reliability –

Robustness
Clustering

Flexibility – It is defined as the ease with which a system can be modified for use in applications or environments other than those for which it were specifically designed. Following principles promote flexibility of a system --

Layering
Loose Coupling
Single Responsibility Principle
Open-Closed Principle
Code To Interface
Interface Segregation Principle

Of course, there are always trade-offs one need to consider when thinking of one ability over another. For example, if one is designing for flexibility that one should be ready to let go of some performance constraints.

References – Implemeting System Quality Attributes by Gabriel Morgan