CodeClimber

Processing large files with Ruby and Rails

2009-06-04T18:00:00.006-05:00

UPDATE: As "stefano" pointed out in the comments, the standard "gets" method does indeed accept a parameter. My fault for not checking the documentation!

although we as web developers prefer working on neat features for our websites, sometimes we need to get down and dirty with data processing. I know, I don't like it any more than you do, but if you want to run a business sometimes you have to do stuff that isn't that fun. That's one of the reasons I liked Ruby; the file library for reading and writing files makes data processing a lot simpler than similar tasks I written in C++, Java, and other mainstream languages. Usually I just do something like this:



file = File.open("some_file")
#read all the contents of the file into "str" variable
str = file.read
file.close
...#do some processing...

Wow, that was easy! However, sometimes code like this just won't cut it. For example: if your file is too big to read into memory all wants and could cause performance issues for the server, you may want to process the file iteratively. Again, and Ruby provides a pleasant way for handling the scenario. The "gets" method accepts a block and yields back to you each line of the file one at a time, thus conserving your precious memory. See the example below:



file = File.open("some_file")
while(cur_line = file.gets)
   ...#do some processing...
end
file.close

Also pretty easy. Today, however, I ran into a new problem. What happens when your file is too big to read into memory at one time, but all the data is all in one single line? Don't believe that would ever happen? Check out EDI sometime and see what you think about it (On second thought, never checked that out. Never ever look at EDI. I wouldn't want to make you cry). Sometimes even XML or HTML files are written all on one line in human readability isn't of any particular concern.

Well, I have never dealt with that situation before. I had this file I needed to process, roughly 30 MB, all on one line. now, iterative processing would be okay, because each segment in the file was in the proper order for processing and it was delineated by a pipe character, but there just aren't any built-in methods all in the file object that do what the "gets" method does on a delimiter other than newline. So I wrote one:



class File
  def uber_gets(delimiter)
    segment = ""
    self.each_byte do |byte|
      char = byte.chr
      if char == delimiter
        yield segment
        segment = ""
      else
        segment = "#{segment}#{char}"
      end
    end 
  end
end

with this modification, you can now do small iterative processing based on any delimiter. In my case, using EDI files, each record is separated by a "~". so, I used the above method as follows:


file = File.open("some_file")
file.uber_gets("~") do |segment|
   ...#do some processing...
end
file.close

There you go. The whole file is on one line, but the code is still respecting memory consumption. If it helps you, enjoy. I'll post a link to the gist:

http://gist.github.com/123924

A new way to Blog

2009-06-04T08:39:00.004-05:00

Most of my day I spend in front of a computer. I write code, I answer e-mails, and then when I get home, I blog. All that typing can be hard on the hands. I try to do most of the right things, ergonomically speaking. But I still end up with tendinitis. Being only 22 years old, this is obviously something I really want to avoid if I'd like to continue a career in the software business.

Enter dictation. This isn't the first time that I play with the idea of speaking to my computer. Both Windows Vista, which I had installed in my old computer, and Mac OS X, which I have all my new computers, have built-in speech recognition software. However, this is all mostly for command and control. The software helps you do things; you can open new windows, push menu buttons, click links, and do all sorts of other command based tasks. But when it comes to actually writing, these solutions fall short. Today, though, I'm happy to say that I'm now past that point. every word that you are reading on this page was put there by dictation software. MacSpeech dictation is the program I'm using, and I have to admit I'm impressed. Everything I say seems to just end up on the screen without me having to use my hands or forearms.

there are some drawbacks to the software. For one, it's not cheap. $200 for the current release. Now admittedly, that's not the most expensive piece of software ever seen, but for regular consumer consumption of price point seems a bit steep. On top of that, there is no trial version you can download. In fact, even when you buy it you can't download it. You have to have it shipped to you as if we were back in the 90s. so if you decide you want to buy MacSpeech Dictation, be aware that it's all or nothing.

For someone like me though, the advantage of being able to do my blog posts without my hands far outweighs the drawbacks inherent in MacSpeech's distribution system. I hope that as my experience with the package progresses, I'll be able to say that all my e-mails and all my blog posts are done without putting any unnecessary strain on my forearms. That way, I can save my limited typing capacity for what I enjoy most: code!

If this is something you'd like to try, check out the link below.

Mac Speech

Metric-Fu

2009-05-26T22:48:00.002-05:00

The latest addition to my utility-belt of Ruby tools, Metric-Fu is giving me plenty of ideas on how to refactor my codebase.

Essentially, this library gives you the ability to take 8 of the most common code analyzing tools and run them all on your codebase at once, producing one consolidated report. I love it!

The full list can be found at the Metric_Fu page on rubyforge.org, but 2 of my favorites are listed here:

Roodi gives you some design help as it checks all kinds of common programming problems. Method have too many parameters? Cyclomatic complexity too high? Forget the else clause on a case statement? Roodi will give you the heads up you need.

Flay checks out your code for duplicate constructs and segments. It found several pieces of duplicated logic that I had never noticed before, I was really impressed. It also does soft matches on "similar" code, things that could probably be combined and simplified if you're clever. Very nice.

The great thing about metric_fu is that you just run one command, and all the packages get run for you, giving you one page afterwards containing links to the results of whatever package you want. Check it out, you might be surprised at how much your fingers start itching to go back and fix all the problems you didn't even know you had.

Curse you, rake db:migrate!

2009-05-25T12:32:00.003-05:00

Have you ever been in this situation?

"Hmm, this feature will require a big change to my database! I mean, we're going to have to touch every single record in that table."

If you have, you've probably followed up with this thought:

"Good thing I'm using Rails! They make it so easy!"

And then you went and wrote this migration:


def self.up
  Model.all.each do |m|
    #..some important function 
    #performed on every object..
  end
end

And then you were really proud of how quickly that went, and you run it on your development machine, and it works really well. But THEN you push it to your staging or production server that has way more data than your dev machine, and you get this staring back at you from your command line:


 ** [out :: 123.123.123.100:8063] ==  
 ** YourCrazyMigration: migrating 
  =========================================

And you stare at that for about 30 seconds before shouting:

"Mother F%^&er! I did it again! I can't believe I did it again! I built a stupid migration that uses the stupid 'all' method which is now dominating the memory on that box and I can either kill it and pick up the pieces or let it run for the next 3 hours as it pages the hell out of the hard disk!"

Well, since I did EXACTLY THAT just now, I decided that from now on we'll be using a new migration task at our development shop called "safety_migrate", which you're welcome to take advantage of if you'd like. It runs through every file in your migration directory, checking for the dreaded "all" method, and WILL NOT run the migrations unless every file is clean!

check the gist:

http://gist.github.com/117625

Happy Migrating!

Maximum Impact for Minimal Effort

2009-05-17T20:10:00.002-05:00

Today was a day for working outside, and I got a lot of mulch spread over the landscaping at my house.

I love mulch because it's really easy to use, really cheap to buy, and really makes a difference in the way your yard looks. You can't get much more bang for your buck as far as landscaping goes.

Believe it or not, I was thinking a lot about work as I was hauling chips of cedar over to my flower beds, and I started considering what sorts of things are "mulchy" in the web development world.

Color

Unfortunately, people aren't often impressed by performance or functionality nearly as much as they are by aesthetics. A white page full of blue links that all do really cool stuff is just a huge turn off. The difference that can be made with a simple header, a 3-column layout, and a pleasant color scheme is phenomenal.

Central Navigation

Sometimes you have websites that have spaghetti links all over the place. Trying to get back to the homepage usually means either clicking the back arrow 14 times, or re-typing in the domain address. A very common and successful approach is to have a set of navigational links as part of the header, and it's successful for a reason: people know where to go. No matter where they are, the critical places on the website can be accessed easily.

Intelligent defaults

If you have a dropdown that has a list of states, and 80% of your users are local to your state, go ahead and default the selection to your state. If you have 25 reports users can run on your website, and 3 of them are used more often than any others, put those three at the top. It doesn't take a lot of work to reposition things, but it makes a big difference.

There you go. Three easy things to do that will make a large impact on your users.

Joins and named_scopes in ActiveRecord

2009-05-14T08:18:00.007-05:00

So many of us know how cool named_scopes are in ActiveRecord; they really make building complex queries quite pleasant compared to writing out big hairy SQL strings all over the place. However, in the past week as I have refactored my whole web application to use this excellent feature, I've run into some little-discussed items that I feel should be shared somewhere. Hope you enjoy, and anybody who is more advanced in their Rails-fu is welcome to give me some schooling as to where I've gone wrong on any of these, as most of my discoveries here are through trial and error.

joining "through" associations

Say you have a three-layer association. The following would be an example of this:


class Company < ActiveRecord::Base
  has_many :employees
  has_many :children,:through=>:employees

  validates_inclusion_of :industry,
                 :in=>[“TECHNOLOGY”,”FINANCE”,”AGRICULTURE”]
end

class Employee < ActiveRecord::Base
  belongs_to :company
  has_many :children
end

class Child < ActiveRecord::Base
  belongs_to :employee
  has_one :company,:through=>:employee
end

So a company has many employees, and an employee has many children. This is pretty straight-forward. Now, let's say that we want to find all the children who's parents work for any company in the “TECHNOLOGY” industry. How could we accomplish this? Well, certainly one way would be to iterate over the associations, starting by finding all the technology companies, then iterating over each employee, and adding their children to a growing array. It would look something like this:


children = []
Company.find_all_by_industry(“TECHNOLOGY”).each do |company|
  company.employees.each do |emp|
    emp.children.each do |child|
      children << child
    end
  end
end

But that's not great. That's a lot of lines of code to read for one qeury. Let's put the meat of this finding algorithm into a named scope instead:


class Employee < ActiveRecord::Base
  named_scope :works_in_technology, 
           :joins=>:company,
           :conditions=>”companies.industry = 'TECHNOLOGY'”
end

children = []
Employee.works_in_technology.each do |emp|
  emp.children.each {|child| 
    children << child 
  }
end

That's a little better. You can add in joins to your query just by giving the symbol name of the association, so that's really nice looking. However, as long as we're joining on associations, why not just have another named scope in the Child model and do this whole thing without any iteration?


class Child < ActiveRecord::Base
  named_scope :parent_works_in_technology,
              :joins=>:company,
              :conditions=>
                ”companies.industry = 'TECHNOLOGY'”
end

children = Child.parent_works_in_technology

Hey, that's a lot better! Only one problem: it doesn't work. Yeah, we do have an association between the child and company models, but it's “through” the employee model, and the named_scope doesn't know how to assemble that query properly (try it, you'll see the sql it generates isn't quite right). We can still do this as one named_scope join query, but we're going to have to get a little more detailed level of control:


class Child < ActiveRecord::Base
  named_scope :parent_works_in_technology,
              :joins=>”, employees, companies”
              :conditions=>
                   ”employees.company_id = companies.id and 
                    children.employee_id = employees.id and
                    companies.industry = 'TECHNOLOGY'”
end

children = Child.parent_works_in_technology

Now THAT works just fine. It's a little more verbose in the named_scope declaration itself, but it's really nice for wherever else you need to use that query in your codebase. Notice that you have to do the foreign key joining yourself in the conditions string, don't forget to do that or you will definitely not get the results you are expecting.

Watch out for duplicates in join queries

So now we're joining across multi-layer associations successfully, and now we have a new requirement: We're rolling out a new children's special to all the companies in our system and we need to get a list of all the companies who have employees with children younger than 12. Based on the last named_scope we built in the child model, it seems like we could do another one just like it in the company model.


class Company < ActiveRecord::Base
  named_scope :has_employees_with_young_children,
              :joins=>”, employees, children”
              :conditions=>
                     ”employees.company_id = companies.id and 
                      children.employee_id = employees.id and
                      children.age < 12”
end

companies = Company.has_employees_with_young_children

If you don't know how table joins work, it might seem like this would be fine. The problem comes up when you have a company that has more than one child under age 12: because you're asking it to return you one record for every match of the conditions, you'll get a company record back for every child at that company that's under 12! If “Innitrode” has 50 employees, and those 50 employees have collectively a total of 75 children under age 12, then this named_scope will return 75 innitrode records rather than the 1 you were expecting. This certainly would not be the behaviour you were looking for. Enter the “group” option. Grouping allows you to take the set of records you get back and compress them depending on a matching value in each one. For example:


class Company < ActiveRecord::Base
  named_scope :has_employees_with_young_children,
              :joins=>”, employees, children”
              :conditions=>
                  ”employees.company_id = companies.id and 
                   children.employee_id = employees.id and
                   children.age < 12”,
              :group=>”companies.id”
end

companies = Company.has_employees_with_young_children

Presto, now the results you get back will be flattened so that each record with a company id of n is grouped into a single record, so you will get no duplicate company records in the array you have returned.

ReadOnly records

So now you're using these great joining named_scopes to find records based on all kinds of cross-table conditions, and things are working great. However, they will continue to do so until you need to update one of those records you retrieved, and then you will get a bit of a surprise when your save method throws an error saying that the record you are trying to update is read only.

Actually, there's a very good reason for this, you just have to know a little bit about how active record works. When you run a query with joins in it, you're getting back single records that have WAY more attributes than just the ones found in the model table you're dealing with. If you join employees to companies, than your employee object will have a hash of attributes that has all of the data in it's table, PLUS all the data in the table for the company he works for. So what if you update one of THOSE attributes instead of the ones that are part of the employee table? The update statement to the employee table wouldn't be able to find the column named “corporate address” or whatever, and would crash. So, the records that are returned from a joined query are just marked readonly to prevent that from ever happening. In a rails action, this would rarely be a big deal because you're usually just passing the ID of the object you want to update, plus the parameters that you need to change, so you'd be loading a new record out of the database anyway based on the ID, and that object will be writable. However, it's important to know how to combat this in case the situation every arises where you DO need to update one of these records. There are 2 ways that I can think of around it.

First, you can just reload the record you want to update by running another find just using the ID of the object.


  company = Company.find(company.id)

Another approach that works is to update your named_scope declaration to use the “select” option:


class Child < ActiveRecord::Base
  named_scope :parent_works_in_technology,
              :joins=>”, employees, companies”
              :conditions=>
                ”employees.company_id = companies.id and 
                 children.employee_id = employees.id and
                 companies.industry = 'TECHNOLOGY'”,
              :select=>”children.*”
end

Now that you're only pulling back the attributes specifically from your model table, the objects won't be read only. Be careful with this one, though, because you'll run into a runtime error if you try to use anything that modifies the query like the “size” method.


children_count = Child.parent_works_in_technology.size

This is going to alter your query to call “Select count(children.*)”, and your database will reject this as invalid.

Chaining Join Queries

Finally, what if you have a few named scopes on the same model that join the same tables? Is this a problem?


class Company < ActiveRecord::Base
  named_scope :has_employees_with_young_children,
              :joins=>”, employees, children”
              :conditions=>
                ”employees.company_id = companies.id and 
                 children.employee_id = employees.id and
                 children.age < 12”,
              :group=>”companies.id”
  named_scope :has_employees_with_male_children,
              :joins=>”, employees, children”
              :conditions=>
                ”employees.company_id = companies.id and 
                 children.employee_id = employees.id and
                 children.gender = 'M'”,
              :group=>”companies.id”
end

It's not a problem as long as you don't chain them together. But as soon as you do this, you're in trouble:


  Company.has_employees_with_young_children.has_employees_with_male_children

You're SQL query will now try to join in both of those other tables twice, which will result in ambiguous column references. Usually the answer to this problem is just to not chain joined queries together, but if you need to (like I did in one particular case), you can have the join itself be a different named_scope, so that you only run the joins once:


class Company < ActiveRecord::Base
  named_scope :has_young_children,
              :conditions=>”children.age < 12”
  named_scope :has_male_children,
              :conditions=>”children.gender = 'M'”
  named_scope :join_in_children, 
              :joins=>”, employees, children”
              :conditions=>
                ”employees.company_id = companies.id and 
                 children.employee_id = employees.id”,
              :group=>”companies.id”
end

This will work, you just have to remember to chain it in every time:


   Company.join_in_children
          .has_male_children
          .has_young_children

Hope this helps someone. Any reader with other tricks they've learned with joins, or better ways to do something listed here, put it in the comments and if it's good I'll move it to an update in the post (with credit given to the author, of course).

Git solves all your branching and merging problems! Almost!

2009-05-11T12:25:00.009-05:00

The entire development team here at Research to Practice (that is, Ray and I) have been working entirely out of git for a little while now. The problem was, we were using it just like Subversion. You see, having experienced a bad branch merge before, it's tough to get up the gumption to try again.

Why? Imagine getting food poisoning from a questionable sushi place in the midwest. Yeah, you know that it was totally the health code violations and 12-day-old-fish at that place that made you vomit 9 shades of purple, but you'd still have a hard time bringing yourself to eat anything at an upscale sushi restaurant on the coast of japan, wouldn't you?

Yeah, it's kind of like that.

We know that git is good at this stuff, we know that we want to take advantage of the great flexibility that using branches offers, but when it comes down to actually DOING it, there's always a reason why maybe we shouldn't try this just yet.

Last week, we decided to lay down the law. We wanted to have some features be worked on for a 2.0 release of our website, but we also wanted to continue deploying fixes and small enhancements to the currently running application. No way around it, it was time to branch out.

So it was with great fear and trepidation that we first unleashed this phrase upon our command line:


$> git checkout -b version_2_features

Oh Gods, what have we wrought?!

...Ok, so it wasn't that bad yet. The checkout command with the "b" option created a new branch for us, and after developing a small and inconsequential feature (a necessary precaution because we were fully fearing that the demons-of-parallel-development could swoop in at any moment and destroy everything we had done so far) we pushed this branch to the github repository so that we could both develop on it:


$> git push origin version_2_features

Alright, so far so good. Now we have our "master" branch, which was esentially "What is currently running in production", and our "version_2_features" branch where we could start developing everything we wanted to put into the version we were planning to release at the end of the summer.

Great Success!

So we started developing that way, and everything was moving along splendidly, until we hit something of a snag. Well, a boat anchor, really. See, the whole point of having this 2.0 branch be separate was so that it's content would not be deployed until much later, but SOMEhow, SOMEone, did SOMEthing (I'll let you ask Ray for the details behind that), and suddenly the version_2_features branch ended up being accidentally merged into master.

This resulted in an hour of us assuring ourselves that we could probably find a way out of this without having to admit we (he) ever screwed up, and we came up with a list of options.

1) Use git reset on the repo to bring it back to before the bad merge
2) Check out a previous commit version (before the bad merge), and then commit that as the head
3) Use git reset on a local repo, and push the result to github
4) Revert the merge commit, than when you want to merge for real later revert the revert.

Now out of the first 3 options, it turned out that exactly 0 of them were possible. There's no way (that we could find) to call "RESET" on our remote github repository, checking out a previous commit disconnected you from the head so you couldn't commit and call it the new head, and trying to push a branch that had been reset to the remote repo was met with a regular level of failure. Option 4 looked plausible, but beyond our ken, and neither of us is a git wizard just yet.

So we went with option 5, check out an old version of the repository, copy the entire directory structure into our working directory, commit the deltas, and bam, we're back to premerge state.

NEVER DO THIS

Little did we know at the time that this would later take us beyond regular and straight into the epic level of failure.

"What's the matter?" you might ask. "Isn't your directory back to the way you wanted it?"

The answer to that question is yes, our working directory was back to the production version, which was great for the master branch....but not for the history. You see, we still needed to be able to merge features that were being added to the version_2_features branch back into the master at some point in the future, and that's really hard to do if your repository version believes that those 2 branches have already been merged!!!

By the time we realized this, we'd already pushed these changes to the repo, and were beginning to consider just how screwed we were. Thus began the wailing and gnashing of teeth.

"We were right! Branching was a horrible idea! We've done exactly what should never have been begun!"

So we rebased and merged and copied and merged and went through every combination possible of recursive/3-way/octopus merging until we finally were PRETTY sure that everything was ok (about half a days worth of work), and do you know what I realized after that?

6) delete the remote github repository and create a new one from the one on your local machine that's still in a good state.

Yes. It was possible. It was even a good idea. if only I had thought of it sooner.

Moral of the story? git is great for branching and merging, far better than any solution I've used in the past. Unfortunately, even git can't protect you from your own stupidity. No tool is that powerful (except Java, that's what it was designed for).

On Learning

2009-04-30T07:58:00.002-05:00

In a previous post, I mentioned that in some of my spare time I like to give private music lessons to kids here in Columbia. Well, that part of my life is now temporarily closed as the demands of my business continue to grow leaving me very little "spare time" left over. It's been tough having to tell each one of my students that I'm not going to be working with them any more, because I've really enjoyed working with all of them and have kind of become attached to it, and as I said goodbye to each one I would find myself thinking about how much they had improved since we started working together. Now I'm not going to take all the credit for that, because they are the ones who practiced, so they are responsible for their own gains in skill. However, I feel that something I was doing worked right, better in fact that my previous experiments in the field of teaching.

What was the biggest difference this time around? Well, my approach to teaching in previous years could be summed up by saying "These kids don't know very much yet, so we'd better scale down everything we're working on to make sure it's not overwhelming and just learn a little bit at a time". That could be compared to my approach this year, which was more along the lines of "This kid already knows how to do X, so let's find something they aren't good at yet and specifically pick a piece to work on that requires that skill". Although the evidence is anecdotal, my experience seems to show that people learn better when they are challenged than when they are coddled.

There are three reasons I can think of that support this idea:

1) slow going is boring

Picking up a little at a time is a quick way to lose interest in anything. This is why I still don't speak Russian very well. I really want to learn Russian, but I've been trying to just spend a couple hours a week on it, reviewing many of the words I already know and just adding a few more. You know what? People who go participate in an immersive language program learn fast. People who pick up "Russian for Dummies" and peruse it a few times a week don't, they just get a nice paperweight that looks good on their coffee table.

2) harder to achieve = more satisfaction

Since most of the readers here are programmers, I can feel confident that this point will garner support. If you are trying to tackle something tough that you've never had to deal with before, then your feelings upon success are made up mostly of excitement and joy. If you just finished coding up another CRUD application which you've done 100 of before, your feelings are probably more of relief than anything else. Difficult and new challenges produce satisfaction that makes you want to come back and do more.

3) expectations cause results

This is probably the most important thing a teacher can do. If you expect your student to make minimal progress, and you convey that expectation in the assignments you give and in your assessment of their work, than they will definitely make minimum progress. If, on the other hand, you expect more out of him than even the student himself thinks he is capable of, he will tend to make strides that would surprise anyone because when he is working he is bolstered and driven by the fact that somebody else who has authority in this subject believes that he is capable of doing what he's been assigned. It's amazing, but true (at least in my limited experience).

So as I said, I'm sad to leave this part of my life behind for now, but the lessons that I have learned from it are worth taking with me into other parts of my life. If I want to pick up something new, I think the best way to go about it would be to pick up a challenging project (for my level), dedicate a good amount of time to it, and expect more out of myself than I think I'm capable of. In fact, I'll go so far as to say this would be a good way for you to learn anything too.

Net::HTTP and SSL

2009-04-28T14:32:00.002-05:00

Just ran into an interesting problem today that I thought I'd post the solution for. Everybody knows about the net/http library, it's a core bundle that lets you do web requests. Well I was using it for posting to an experimental web service a friend had set up, and I ran into a problem. Here was my code:


url = URI.parse("https://someaddress:443/web/service/path")
xml = generate_some_xml
request = Net::HTTP::Post.new(url.path,{"Content-Type"=>"text/xml"})
http = Net::HTTP.new(url.host, url.port)
response = http.start {|http| http.request(request) }

Seems like it should have worked, but I kept running into this message:

wrong status line: "\025\003\001\000\002\002"

after googling around a bit, I figured out that this error message comes up whenever you are trying to connect to an SSL enabled site with plain text. I figured that, like curl, if you used "https" in your url, then you were telling the library that you wanted to use SSL. Well, that's not exactly the way it works. Although I put my "https" into the original url, when it gets passed into the HTTP library on line 4 of my code up above, I'm just handing it the host and the port, not the protocol. This requires the use of an aditional line of code:


url = URI.parse("https://someaddress:443/web/service/path")
xml = generate_some_xml
request = Net::HTTP::Post.new(url.path,{"Content-Type"=>"text/xml"})
http = Net::HTTP.new(url.host, url.port)
http.use_ssl = true
response = http.start {|http| http.request(request) }

there you have it. Hopefully by finding this post, you figured this out a lot quicker than I did.

Improving your Rails-Fu

2009-04-24T18:33:00.003-05:00

This month I've been making a conscious effort to improve my competence with the Rails stack. It's not that I feel I'm a bad developer (nobody feels that way, not even bad developers), it's just that the last time I'd been really honing my craft was around rails 1.2, and it's been a while since then (2.3 just released recently). I knew there were some major changes out there that I wasn't taking advantage of, and that there were several things I could improve upon. This was evidenced both by my lack of experience with things being discussed on the rails core list, and the reluctance of other developers to contract in for work on my codebase. Something had to give.

So I went to the same place I always go when I want to work on my skills: The Pragmatic Programmers. I'm not affiliated with these guys in any way, I just really like their stuff. In this instance particularly, I selected a series of screencasts called "Everyday Active Record" done by Ryan Bates of Railscasts fame. Sort version of the story: they were excellent.

I have been a subscriber of the Railscasts feed for sometime, so I knew that Ryan was good at this stuff, but I learned quite a bit from this series in particular that made me slap my forehead as I thought about all the problems they were discussing that I had gone and wasted time trying to solve myself when it was BUILT INTO ACTIVE RECORD THE WHOLE TIME.

A good example of this was named scopes. If you are a Rails developer, you've probably heard of them already, and you're right now navigating away from my article. That's fair, I should have been keeping up better with the industry. Believe me, it didn't take me long to dig into my code and start name_scoping everything more complicated than matching on one column.

If you are like me and HAVEN'T exactly done a stellar job of following the developments in the rails stack (this was introduced in Rails 2.1), you might not know what I'm talking about. Let me point you to a good article on the subject. Read it. Cool, huh?

There's lots more, with easy to follow examples, and at $5 a pop they're a pretty great deal. Do yourself a favor and check them out.

DRY up your views with HAML

2009-04-17T19:59:00.003-05:00

About 6 months ago, I read an article about this great templating engine called HAML. I thought I might try it out. I didn't.

Giles mentioned it back in January. I read that article. "Oh yeah!", I thought. "I read about this! it's cool, I should give it a try!". I didn't.

Daniel Morrison gave it a review in February. I read that article too. He said it makes html unmaintainable for non-rubyists. I thought I should see for myself. I didn't.

Yesterday, Ray Myers (employee #1 of my startup) mentioned that he had found this thing called HAML and wanted to try it. I tried it.

Good things happened. My templates got short. I liked reading my markup. Everything was concise, kind of like this blog post.

Wish I hadn't waited so long.

TRY IT.

Profitable on Paper

2009-04-09T23:02:00.002-05:00

This year has been my first year as a startup founder, so I'd never attempted anything like this before. In my previous jobs, I got a paycheck every 2 weeks, just like clockwork.

Now, being in the midst of a cash crunch, I sometimes miss that security a little. Not enough to want to go back to being chained to an office working on something I don't care about, mind you.

You see, my company has performed services, and invoiced for them, in amounts that would keep our heads above water. However, some of our clients tend to take upwards of 90 days to pay up. If you consider receivables our situation looks pretty great, but without cash in the bank it means almost nothing.

I wrote a post a few months ago that said "Customer Service is King". I think that in light of new information, I'd rank Customer Service as more of a prince or duke. The real king is cash, and managing it's flow is a major art form.

Some things I've learned about managing cash:

1) slash expenses everywhere, all the time. Any money you aren't spending is money you will have in reserve for things you might suddenly be forced to pay for. Don't cut corners, though. Angry customers soon become non-customers.

2) You are an expense. The less you can live on, the better for the long term.

3) Early on in a startup, cash is scarce, so your time is less precious than your money. Spend time negotiating deals or discounts. It's well worth it.

I'll post others as I think of them, but the main thrust here is that as long as you have less than 6 months of operating expenses in reserve, one of your main jobs is to watch what money you have like a hawk.

Big Headaches for MYSQL users

2009-03-27T08:09:00.006-05:00

Here's something I didn't know, but which hopefully will help someone else out in the future:

DON'T USE NESTED SUBQUERIES!

Ok, if you know what that sentence means, and why I said it, you're done with this article, continue to the the next item in your RSS reader.

Now, for all the rest of you, let's start with what a nested subquery is. Let's say, for example, I have 2 tables in my database: students, and schools. Students each have a "school_id" as a foreign key. I want to retrieve from my DB all the students who go to a school that is in the zipcode 65203. I might write a query like this:

"select * from students where school_id in (select id from schools where zip = '65203')"

Or if I was working in rails (which I usually do):

Student.find(:all,:conditions=>"school_id in (select id from schools where zip = '65203')")

It looks really nice, it's easy to understand, and under certain circumstances it can DESTROY your database performance. There's a lot of reasons why this is the case, not the least of which is that query optimizers just don't have the time to figure out each one of your subqueries and fix them up. If you want a nice discussion about the why, I recommend This Blog Post wherein Scott Selikoff does a great job of explaining the problems.

For those of you who just want to fix it, do a join. Yeah, it's a little less pleasant to look at, but your hosting provider will thank you (or, in our case, will stop yelling at you for hurting their other client's database performance times). Here's an example of the previous query joined up:

"Select students.* from students, schools where students.school_id = schools.id and schools.zip = '65203'"

And the rails version:

Student.find(:all,:conditions=>"students.school_id = schools.id and schools.zip = '65203'",:joins=> ", schools")

May your DB feel better, and may you stop getting angry emails! Happy coding!

Opportunities where you least expect them

2009-03-25T15:36:00.004-05:00

So for the last year I've been writing about software, trying to become better at what I do. In the process, I like to think I've become a better writer as well. Apparently, there are other people who think so too, because this last week I was asked to become a featured writer on firelink.com (related to my new experiences as a recruit firefighter). Check the link!

http://www.firelink.com/benefits/articles/6802-firefighter-origins?page=1

What you say and how you say it

2009-03-20T10:29:00.003-05:00

I was sitting in on a meeting with my wife and one of our business service providers a couple days ago, and an interesting thing happened.

We were meeting with two individuals from this company, both from the same department, both with approximately equal authority to get things done. We talked about some problems we were having with their software system, and they told us about the schedule they'd be able to address it on.

After we had left and were driving home, my wife turned to me and said "Which one of those two ladies did you like more?"

I hadn't thought about it, but I immediately said that it was the woman on the left.

"Me too!" my wife said.

We talked about it for a while and realized that both of them were telling us the same thing: "We understand your concerns, we're a little backed up, but we'll get them addressed within X timeframe". However, the woman on the right had mannerisms that were off-putting. She had her arms crossed the whole time, every time we tried to bring up an issue she was quick to come into direct conflict with it, she never smiled. The woman on the left, however, leaned in towards us when she talked, let us finish our sentences, smiled generously, and gave us phrased things with a "we" instead of a "you" in front of them. The first woman seemed to be accusing, the second was accommodating.

The first woman made me want to find another company to work with. The second made me want to return the favor sometime in the future.

It's not the message you're giving that typically affects peoples opinions of you, it's the way you deliver it.

3 Stereotypical Types of Software Labor

2009-03-13T14:32:00.004-05:00

The last month has been really hectic here for both startups that I'm working on. With more and more that needs to be done, there's less and less time for me to get to it all, so the time finally had to come when I said "There's too much for just me to be working on". So we decided to hook up with some contract labor to bridge the gap until we could get a full time employee brought on, but the results have been a mixed bag. That's why I thought I'd put up a post that shows the experiences we've had with the 3 most common types of contract labor. (Please note ahead of time: This is just my honest experience. It does not automatically extrapolate to every person who falls into the categories I will define.)

1. The Cheap Off-Shore Worker

Cool! Software development for less than $10/hr! We couldn't find ANYBODY here to work for that price. When you're in a startup, cashflow is always at the front of your mind, so there's a big appeal to a little less value for a lot less money. When we found a gentleman in India who was willing to work for around half of what we would have to pay local labor, we couldn't pass up the chance to at least try it out.

PROS: Cheap is good. We got a lot of features started that had been waiting on the back-burner while I worked on more important stuff, and for the most part what came back to us was in working order.

CONS: It can be tough to explain what you're asking for when there's a language barrier. Also, frequently the finished work that we received had small problems with it that were not difficult enough for us to want to keep paying someone else to go back in and fix them, but were bothersome enough that they took some of my time to fix before we could deploy them to production.

VERDICT: Good for getting little things done, or maybe for giving you a place to start on larger features. Not to be relied upon for anything mission critical to your application. ALWAYS look over the code yourself before blindly including it in a deployment.

2. The Overpriced Consultant

I had one guy contact me who was very experienced and had a lot of great items on his resume. The problem? He wanted $100/hr to work (but was willing to start at $75/hr). Are you kidding me? Don't get me wrong, I'm sure this guy's very good at what he does, but I wasn't kidding when I mentioned cashflow earlier on. I don't care if this guy has six arms and 3 workstations that he can hack on simultaneously, there's no way he could produce anything that would be worth $100 every hour to a cash-poor startup operating out of the mid-west.

PROS: The gentleman was very nice to talk to, very experienced, and if we had worked with him I have no doubt that he would have produced excellent code and done fine work.

CONS: We have only so much cash, are barely profitable, and any expenditure at that level would quickly decrease our run-way of operation. We just can't afford that kind of recurring expense.

VERDICT: Better leave these blokes for the deep-pocketed mature companies.

3. The Local Student

The dude's never had a shot in the real world yet. Great! He's got lots of academic training, he's probably quite bright, but he won't charge as much as someone who's been in the industry for some time. What a deal, right? Well, maybe. You never know someone's ability level until you've tried them out. In my experience, the huge amount of intelligence you may have in a bright kid doesn't really translate to a great deal of productivity. Why? Well mainly because part of the experience you get from the industry is knowing what things have already been implemented and NOT REWRITING THEM YOURSELF. I don't care if you can implement a super fast bloom filter, what we need right now is the ability to upload pictures for the user's profile.

PROS: Pretty cheap, that's good. Also they're usually on-site, which is to me much easier than working with a remote developer.

CONS: Be prepared to watch every check-in that is made to the repo, because they probably haven't used attachment_fu yet and they may just decide to write their own image attachment library (which is probably very good, but we just don't have the time for that!).

VERDICT: Can consume as much time in supervision as they save you in implementation.

My conclusion: The best contractor we've had is the guy we're going to hire full-time anyway. There's no such thing as a bridge. My impression is that I want the same thing in a contractor that I want in an FTE, but I'm less likely to find it in someone who's not going to have a stake in the outcome. Thank God we're hiring.

a little git

2009-02-13T10:12:00.002-06:00

In a previous post I mentioned that I had switched my SCM solution on the startups I'm working on to "git" away from "svn". Up until now, though, I hadn't really been making the most use of the new powers git gave me. My workflow had been very similar to the way I used to do svn:

checkout the repo (git checkout)
make changes
run pull in any changes that have been sent to repo during my work time (git pull)
commit my changes to the repository (git push)
repeat

Recently I had a need to do more, though. There's one section of one of web applications I'm developing that needs a serious redesign (I mean several days worth of work). So I'm working through it, but then I have a new "emergency" that comes in that requires I deploy some code NOW. Usually, I could solve this problem by making the change in my working directory, and then just selectively adding and commiting that change. However, some of the work I needed to do was in one of the files I had already edited, so there was no way without losing all my work for the day that I could swing getting this fix out (short of checking out from head to a different folder, changing, pushing, and then pulling in to my modded directory again).

Fortunately, git provides a better way:


$> git stash save

I've just saved all my local modifications to the "stash", which you can think of as a temporary shelf you can set stuff on while you're doing branching stuff.


$> git checkout -b working_branch

Now I've created a new branch named "working_branch" on my machine from this repository.


$> git stash apply

My changes that I saved to the stash have now been resurrected and applied to the new branch that I'm working in. I can now make changes and commit to the "working_branch" branch as often as I need.


$> git add .
$> git commit -m "commiting to branch"

Now I need to go back to my master branch to make the emergency change:


$> git checkout master

OK, make my emergency fix, then commit:


$> git add .
$> git commit -m "commiting to master"
$> git push origin master

Now maybe I want to merge these emergency changes into my current working branch so that I can keep the two branches from diverging to far:


$> git checkout working_branch
$> git merge master

Resolve any conflicts, and commit. Done!

There you have it, simple branching in git for great gains in organization and productivity (IMHO).

PDF Attachments with attachment_fu

2009-02-09T15:22:00.002-06:00

The great thing about frameworks like Rails (in my opinion) is that the large number of people doing many of the same tasks results in some really great reusable pieces of code that make common complex features almost trivial to implement. This was brought right to the forefront of my attention again today when I started working on a new feature for our medicaid billing system.

The story is this: We have a physician who writes prescriptions for students who need them. It's better if we don't have to mail him all the documentation for every student he's writing a script for. The best conceived solution was to have school districts upload their docs to our website (associated with a student), and let the physician download them on his side.

Now, surely I'm not the first person to ever want to have uploaded files associated with an entity (that's basically facebook's whole business model, after all).

Sure enough,
attachment_fu to the rescue!

This was dead simple. After installing the plugin (ruby script/plugin install git://github.com/technoweenie/attachment_fu.git), I dug into the docs and started coding.

First I created a new model to house the necessary file for the student (it's called an "IEP", for Individual Education Plan, so don't get thrown by the acronym):


class Iep < ActiveRecord::Base
  belongs_to :student
  
  has_attachment :content_type => "application/pdf", 
                   :storage => :file_system, 
                   :path_prefix => "public/iep",
                   :max_size => 1000.kilobytes

  validates_as_attachment
  
end

Small and simple, right? Here's what it means:
the "belongs_to :student" is the standard ActiveRecord association, indicating that a student entity is the parent of this object (and also implying that the "IEP" object has a foreign key called student_id).

"has_attachment" is the method that configures the file you're going to upload. :content_type, obviously, is the type of file you expect. In my case (and yours if google brought you to this entry) that type is "application/pdf", and if anything else comes in it won't be accepted.

":storage" is how you tell attachment_fu where you want to store these files. Other options besides the filesystem are the database and amazon_s3, but those are both beyond the scope of this article. I just wanted to get these items onto the disk, and then back off on another computer, and the filesystem was the way to go that required the least configuration.

":path_prefix" is the file path you want prepended to the storage location for all files stored by attachment_fu. In my case, "public/iep" is just a directory on my local filesystem, but in production it's a symlink to a location outside of the application folder so that the uploaded files will persist across new deployments (handy strategy for anyone who needs to upload files, thanks to EngineYard for handling that for me).

":max_size" should be self-explanatory. If it isn't, you may be in the wrong job.

Next I set up a couple controller actions to handle the actual uploading of the file:


def upload_iep
  @student = Student.find(params[:id])
  @iep = Iep.new
end
  
def exec_upload_iep
  @student = Student.find(params[:id])
  if @student.iep.nil?
    @iep = Iep.new(params[:iep])
    @iep.student_id = @student.id
  else
    @iep = @student.iep
    @iep.update_attributes(params[:iep])
  end
    
  if @iep.save
    flash[:notice] = 'IEP Uploaded!'
    redirect_to :action => :all_students
  else
    redirect_to :action => :upload_iep, 
                :id => @iep.student_id
  end
end

It ain't restful, but it works. One action to load the form, the other to accept the actual upload. Nothing in there should look different than dealing with regular rails forms. Next we have the view:


<% form_for(:iep, :url => {:action=>:exec_upload_iep}, 
    :html => { :multipart => true }) do |f| -%>
  <p>
    <label for="iep">Upload An IEP:</label>
    <%= f.file_field :uploaded_data %>
    <%= hidden_field_tag :id,@student.id%>
  </p>
  <p><%= submit_tag 'Upload' %></p>
<% end -%>

The only important bits to remember here are the ":multipart => true" which makes sure that the form can accept file uploads, and the ":uploaded_data" attribute which is what attachment_fu searches for when manipulating attached files. Now it's easy to upload an IEP attached with a student anytime. And the best part is, retrieval is a snap:


<td>
  <a href='#{student.iep.public_filename}'>Download IEP</a>
</td>

that "public_filename" method returns the necessary path to retrieve the file, making it easy for the physician to download the IEP on his computer. Done and done. Thanks to technoweenie for writing and maintaining this plugin for the community!

Yahoo killed my buzz

2009-02-04T11:47:00.003-06:00

So I was having a really good week until today. Billing is coming along nicely for our medicaid project, and the other startup I'm working on will be demoing at the Lake on saturday (I'll post a link on this blog after that date, so that I can get feedback from all of you readers as well).

Unfortunately, today my experience with Yahoo really destroyed that "high-on-life" feeling I'd been having. You see, we had this idea for handling incoming email with the application. I wanted to have users be able to send email to the app that would cause things to happen. So I went to our friendly and efficient hosting provider, EngineYard, and asked them to setup a mailbox for us to accept incoming emails. Like usual, engineyard's customer support got done what we needed and did it with a smile. They then told me through email that in order to start receiving email there I would need to add an MX record to my DNS listing that would make "msg.domain.com" point to their mail server. Basically, I just wanted that subdomain to be redirected, while leaving all regular mail sent to "domain.com" to continue as normal.

Figuring that this would not be an issue, I asked our business expert to take care of it. As the title of this post probably indicates to you, it WAS AN ISSUE. This poor woman spent the first half-hour of this project playing with Yahoo's web interface for their small business registrar system. No dice. She called them and spent half an hour waiting, and then longer actually on the phone with a customer service representative who didn't understand what we were asking for. I got on the next call with her and together we talked with a Yahoo support guy named "Phillip" who told us in an apathetic voice that what we were wanting to do simply wasn't possible.

Bullshit! I'm willing to accept something along the lines of "I don't know how to do that", because there are plenty of things that I don't know how to do. But you're going to say to my face that something is downright impossible you'd better be sure that I'm not going to be able to embarrass you with something as simple as a google search: I have Many Online Resources that say this is a perfectly reasonable request that is implemented in DNS records all over the world all the time. Knowing that I have to demo by Saturday, I'm probably just going to have to work around this by directing all email traffic to the engineyard server and forwarding the regular stuff back to our personal email addresses, so I can get past this, but what a freaking drag!

Long story short, don't be suckered in by Yahoo's low prices for domain names. They will screw you if they get the chance. As soon as this demo is over, we're switching. UPDATE: I just checked, and GoDaddy supports this kind of MX record, so that's who we'll be switching to. I recommend you all do the same.

Variable Naming Matters!

2009-01-30T10:48:00.002-06:00

I woke up this morning with a production bug on my plate. Those of you who read this blog regularly probably know that one of the startups I'm working on is a medicaid billing software suite for school districts. The problem I was having was that there were some transactions being added to our billing batches that shouldn't have been. See, in order to bill a transaction to medicaid, several things must be in place: the student must have a prescription, the therapy provided must be one of the federally approved therapies, the therapist must have a federally issued NPI number on file, etc.

These transactions were failing validation, obviously, because they were invalid. They should never have been billed in the first place. When I dug into my code, I couldn't see the problem, although based on the title of this post you will probably see it immediately:


def get_student_detail(s)
  group = s.appts.select{|sa| s.is_medicaid_eligible} 
  if group.size > 0
    claim = Claim.new(group,{:batch_id=>self.id})
  end
end

I spent a lot of time on this. I wrote more unit tests, couldn't figure out why they were failing. I built mocks and tested many underlying methods to prove that they were working properly. Eventually the only way I figured it out was when I kicked up an "irb" session on my production box and started stepping through this code one piece at a time. what an embarrassment when I realized what I had done.

Yes, I went and named my "student" object instance "s". And then I named my "StudentAppointment" object instance "sa". And they both have a method called "is_medicaid_eligible" that returns a boolean value. So, if the student is eligible, all his appointments get billed regardless of whether those appointments are eligible or not, because I mistakenly wrote "s" instead of "sa" in the block on line 2 of the above code. There are plenty of arguments out there about naming conventions and whether they really matter, and I may not be qualified to participate in many of them, but here's one lesson I won't soon forget:

if I had used the name "student" for my student variable, and "appointment" for my appoitment variable, I probably wouldn't have spent an hour trying to find this bug.

Breaking my own Rules

2009-01-27T15:15:00.004-06:00

OK, I'm going to post a link to somebody elses blog post. I almost never do this, because I usually feel like there's a comment section on the other person's blog for a reason, so if you have something to say you should "Say it to their face" so to speak. But in this one case, what I read made so much sense that I had to share it with my audience:

http://www.defmacro.org/ramblings/taming-perfectionism.html

Check it out, and note especially the very last sentence.

Permanent bug fixes

2009-01-24T15:52:00.002-06:00

One of the startups I'm working on right now involves a lot of SMS communication. Threading and queueing messages, although not difficult, can present some edge cases that need attention. One such case was brought to me this week when the application seemed to freeze and error out after a certain message type was sent, followed immediately by a cancellation of the same message.

When presented with a bug, my first instinct is always to "think through" the code that I believe is the most probable culprit. This is usually not the best approach. Yeah, that's an opinion, and I only have anecdotal evidence to support my viewpoint, but this is MY blog so if you don't want to hear my opinion you don't have to read it.

To me there are two important components of a GOOD bug fix. The first is natural: make sure the error/undesireable/unpredictable event stops happening. The second is too often overlooked: make sure it doesn't happen again. Because of the second point, and also because you don't always have a stack trace to show you where the problem is (if, for instance, the problem is not an "error" but an undesireable outcome), I think it's important to suppress the "dig in and find it" urge, and to instead start writing tests. Why? 3 reasons:

1) unit/functional tests allow you to precisely recreate the circumstances under which a bug occured (or under which you believe it to have occured). This means that if you don't yet know what's causing the problem, you can use assertions at every step along the way to make sure every bit of state maintained in the code is as you would expect. In the case of my SMS processing bug, there were many many objects involved in the transaction (because various notifications had to be delivered) and no error was being given so I had no way to know instantly where my data was being monkeyed with. If I wanted to do an "intuitive" diagnosis (as my insticts would have me do) I'd have been dealing with maybe 60 lines of code to look through. By setting up my test case and writing assertions at every point, I was able to isolate the place where the problem was occuring (where my first assertion failed) and thus the number of lines of code I had to apply my diagnosis-thought-process to was exactly one. It's always good to reduce the problem space anyway possible, it can only make your job easier. (As a bonus, if you're using test-coverage as a metric, then you're improving your numbers WHILE bug-fixing)

2) By using tests to diagnose the problem, you also have a feedback mechanism that will tell you as soon as it's fixed. Isn't that handy? When the test is passing, the bug is dead. You might write a few more tests afterwards if this bug has opened up a new path in the code that needs coverage, but basically you won't have to go through the process of "Let's try this, deploy it, did it help? no? Ok, what about this, let me deploy the website again. How bout now? No? Damn! OK, one second". You still might be following the same path of "try this, try that" but your feedback iteration is reduced significantly. Running your test suite takes a lot less time than redeploying the code over and over again.

3) Never experience the same bug twice. With that test in place (as long as your team is disciplined about running the test suite before deployments) that bug is dead for good. Were anyone to adjust the code to the point where it would reoccur (something I've seen happen on more than one project), the test will fail, stopping the release cycle until the problem is solved. You shouldn't ever have to solve the same problem more than once.

When it comes to pest control, your number one weapon should be an up-to-date suite of unit tests.

Advanced Reporting with Excel in Rails

2009-01-19T16:14:00.002-06:00

This is a bit of an expansion and synthesis of two previous posts, one referring to Rails Reporting in Excel, and one dealing with Processing large amounts of ActiveRecord Data. I had to combine those two skills today as our database size is increasing to the point that some of our larger user reports are having to aggregate more data than is pleasant to store in memory at once. In the interest of providing a recipe that others could use, I've pasted an example of this approach on Gist. Essentially, it uses a pagination library (paginating_find, in this case) to page through result sets for reports, limiting the amount of memory space being taken at any one time. Check it out, leave a comment if it helps you.

Simplify, Simplify

2009-01-16T16:15:00.007-06:00

I was presented with a challenge in my work today, one which I was not immediately ready to solve. One of the startups I am working on is a google maps mashup, and a certain requirement we'd decided on earlier involved having certain "markers" on the map blink so that the user's attention would be drawn to them. The following is a sort of "stream-of-conciousness" recording as I thought about making an image on my google map blink:

"OK here's my image:"

"I need to make it blink. It's part of a GMarker object (from the google maps API), so maybe there's some sort of animation API call I don't know about yet....Let's go check the Docs [navigate to google maps API docs].....OK, there's a hide method and a show method, those might be useful.....Not much else though.....I guess I could maintain an array of all the markers that need to blink, and use the "SetInterval" javascript function to toggle their visibility once a second.......yeah, that would work, but if I have a hundred or a thousand markers it might slow down a bit.....still, workable....maybe I'll google and see if anyone's done this kind of thing before....[google for "GMarker blink"].....now we're talking...cool, there's like three open source libraries that actually add a "blink" method to the GMarker class, that's handy....OK, so I just need to plugin one of these libraries and then call the "blink" method only on the ones I want to blink, seems pretty simple....It's going to add a whole new javascript file, but hey, that's not really much of a load time drag....."

By this point I had already constructed the code in my head to an extent, and I was ready to implement it. Then, as I was downloading the library, I noticed something. As I watched the progress bar on this third party website, I was mesmirized by the little circle-spinny-thingy....you know, it was using that usual symbol for "please wait"......the segmented circle that gets highlighted constantly around and around. I always thought that was a cool effect, and I remembered that I'd created that effect a couple weeks ago for one of my other projects. It was pretty neat putting this image together, layer by layer, timing out how long each layer should show for and then saving it as an animated gif.

Then I slapped my forehead, opened up GIMP, and produced this little baby:

Moral of the story, stop and think before you dive into something complicated, there's probably a better way.

Crunch Time

2009-01-13T21:17:00.002-06:00

No matter the industry, or the development methodology, or the size of your company, or the scale of your project, everyone has experienced a time of "push". Call it a hard deadline, or non-negotiable due date, or whatever you want, it all means the same thing: you'll be working extra hard between now and then on little else.

Well, that's where I am right now. We're already sending out marketing information regarding a new product that is theoretically ready but perpetually not quite there, and now the race is on to put the finishing touches together. Stress is high, task list is endless, and I ask myself "Is being part of a startup worth it?"

Certainly there are people who would answer "no" to this question, and you couldn't fault them for it. It is perfectly valid to say "I value my home life enough that I never want my work to encroach upon it", and if that were your point of view than working in a large industry where many people share the responsibility for the success or failure of a project would make perfect sense.

I can't quite go there, myself. It's not that I don't value my home life, it's just that I feel I'm an all around happier person when the things I do at work feel like they have impact: when I have a stake in the outcome, so to speak. I would not suggest that this feeling can't be found in a large company, but I guarantee that when you are the sole technical muscle behind a small and rapidly growing project, EVERYTHING you do carries weight. The cost is just that when it comes to crunch time, you endure it alone.

A price, yes, but one I pay gladly.