Jungli Geek

Unicode encode decode error in python email

2008-04-15T00:24:00.000-07:00

Been bothered with these on and off errors in our application while sending email via python:
File "/usr/lib/python2.4/email/Generator.py", line 182, in handletext self._fp.write(payload)
UnicodeEncodeError: 'ascii' codec can't encode character u'\u2014' in position 4986: ordinal not in range(128)

The reason is that Python's email module needs some information about charset used when a MIMEText object is created.

Here is a link which explians how to solve this:
http://mg.pov.lt/blog/unicode-emails-in-python

Links to understand unicode in Python:

http://effbot.org/zone/unicode-objects.htm
http://www.amk.ca/python/howto/unicode

Enduro3

2008-02-11T20:30:00.000-08:00

So we competed in and completed the Enduro3 race. Sujoy, Sunil and I cycled some 45 kms over mud tracks and hilly roads, trekked another 50, and eventually survived the adventure race unhurt, and still friends. We started around 10:30 am on Friday and finished around 6:30 pm on Saturday, having slept some 4 hrs in between.

The race:
Cycling:
From Paud Road to NDA.
On some muddy terrain inside NDA. Was good fun.
Back on the road, up and down, up and down and up up - ending at Panshet Water Sports Center.
(One of us and his cycle have since been divorced.)

Trekking:
Start from Panshet, walk along the ridge, get totally lost for a couple of hours, reach one end of the lake. (9:30pm)
Stop for the night.
Do river crossing. Only one in the team had to and Sujoy, you rocked then!
Sleep till 2 am. Trussed up in a dryfit t-shirt, the poncho, a full cotton t-shirt, a nice jacket, a monkey cap and inside a sleeping bag. felt good.
Wake and start trekking around 3 am.
Sleep again from 5 to 6 am because we got lost.
Wake (6 am) and trek trek around the lake, on the road, up koshimgarh and trek trek along the ridge till we reached Panshet WSC. (6.30 pm)

The lake and mountains were beautiful. The morning trek was quite refreshing and we saw a lot of birds. The sudden burst of wings from a group of quails from under my feet took me totally by surprise. So close were they, our bodies could feel the drumming vibrations of the frantic wing beats. Also saw a hovering black shouldered kite, a grey jungle fowl, a large green barbet(?), loads of red whiskered/vented bulbuls, purple sunbirds, some flowerpeckers, puff throated, jungle babblers, magpie robins, and rufous backed shrikes.

The race was tiring, and we lost our way more than once. But it was a lot of fun and it definitely tested our endurance and it feels good to have completed the race and be writing this blog.

Information for the next time (IT category):
Water was available at a lot of places and was not a problem.
Even though the race director laughs at you in the briefing session when you ask him about carrying sleeping bags, know that they are absolutely needed.
Do NOT expect flags where you need them.
Finish cycling as fast as possible so that you can navigate properly in daylight on the trek. On a moonless night like we had, it was impossible to look at the lake's shape and find our bearings from the map.
The river crossing is in a deep part of the lake and you have to swim (and not wade) across. You get a life-vest so you won't drown and there are ropes at water level to pull yourself along. The water is placid and does not have a current. The distance is around 100m I think.
The river crossing and rifle shooting timings are only useful when you have a tie. So really don't bother much with these.
The rifle shooting was dropped for our event because of time constraints(?).
Glucose is the best. Food may be taken but you would be better off with chocolate bars and energy bars.
Carry a couple of 1L water bottles per person.
Practise with your own cycle, and from well in advance. Don't fatigue your muscles just a day or two before the race.

Javascript sleep() or wait() in Sahi

2007-12-19T19:09:00.000-08:00

My previous post on 'Javascript sleep() or wait()' is visited a lot of times and I think I should write how the problem was eventually solved in Sahi.

Sahi needs to playback scripts written in javascript on the browser.
Let us consider a small example:

We are trying to automate an AJAXy mail client application which has an Inbox button, which when clicked, loads a part of the page, and this newly loaded content has a link 'Unread'.

_click(_button('Inbox'));
_click(_link('Unread'));

When the first line is executed, the Inbox button gets clicked and the script has to wait till the relevant portions of the page load, before it can click 'Unread'.

To solve this, Sahi has a script parser which modifies this script to look something like this:

steps = new Array();
steps[steps.length] = "_click(_button('Inbox'));";
steps[steps.length] = "_click(_link('Unread'));";

And to execute these steps, it does:

var currentStep = 0;
function execute(){
if (currentStep == steps.length) return;
window.eval(steps[currentStep]);
currentStep++;
window.setTimeout('execute()', 1000);
}

execute();

Hope this helps people who are looking for a solution for javascript sleep.

java -cp does not work?

2007-12-11T20:48:00.000-08:00

Was trying to get Sahi's batch and shell scripts to work properly since I get a lot of queries regarding database drivers not being found instpite of adding to the classpath.

The problem is this. I need to add a mysql driver to Sahi's classpath so that the scripts can access the driver to do some data driven testing.

So I had to add the mysql driver to this:

java -jar ../lib/sahi.jar

Trivially, I added a -cp option so the command looks like this:

java -cp ../extlib/mysql-connector-java-5.0.4-bin.jar -jar ../lib/sahi.jar

But this does NOT work. Why? Because an executable jar should have its classpath in its manifest file and not outside.

The way to solve this is to use:

java -cp ../extlib/mysql-connector-java-5.0.4-bin.jar;../lib/sahi.jar net.sf.sahi.Proxy

So I made these changes on my windows machine, and it worked. Then logged on to my newly installed ubuntu to check if the shell script works with the same changes.

Made the necessary semi-colon to colon conversion in the classpath, so it looks like this.

java -cp ../extlib/mysql-connector-java-5.0.4-bin.jar:../lib/sahi.jar net.sf.sahi.Proxy

The sahi.sh file has a few other lines too.

When I ran sahi.sh, it gave me a NoClassDefFound error for net/sf/sahi/Proxy! After hunting around quite a bit on the net to see if there was such a problem on linux, I realized that nobody on the forums seems to understand the problem even when somebody reported it.

Eventually I realized that since I had edited the file in windows, it was adding ^M characters at the newlines which was not being recognized by the shell. So I used another editor and got rid of them by deleting and reentering the newlines. (I could also have used dos2unix, but did not have it installed on my machine)

I also realized why the people in the forums did not understand the problem. Most of the replies were from people using linux all the time and the question posters would have mostly been using windows but since the problem was on a linux machine would have posted the question on linux forums!

Anyway, now things work properly :)

Sahi at FOSS.in

2007-11-29T03:19:00.000-08:00

I will be presenting on the internal workings of Sahi at FOSS.in at Bangalore. This is not a demo level session but will involve a deep dive into the internals of the proxy, the script, extending the proxy, adding new APIs and other unmentionables. Hope to meet some interested and interesting people there.

Cycling trip to god knows where

2007-11-18T22:05:00.000-08:00

The cycle had been well over hauled. Had to replace the fork, both tyres and tubes. But the result was good. A very smooth, no hassles ride. The gear shifts were working well and the rolling was good. Except for the loose pedal shaft which had worn out and kept shaking and threatened to come off anytime. Saturday morning I set out from home at around 9, got the pedal shaft replaced and was on my way by 9.25. Was to meet Shrik near my office which was a good 17 kms away. Reached the office by 10.10. 45 minutes was a decent time and made me feel that I could go the office on cycle at least once a week. Had a vada pav and lime juice.

Shrik's Hero Octane looked good, and we started at 10.30 towards Hinjewadi. Past Hinjewadi, towards Paud, and we were in the villages and we kept going uphill and then eventually had a long long descent. Rested at the end of it. Had a limca and decided where to go next. Turned right on some local advice and went on a really bumpy bad road for quite a distance. The scenery around was good though it was all villages. Suddenly we were pleasantly surprised to see three woolly necked storks on the side on an inundated field. Took a few snaps and moved on in search of some hotel to eat.

We saw a lot of sugar cane laden tractors and got a sugarcane from one of them. Had some of it and washed ourselves in a nallah. Eventually found a hotel, but we could not get any meals there. Just had vada sambar, pav and a tea. The hotel guy suggested we go a little ahead and go to a Satya Sai temple nearby if we wanted to have a nap. So off we went to the temple, but it turned out to be one steep climb. Half way through we came across a short cement platform and we parked right there and went off to doze in the shade. Half an hour later, a little refreshed, we proceeded and came across Tikona on the right and a lovely lake on the left. It was beautiful. Ahead, after a friendly water break with some locals, we hit a steep long ghat road which was quite a challenge to go up. At the top, rested for a while with tea and a cream roll. Moved on from there down a lovely steep winding road, and then climbed again all the way to the Kamshet Dam and then downhill again to Pavana nagar village. At a fork where Kamshet was 10 kms to the left, we decided to go right because we were told that the highway was closer in that direction. Was that a mistake! We cycled for a long long distance with no sign of a highway. Twilight became night and we were either guided by the less than half moon, or blinded by the oncoming vehicles. Had a very needed halt where we hogged five bananas each and to wash off the sweet taste I had a tomato too! Tried to hitch a ride back with a few vehicles but no luck. So we kept going and eventually, we went over the Bombay Pune expressway, and down further to join the NH4. We went to the nearest dhaba, which was quite comfortable with beds to rest, had a beer and some not so great food, and chilled out for a while.

Around 8.30 we started cycling towards the nearest railway station, Begadewadi, which was hardly a km away from the dhaba thankfully. Got on the Lonavla-Pune local at 9.20 and after getting a few excited and curious glances and comments about our cycles, Shrik got down at Khadki and I at Pune. Pedalled back from the station to home and reached around 10.30, by when Shrik had also reached home. Had a bath and crashed.

Sahi no longer needs waiting for AJAX calls

2007-10-10T22:11:00.000-07:00

A new version of Sahi has been released. Download from https://sourceforge.net/project/showfiles.php?group_id=151639

More details on the release are here: http://sahitest.blogspot.com/

0=="" is true

2007-09-18T20:31:00.000-07:00

Debugging something on Sahi, came across this interesting equality in javascript:

0 == "" is true!

So are:

1 == "1" etc.

An easy way to test these out (and any other single line javascript) is to type

javascript:0==""

on the browser url navigation bar.
Yes, it works on IE and firefox.

Vote for Sahi on SourceForge Community Choice Awards 2007

2007-06-24T04:46:00.000-07:00

If you like Sahi, nominate and vote for it in the SourceForge Community Choice Awards 2007.
Ask your friends, colleagues and other bloggers to express support too. Thanks :)

http://sourceforge.net/projects/sahi

Trip to BR Hills

2007-05-26T03:22:00.000-07:00

Rashmi and I had been to this place called BR Hills near Mysore. It was early April and it was supposed to be a good time for sighting animals. But we were disappointed to know that a forest fire had burnt very large tracts of the forest and since there was no vegetation for the deer and other grazers, they had all moved away.
But luck was on our side. On the second night, we saw a bear in the moonlight, quite close to our log hut. It was scratching the ground and lazily moving around, making a lot of rustling noises among the fallen dry leaves.
The next day evening, as we were coming back from the safari, we saw a couple of deer, right by our side, totally ignoring us, and staring at something right ahead of us. They called twice, the sharp and loud "cow", the alarm call of the spotted deer. And then, from the grasses to the left, something came out on the road. In a second it vanished around the bend in the road. In a frenzy of excitement we urged the driver to drive down. He had not seen the animal come onto the road, but started driving down slowly to the bend. But the animal was gone.
We looked around, excited, frustrated and expectant. Someone saw a movement on the raised bank on the right. There it was, a full grown leopard, moving stealthily towards cover. A few more moves and the vegetation consumed the cat.
The now very excited driver guessed it must come out again a bit further and he started moving. And then we spotted it again. A magnificent animal, sitting right there, a few feet away, at a height on the raised bank which let us look at it straight in the eyes. I clicked away shaking with excitement. The leopard was just beautiful. I had never seen one so close and for so long. It just sat and looked at us. And slowly moved back, while still sitting, so that it was covered by some undergrowth. This is exactly how a tiger had behaved a few months back in Bandipur. Slowly creep backwards in imperceptible movements but soon be hidden from view behind some neighbouring undergrowth.

And then the leopard started moving. It came down the embankment and walked across the road and then down the curvy road which bent in a U to be parallel with us again. And man, was it brilliant. The leopard moved effortlessly, purposefully, silently, stealthily and man, so beautifully. And it was in full view of us all the time!
15 minutes had gone by and it was still around! Perfect sighting!
And then we started moving towards it, as we had to return to camp, and by then the leopard decided to vanish. And vanish he did. We could not figure out where he went.

We were again rewarded th next day with a bear sighting while we trekked nearby! Quite an eventful and satisfying three days!

Inter-linking of rivers in India is a bad idea

2007-05-26T00:40:00.000-07:00

I think inter-linking of rivers in India is a bad idea.
Whenever any friend of mine sounds impressed with this grandiose idea, I have felt frustrated and disturbed. We tend to overlook smaller more effective solutions and go for more romantic and glamorous 'solutions'. So I had been hunting around to gather information on this and make a case of it to people who want to understand what the pros and cons of this effort are. Following are a few links which spell out the danger of going ahead with this plan which does not have enough scientific evidence or prior experience to back it.

The Interlinking of Indian Rivers

Some Questions on the Scientific, Economic and Environmental Dimensions of the Proposal

Paper presented at Seminar on Interlinking Indian Rivers: Bane or Boon?
at IISWBM, Kolkata
17 June 2002

The conclusion of the above goes thus: "At the end of the above review and analysis made on the basis of whatever open information is available on the project for interlinking the rivers in India, there appears a great inconsistency in the declared claims of the project, and their feasibility"

This link, by Shailendra Nath Ghosh, talks about what is wrong with the concept, and what questions need to be answered, to make a proper assessment of this project

More articles and opinions. Quite insightful.

Bangladesh's views on India's inter-linking of rivers (They too oppose this)

Narmada Bachao Aandolan's views in "Dams, Rivers and People": This talks about rehabilitation failures and need for local management and conservation than an imposed, top down solution. They may appear biased because of their image in the press but they are people who have seen and fought unselfishly for rehabilitation of displaced people.

"The proposal is even more dangerous as attempting to link up veins of different persons without trying to find out the blood groups of the individuals. He said consequences will be disastrous." from National Citizens’ Meeting in Delhi concludes: River Link Proposals ill conceived, not in national Interest

More:
ILR in Supreme Court

Bahuguna opposes interlinking of rivers

I have not included any links which talk about the environmental aspects of this inter-linking. While the impact will be enormously destructive, talking about it seems to turn off people because they cannot see how economic growth and environmental conservation can co-exist. (May these lesser mortals exit soon)

So next time you hear of the inter-linking of rivers , doubt it, question it and take the side of what then comes out as true and right.

Sahi Nightly Build 2006-02-02 released

2007-02-02T05:10:00.001-08:00

Released another version of Sahi.
Added a few fixes to the release made 2 days back.

This is a fairly major release with the following changes:

* APIS added
_mockImage(pattern, clazz)
_assertContainsText(expected, el, msg)
_enableKeepAlive()
_disableKeepAlive()
_style(el, property)
_execute("commandline command");

* Feature additions
Test status on Controller
Launch test from command line
Add line numbers to script in logs
Back button support
Keep-alive support to tackle too many connections in TIME_WAIT issue
regular expression support in all APIs
Safari support
junit style logs added
jira issue tracking added

* Bugfixes
_condition when used with _include was stopping execution of script.

Of significance is the regular expression support in APIs.

So if you wanted to find a link which looks like "my_link_KJHSA" you could just do _link(/my_link_.*/).
Most APIs which took strings as parameters can now take regular expressions.

The communication between the browser and Sahi now uses Keep-Alive thus solving the major issue of too many connections in TIME_WAIT state.

Documentation is still being added for the new features on http://sahi.co.in

Sahi Nightly Build 2006-02-02 released

2007-02-02T05:10:00.000-08:00

Reward of Kabini

2006-12-12T19:12:00.000-08:00

Call after alarm call signalled the presence of a tiger very close to us. Chitals stood alert with their tails up and ears perked up giving out the loud shrill call. A loud "Cow" - an abrupt shrill call very close to us. A flurry of running dots as the deer bolt. The lantana bushes still cover the tiger from us. We wait patiently and impatiently, wait intently, expectantly. A few minutes pass. A lull. Long laden lull. The deer grow silent. No more alarm calls. The tiger is gone. Not even a glimpse.
The aura of the tiger is no longer felt. Deer go back to their grazing, langurs back to their mischief.

We move on. Disappointed. The night is closing in. We exit the jungle right on time. Our guide Sivanand suggests we try our luck along the road. Our headlights on, we go at a slow pace. A small herd of wild gaur. Huge animals, very close to us. Unperturbed, happily grazing away. One crosses the road in front of us as we wait. Slow, deliberate, unconcerned by our presence. A few metres ahead, a big tusker. Elephants close to your vehicle are not very comforting. A lone male tusker doubly so. We watch, torchlight on the tusks. The elephant leaves us alone.

Alarm calls again. Chital very close. A predator nearby. Experience braces us for disappointment, but optimism still lurks. Sivanand points in the other direction from where we are looking. A smallish stocky animal. Comes out of the cover of the forest. Starts walking on the road. Slightly hurried. Aware of our presence. We move slowly towards it. Our headlights on, but slow enough not to frighten it away. Slow enough to frustrate us to know there is a leopard in front and we cannot see it clearly. Frustrating.

It was a good decision. The leopard moved off the road to the right. We moved in slowly. The pale yellow beam lights the animal's rump as it ambles slowly into the bushes. Then it stops and sits down. In the light of the torch. A female, not too big leopard. The eyes glint reddish in the light as she looks at us for a brief moment. Into the light, into our eyes. Lovely. Brief. She gets up and walks on, away into the bushes, into the darkness, and into the jungle. Back to her comfort zone. And we move on to ours.

Button tag problems and IE 6

2006-12-02T05:12:00.000-08:00

Discovered that IE and Firefox handle the <button> tag differently.

If you have a tag like


< button type="submit" name="b" value="abc">
Button Text< /button>

When the form is submitted, the value of the parameter "b" is "abc" in firefox (the value attribute) while for IE, it is "Button Text" (the innerText)

Workaround?

We needed to submit a form using different buttons and each should send back its respective value.

We used


< input type="submit" name="b" value="Button Text 1">
< input type="hidden" name="Hash_of_Button_Text_1" 
value="value1">

< input type="submit" name="b" value="Button Text 2">
< input type="hidden" name="Hash_of_Button_Text_2" 
value="value2">

and on the server, accessed it by

params[hash(params["b"])]

where hash("Button Text 1") returns Hash_of_Button_Text_1

Ruby mocking - Using flexstub

2006-11-27T09:48:00.000-08:00

Ruby proves to be as sweet as most people have claimed it to be.

Pick testing and and you hit flexmock and flexstub.

So how do you use them? Here goes:

You have


class CA{
  def method_1
    do_some_processing
    if condition method_2
    else method_3
  end

  def method_2
    do_some_slow_processing_worth_mocking_out
  end

  def method_3
    do_some_more_slow_processing_worth_mocking_out
  end

}

If you want to test method_1's conditonal logic, with method_2 and method_3 stubbed out, here is how you do it with flexstub:


require 'flexmock'

class MyTest< Test::Unit::TestCase
  include FlexMock::TestCase
  def test_method_1
    ca = CA.new
#   do something to make the condition be true
#   stub some methods of the instance
    stub = flexstub(ca)
    stub.should_receive(:method_2).once
    stub.should_receive(:method_3).never
    ca.method_1
#   Note: method_1 is invoked on ca.
#   stub.method_1 will fail!
  end
end

So when would you use flexmock?
If you wish to mock all the methods being called on an object, you may use a flexmock.
A flexmock is an object which will just receive the methods you set expectations for, using should_receive.
If you wish to mock only one of the methods of an object but the other methods need to be invoked with the real functionality,
use flexstub to mock only that particular method.

As you see, method_1 is invoked properly, but method_2 and method_3 are stubbed out.

One remarkable thing about this is mocking out an object's new method.


flexstub(Customer).should_receive(:new)
                  .and_return(mock_customer)

Anywhere Customer.new is called inside any class during your test, it will return a mock_customer.
This is quite powerful, because we no longer need to use dependancy injection for just test purposes.

So much for security!

2006-11-27T01:14:00.000-08:00

If you are booking online on Spicejet, beware.
The lovely site which says verisign secured etc. actually sends all your credit card information as a query string!
So much for SSL!

I had written to them around 2 months back and it still remains the same!

Sahi - New Website

2006-10-20T01:39:00.000-07:00

Sahi has a new look and location at http://sahi.co.in.

A new release (Nightly Build 2006-10-20) of Sahi is also available. Download from here

Bird Fight

2006-10-09T06:19:00.000-07:00

Just posted an image of a fight between a brahminy kite and a purple swamphen.
The swamphen stamps and chases away the kite!
http://indianaturewatch.net/displayimage.php?id=4927

Jamming thoughts

2006-10-04T22:49:00.000-07:00

Travelling alone by car is a sin.
Why? It takes too much space on the roads.
The bigger the car the worse it is.
One car blocks roughly the space of four bikes.
So unless there are four people (at least) in a car, we are blocking resources which can be used more effectively.

Btw, bikes pollute more than new Euro IV cars (Source). So the problem is not straight forward.

There are a few things to consider:

What is the real problem? Road congestion or air pollution?
In Bangalore, it is road congestion. And it compounds air pollution. A car caught in congestion pollutes four times more than a running one (Source).

So if we have more free flowing traffic it is a big win. That means bikes are a decent option, and cars are not bad if fully utilized. Pooling in any form would help. Walking and cycling really are the best, but may not be feasible for everyone.

Pooling is kind of helpful in another way. One shares the stress of commuting and makes us human beings who snap less and who do not yell at the already poor pedestrian and the occasional cyclist.

Sad point is how the world sees it as a sign of success if you consume more resources and it may be perceived as below dignity for people to travel by cycle. Applause to those who have persevered cycling in Bangalore and because of their own profile are slowly making cycling be seen as cool and desirable.

Sad again that the right things have to be packaged as cool for most of us to accept it.

On a happier note, I hear less honking nowadays than I used to. People in Bangalore seem to be maturing to the traffic problem now. Cheers!

Treepie

2006-09-27T22:55:00.000-07:00

Treepie.org is up again. So if you are a birder and are looking for a site to consolidate your bird lists, join it and manage your lists.

I am now linking birds with superb pictures posted by the growing community of excellent wildlife photographers in India Nature Watch and pictures from google image search.

Turahalli

2006-09-25T04:31:00.000-07:00

Chirdeep and I had gone owl hunting to this place called Turahalli. Do not know if we actually reached there but we did end up on a nice patch with lots of rocks around. No owls, but we managed to see rock agamas. Loads of them. Every rock was ruled by one.
Soon bored, bouldering entertained us briefly.
Then we came across a neat white cheeked barbet surprisingly sitting on an electric line absolutely unperturbed by our photographic intentions and let us come quite close to it.
Also managed a decent shot of a pied bushchat.

Birding in Bannerghatta

2006-08-06T02:21:00.000-07:00

Chirdeep, Nikhil and I went birding on Saturday. Got a few pics.

Purple sunbird in non-breeding plumage

Plain flowerpecker

Lighting a mosquito coil ...

2006-08-06T00:43:00.000-07:00

Blog for Sahi

2006-08-05T23:17:00.000-07:00

Created a new blog for Sahi since there seem to be too many posts just on Sahi here.

That said,
Release 2006-08-06 with file upload facility has just come out

Toggling proxy on IE became easier with toggle_IE_proxy.exe and ProxyPal

And tons of thanks to Sriram Narayanan (Ram) for creating toggle_IE_proxy.exe
Seen few people who attack someone else's problem with such vigour and effect.
Taught me something...