Pocket Dope

Getting song IDs from spotify URLs

2009-03-08T16:45:00.000-07:00

When sending a http link or sporify: link to a friend they have the form:
http://open.spotify.com/track/3Z8cX6y0SeJIsI3yxoaQ8K

The protocol that spotify uses has track IDs as 128 bit integers, so how can we get from the text in the URL to this integer? Turns out it isn't base64 encoding, but rather base 62 (and with other alphabet).

Here are python functions for encoding/decoding:


a = '0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ'

def spotify_encode(v):
    res = []
    while v > 0:
        res = [v % 62] + res
        v = v / 62
    return ''.join([a[i] for i in res])

def spotify_decode(s):
    v = 0
    for c in s:
        v = v *62 + a.index(c)
    return v

Or, if you prefer java:


byte[] strValDecode(String s)
{
 final String alphabet = "0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ";
  
 BigInteger b = BigInteger.ZERO;
 BigInteger b62 = new BigInteger("62");

 for (int i=0; i<s.length(); i++) {
  int idx = alphabet.indexOf(s.codePointAt(i));
  b = b.multiply(b62).add(new BigInteger(Integer.toString(idx)));
 }
  
 return b.toByteArray();
}

Chrome 2 javascript performance

2009-01-09T16:08:00.000-08:00

Chrome 2 has been released to the dev channel, and I updated the comparison to Firefox 3. Turns out that chrome 2 is almost twice as fast as version 1 was upon release. The latest v1 release is as fast as v2 though.

More detail on the release can be found here.

Oh, and it scores 100/100 on the acid 3 test!


TEST                   COMPARISON            FROM                 TO             DETAILS

=============================================================================

** TOTAL **:           2.39x as fast     5982.2ms +/- 1.4%   2498.4ms +/- 2.8%     significant

=============================================================================

  3d:                  3.06x as fast      732.6ms +/- 0.8%    239.4ms +/- 1.1%     significant
    cube:              3.87x as fast      264.0ms +/- 0.7%     68.2ms +/- 10.3%     significant
    morph:             2.79x as fast      264.8ms +/- 0.9%     95.0ms +/- 8.7%     significant
    raytrace:          2.67x as fast      203.8ms +/- 2.2%     76.2ms +/- 8.1%     significant

  access:              5.62x as fast      965.0ms +/- 1.8%    171.6ms +/- 36.7%     significant
    binary-trees:      7.20x as fast       80.6ms +/- 2.8%     11.2ms +/- 14.5%     significant
    fannkuch:          9.05x as fast      443.4ms +/- 0.7%     49.0ms +/- 8.0%     significant
    nbody:             3.64x as fast      264.0ms +/- 5.2%     72.6ms +/- 78.0%     significant
    nsieve:            4.56x as fast      177.0ms +/- 3.2%     38.8ms +/- 4.8%     significant

  bitops:              7.69x as fast      831.8ms +/- 0.5%    108.2ms +/- 3.6%     significant
    3bit-bits-in-byte: 17.1x as fast      146.8ms +/- 1.4%      8.6ms +/- 26.3%     significant
    bits-in-byte:      16.1x as fast      215.4ms +/- 0.3%     13.4ms +/- 10.6%     significant
    bitwise-and:       4.98x as fast      193.4ms +/- 1.0%     38.8ms +/- 5.7%     significant
    nsieve-bits:       5.83x as fast      276.2ms +/- 0.7%     47.4ms +/- 7.1%     significant

  controlflow:         13.9x as fast       80.6ms +/- 1.8%      5.8ms +/- 27.9%     significant
    recursive:         13.9x as fast       80.6ms +/- 1.8%      5.8ms +/- 27.9%     significant

  crypto:              3.13x as fast      381.6ms +/- 2.5%    122.0ms +/- 9.6%     significant
    aes:               2.50x as fast      147.4ms +/- 0.5%     59.0ms +/- 21.6%     significant
    md5:               3.56x as fast      118.2ms +/- 8.3%     33.2ms +/- 5.6%     significant
    sha1:              3.89x as fast      116.0ms +/- 0.8%     29.8ms +/- 11.6%     significant

  date:                1.60x as fast      472.4ms +/- 1.6%    296.0ms +/- 4.2%     significant
    format-tofte:      2.07x as fast      297.6ms +/- 1.5%    144.0ms +/- 4.2%     significant
    format-xparb:      1.15x as fast      174.8ms +/- 2.3%    152.0ms +/- 5.8%     significant

  math:                3.22x as fast      747.6ms +/- 3.5%    232.2ms +/- 3.5%     significant
    cordic:            2.70x as fast      355.8ms +/- 1.0%    131.8ms +/- 6.0%     significant
    partial-sums:      3.19x as fast      241.2ms +/- 10.7%     75.6ms +/- 2.5%     significant
    spectral-norm:     6.07x as fast      150.6ms +/- 1.6%     24.8ms +/- 9.0%     significant

  regexp:              *1.44x as slow*    406.6ms +/- 6.5%    587.2ms +/- 3.1%     significant
    dna:               *1.44x as slow*    406.6ms +/- 6.5%    587.2ms +/- 3.1%     significant

  string:              1.85x as fast     1364.0ms +/- 3.5%    736.0ms +/- 3.4%     significant
    base64:            2.11x as fast      159.0ms +/- 2.1%     75.4ms +/- 33.0%     significant
    fasta:             4.18x as fast      290.2ms +/- 4.2%     69.4ms +/- 10.0%     significant
    tagcloud:          1.32x as fast      250.2ms +/- 4.3%    190.0ms +/- 7.3%     significant
    unpack-code:       1.70x as fast      471.0ms +/- 7.3%    276.6ms +/- 7.7%     significant
    validate-input:    1.55x as fast      193.6ms +/- 2.6%    124.6ms +/- 5.1%     significant

A first look at the performance of Google Chrome's V8 javascript engine

2008-09-09T05:16:00.000-07:00

I tried out google chrome today, and I must say that I was really impressed. Mostly by the user interface, it really stays out of the way of the content.

I had read the introduction to chrome where the JIT compiled javascript engine was mentioned so I decided to try it on the sunspider benchmark, and compare it to firefox 3 (which is faster than safari on my system).

Turns out that Chrome beats firefox 3.0, and when inspecting the results it has some fairly low hanging fruit to improve further; the part where it does worst is in date formatting and regular expression functions. These are pretty much C functions that are called from javascript, so should be rather easy to imporove.

Currently the bit manipulation stuff - where Chrome really shines - isn't very important for most webapps, but who knows, once available performance might be used. Also, the DOM speed is a really big part of performance, and is isn't reflected in this test.

For completeness, here are the results for my rather ancient system (athlon 2.0GHz):


TEST                   COMPARISON            FROM                 TO             DETAILS

=============================================================================

** TOTAL **:           1.35x as fast     5982.2ms +/- 1.4%   4441.2ms +/- 3.0%     significant

=============================================================================

  3d:                  2.75x as fast      732.6ms +/- 0.8%    266.4ms +/- 3.7%     significant
    cube:              3.52x as fast      264.0ms +/- 0.7%     75.0ms +/- 15.4%     significant
    morph:             2.41x as fast      264.8ms +/- 0.9%    110.0ms +/- 8.0%     significant
    raytrace:          2.50x as fast      203.8ms +/- 2.2%     81.4ms +/- 5.5%     significant

  access:              5.24x as fast      965.0ms +/- 1.8%    184.0ms +/- 5.5%     significant
    binary-trees:      6.83x as fast       80.6ms +/- 2.8%     11.8ms +/- 13.7%     significant
    fannkuch:          7.59x as fast      443.4ms +/- 0.7%     58.4ms +/- 6.1%     significant
    nbody:             3.87x as fast      264.0ms +/- 5.2%     68.2ms +/- 12.5%     significant
    nsieve:            3.88x as fast      177.0ms +/- 3.2%     45.6ms +/- 9.9%     significant

  bitops:              6.58x as fast      831.8ms +/- 0.5%    126.4ms +/- 3.7%     significant
    3bit-bits-in-byte: 16.3x as fast      146.8ms +/- 1.4%      9.0ms +/- 23.9%     significant
    bits-in-byte:      14.4x as fast      215.4ms +/- 0.3%     15.0ms +/- 5.9%     significant
    bitwise-and:       5.34x as fast      193.4ms +/- 1.0%     36.2ms +/- 6.6%     significant
    nsieve-bits:       4.17x as fast      276.2ms +/- 0.7%     66.2ms +/- 6.1%     significant

  controlflow:         19.2x as fast       80.6ms +/- 1.8%      4.2ms +/- 13.2%     significant
    recursive:         19.2x as fast       80.6ms +/- 1.8%      4.2ms +/- 13.2%     significant

  crypto:              2.86x as fast      381.6ms +/- 2.5%    133.6ms +/- 12.2%     significant
    aes:               2.32x as fast      147.4ms +/- 0.5%     63.6ms +/- 23.3%     significant
    md5:               3.13x as fast      118.2ms +/- 8.3%     37.8ms +/- 12.6%     significant
    sha1:              3.60x as fast      116.0ms +/- 0.8%     32.2ms +/- 8.4%     significant

  date:                *3.05x as slow*    472.4ms +/- 1.6%   1439.0ms +/- 7.3%     significant
    format-tofte:      *2.31x as slow*    297.6ms +/- 1.5%    686.6ms +/- 2.0%     significant
    format-xparb:      *4.30x as slow*    174.8ms +/- 2.3%    752.4ms +/- 13.2%     significant

  math:                2.67x as fast      747.6ms +/- 3.5%    280.4ms +/- 6.1%     significant
    cordic:            2.15x as fast      355.8ms +/- 1.0%    165.6ms +/- 5.0%     significant
    partial-sums:      2.82x as fast      241.2ms +/- 10.7%     85.4ms +/- 16.3%     significant
    spectral-norm:     5.12x as fast      150.6ms +/- 1.6%     29.4ms +/- 2.3%     significant

  regexp:              *1.82x as slow*    406.6ms +/- 6.5%    741.6ms +/- 3.4%     significant
    dna:               *1.82x as slow*    406.6ms +/- 6.5%    741.6ms +/- 3.4%     significant

  string:              1.08x as fast     1364.0ms +/- 3.5%   1265.6ms +/- 2.7%     significant
    base64:            1.13x as fast      159.0ms +/- 2.1%    141.2ms +/- 13.8%     significant
    fasta:             2.36x as fast      290.2ms +/- 4.2%    123.0ms +/- 3.8%     significant
    tagcloud:          *1.34x as slow*    250.2ms +/- 4.3%    334.8ms +/- 6.4%     significant
    unpack-code:       ??                 471.0ms +/- 7.3%    482.4ms +/- 1.2%     not conclusive: might be *1.02x as slow*
    validate-input:    1.05x as fast      193.6ms +/- 2.6%    184.2ms +/- 5.8%     significant

What compression algorithm to use with pack200

2008-04-25T04:09:00.000-07:00

I played around a bit with different compression algorithms after using pack200 to compress a jar file. I choose rt.jar from jre1.6.0_05/lib. Zip does very well when it is fed such well ordered data as the one pack200 produces, and it even beats bzip2. For fun, I included results for paq8n too.

Compression	Size (kb)	Compression Ratio
Uncompressed	47108	100%
pack200	10146	21.5%
pack200+zip	4283	9.1%
pack200+bzip2	4390	9.3%
pack200+7zip (lzma)	3814	8.1%
pack200+7zip (ppmd)	4089	8.7%
pack200+paq8n	3239	6.9%

This means that zip is within 32% of the incredible compression ratio of paq.

For pack200 I used EFFORT=7, for zip -9, bzip2 -9, paq8n -8, and for the two 7zip I used a limit greater than the file size.

Why you shouldn't use LGPL for GWT widgets

2008-02-10T20:23:00.000-08:00

I have been reading the GNU Lesser Public License (LGPL) the last days to try to understand what it means for GWT programs.

The way I see it, there are two main purposes with the LGPL:

To make sure that improvements to the library can propagate back into the main library, and be useful to the maximum number of people.
To let end users upgrade a closed source application with a newer version of the LGPLed library.

The second point is the interesting one, if the library is dynamically linked (java and c programs using dlls fall under this case) then everything is fine, you just can't have a license that prevents reverse engineering and so on, but if an application links statically with an LGPLed library you have to make sure that the end user can re-link the application by distributing object files (source code is fine too of course). And here is the big problem with GWT widgets using LGPL. The GWT compiler takes java classes on the developer's computer, links them together and produces javascript that is then distributed to the user. This is static linking.

So how can one assure that the user can relink the application with a newer version of the LGPLed widget without releasing the complete source code? Well, one could always create a big jar file of the rest of the application and obfuscate it...

This seems very restrictive to me and I wonder if most of the authors of LGPLed GWT widgets know this, because it seems as if they might as well have used GPL then.

Oh, and by the way, I am not a lawyer

What does GWT's "Object is not allowed" error message mean

2008-01-17T07:51:00.000-08:00

I used a custom jar I had downloaded in one of my GWT projects, and started to get the following error message in my ANT build:

[WARN] In order to produce smaller client-side code, 'Object' is not allowed; consider using a more specific type

I looked into this, and it seems that serialization in GWT works the following way:
The GWT compiler starts by finding all types that inherits Serializable and IsSerializable, and creates serialization code specifically for those classes. Now, if one of those classes contains a non-transient member of type Object, GWT would have to generate code for serializing pretty much every class that is being used in the whole project, which isn't very efficient. So it outputs that warning instead.

When I tried that construct in my own code it wouldn't even compile, so I'm not quite sure how they managed to build the jar in the first place.

How to install Java 1.6 on Debian Etch Stable II

2008-01-11T00:52:00.000-08:00

My last post on the subject is here, but one problem with that approach is that it won't update automatically with apt-get update/upgrade. However, it seems that java 1.6 is also available in backports, so just add backports.org to sources.list:

emacs /etc/apt/sources.list

And add the line

deb http://www.backports.org/debian etch-backports main contrib non-free

Then install it with

aptitude update
aptitude -t etch-backports install sun-java6-jdk

And select the new java version:

update-alternatives --config java

How to live 14 years longer

2008-01-10T05:20:00.000-08:00

I was watching the news today, and they reported on research that showed that people who:

don't smoke
eat fruit or vegetables five times a day
exercise
don't drink more than two drinks daily

live 14 years longer than their smoking, drinking, immobile, fast food eating friends.

I'd guess that smoking accounts for the largest part of those 14 years.

And.. if you already do all those healthy things, think of this post as "How to live 14 years shorter"

Interview with The Knife

2008-01-10T04:09:00.000-08:00

How have you changed since the last album?

Well.. we wanted to do a bit more right-on music this time, we tried to make something clearer and more direct, and the last record might have been a bit introvert at times and a bit secretive, and we felt that this time we'll make it the other way around.

On some songs you sing like a woman and on some you sing like a man, why do you do that?

Some messages that we want to tell get an extra twist when I do it as a guy, because there are things that have been said over and over by girls, for example feminist thoughts that girls have been talking about for a long time but guys haven't picked up on yet, so if I do it as a guy instead something happens. Also, many times when we alter my voice one can't really tell either, it's a bit in between, somewhat androgyne, and one can't tell where you have it ant that is pretty exciting, or what do you think?

Yeah.. it's a lot of fun to pitch Karin's voice down, so that makes the work more fun.

Yes

How important is it to have a message in your music?

For us it has become more and more important during our work and we try to explore it, to say more with our music that is directly connected to issues in society, it is something that we're trying a bit more this time, and feel that it is pretty interesting.

If you're interested in politics and what it looks like around you ordinarily, why shouldn't you use that when making music, for us it's connected. Music isn't something we do just to... music is what we do, it's such a big part of our lives so it has to be about the things that we're thinking about.

How come you have never played live?

When we.. The first time when the last record was finished, we had some thoughts about starting rehearsing, but we were so fed up with all the songs on the record because we'd been working on it for so long that it didn't come naturally to start doing all this music live, because it didn't come into existence in that way, we have been creating the music in a computer and cut and built up something in the computer that doesn't have a connection to two people standing in a rehearsal room playing the music.

But I guess that is the big.. we are a studio band from the beginning when we record and create the songs at the same time in a small room somewhere, so we have never done anything to show visually or anything that would be suitable for a stage or so, but it is possible that we'll figure our some form.

What are you doing right now, what are your present projects?

Right now, since autumn we're making music for Kristina Olofsson's new movie, "Hannah med H" (Hannah with an H), that probably will be in cinemas next autumn or possibly a bit later. So we're making the music for the entire film, or rather, we're making all the music that is in the film. And we're almost done now.

How does that work, what kind of film is it and when you're making music for a film, how do you think, do you go about it just as usual or do you try to put yourselves in the character's situation and stuff like that?

It's a youth film, drama-thriller that is about a girl who is a bit alternative. And when we're making the music it is very different compared to how we worked earlier, this time it is about illustrating or trying to enhance the feelings that the director wants the film to mediate, so those feelings don't come from us directly, instead we try to understand what the director wants to be felt in a particular scene and then we try to make music in that direction.

It is quite a bit of mood-music, there are some real songs, perhaps when a character in the film is listening to music or something like that but most of the time it is about enhancing feelings and moods in the film so it is pretty different compared to what we have done earlier.

We like a lot of film, watch films a lot, but well.. there are a lot of exciting directors, David Lynch and Richard Kelly who made this Donnie Darko, then we're very inspired by Peter Wahlbäck (Swedish comedian), his earlier performances "Teater für Alle", "Ballett für Alle", those were very exciting, his ability to combine humor with a very clear and strong political message is very exciting, and it's very powerful and well packaged in a good way.

Now comes one of those questions that we ask everyone: What music did you listen to when you were young?

Tältprojektet (lit. The tent project), that was when we were really young.
A lot of reggae at home, Bob Marley... yeah it was a lot of Bob Marley.
And Jazz

Hur har ni förändrats sedan den förra skivan?

Ja.. vi ville väl göra mer rakt på sak musik den här gången, vi försökte göra någonting tydligare och mer direkt och förra skivan kanske var lite introvert ibland och lite hemlighetsfull och vi kände att nu ska vi göra nåt tvärt om.

På endel låtar sjunger du som kvinna och på endel låtar sjunger du som man, varför gör ni det?

Vissa budskap som vi vill föra fram får en extra tvist när jag gör det som kille, för att vissa saker som har sagts gång på gång av tjejer, t ex feministiska funderingar som tjejer har pratat om ganska länge men som killar inte har annamat ännu, så om jag gör det som kille istället så händer det nånting, plus att många gånger när vi gör det när vi gör om min röst så vet man inte heller, det blir lite mitt emellan också, lite androgynt, och man vet inte riktigt var man har den och det är ganska spännande, eller vad säger du?

Mmm.. det är väldigt roligt att pitcha ner Karins röst så det gör att arbetet blir roligare.

Ja

Hur viktigt är det att ha ett budskap med sin musik?

För oss när vi har jobbat under tiden så har det blivit mer och mer viktigt och vi provar att utforska det, att säga mer saker i vår musik som är direkt kopplade till sammhälleliga frågor, det är nåt vi provar denna gången lite mer, och tycker att det är ganska intressant.

Om man är intresserad av politik och hur det ser ut runt omkring en i vanliga fall, varför skulle man inte använda sig av det när man gör musik, för oss så sitter det ihop. Musik är inte nåt vi gör bara för att.. musik är ju det vi gör, det är så stor del av vad vi håller på med så det måste handla om sånt vi funderar över.

Hur kommer det sig att ni aldrig har spelat live?

När vi.. Första gången när förra skivan var klar så hade vi funderingar på att börja repa, men då var vi så hemskt trötta på alla låtar på skivan iom att vi jobbat med det så länge så då föll det sig inte så naturligt att börja sätta igång all den här musiken live iom att den aldrig kommit till på det sättet, vi har ju gjort musiken i dator och klistrat och byggt upp någonting i datorn som inte alls har nån koppling till ett par människor som står i en replokal och spelar musiken.

Men det är väl det som är det stora.. vi är från början ett studioband vi spelar in och gör låtarna samtidigt i ett litet rum någonstans, så vi har aldig gjort nånting för att visa upp nåt visuellt eller nånting som skulle kunna passa på en scen eller så, men det är möjligt att vi hittar på nån form.

Vad håller ni på med just nu vad är era projekt för närvarande?

Just nu håller vi på sedan i höstas och gör musik till Kristina Olofssons nya film "Hannah med H", som förmodligen kommer komma upp på biograferna i höst ev lite senare. Så då gör vi musik till hela filmen, eller all musik som är i filmen håller vi på att göra. Och börjar bli klara nu.

Hur fungerar det, vad är det för sorts film och när man gör musik till en film, hur tänker man då, är det bara som vanligt, eller försöker ni tänka er in i rollfigurernas situation eller nåt sånt där?

Det är en ungdomsfilm, drama-thriller som handlar om en tjej som är lite alternativ. Och när vi gör musik så är det väldigt annorlunda mot hur vi tidigare arbeta, nu handlar det om att illustrera eller försöka förstärka de känslor som regissören vill att filmen ska förmedla, så att de känslorna kommer inte från oss direkt utan vi ska försöka förstå vad regissören vill ska kännas i en viss scen och så ska vi försöka göra musik åt det hållet.

Det är ganska mycket stämningsmusik det är iofs en del sådär riktiga låtar, när kanske nån person i filmen lyssnar på musik eller nånting sånt men det mesta är ju förstärka känslor och stämningar i filmen så det är ganska annorlunda mot vad vi gjort tidigare.

Vi gillar mycket film, tittar mycket på film, men ja.. det finns ju massa spännande regissörer, David Lynch och Richard Kelly som gör den här Donnie Darko, sen inspireras vi väldigt mycket av Peter Wahlbäck, hans tidigare föreställningar "Teater für Alle", "Ballett für Alle", det var väldigt spännande, hans förmåga att kombinera humor med ett väldigt tydligt och starkt politiskt budskap är väldigt spännande, och det är väldigt slagkraftigt och välpacketerat på ett bra sätt.

Nu kommer en sån där fråga som vi ställer till alla, vad lyssnade ni på för musik när ni var små?

Tältprojektet, alltså när vi var riktigt små.
Mycket reggae hemma, Bob Marley... ja det var mycket Bob Marley
Och jazz

How to install Java 1.6 on Debian Etch Stable

2008-01-10T02:02:00.000-08:00

Update: Here is a smarter way

Presently, only java 1.5 (aka java 5) is in the stable repository. Therefore, a little bit of work is needed to install java 1.6.

Start by downloading the files you need from the unstable repository, here.

Download all the files you need, I got the following ones:

sun-java6-bin_6-03-2_i386.deb
sun-java6-doc_6-03-2_all.deb
sun-java6-jdk_6-03-2_i386.deb
sun-java6-jre_6-03-2_all.deb

If you want the doc package, you need to download the documentation zip (jdk-6-doc.zip) from java.sun.com manually, and do the following:

chown root jdk-6-doc.zip
chgrp root jdk-6-doc.zip
mv jdk-6-doc.zip /tmp/

Then I tried a simple

dpkg -i sun-java6-*

to install the packages but they have dependencies on libraries not yet in stable, so I tried to install them manually just to see what apt would tell me:

apt-get install unixodbc libstdc++5

This didn't work of course, but apt suggested that I'd try

apt-get -f install

which amazingly worked. After that java installed like a charm. Don't forget to select the right java version after you're done:

update-alternatives --config java