Witty Javangelist

Use the brain, not Google

2009-11-11T17:27:00.000+02:00

>> I searched whole web and nothing seem to give me
>> at least a pointer so any help will be greatly
>> appreciated.

> For some tasks it is much more efficient to use the brain
> than Google.

Say, have you got a URL? thebrain.com's search engine didn't match "Newton" at all. What am I missing here?

Mercurial backup script

2009-11-11T16:07:00.003+02:00

A simple cmd batch to clone and pack a mercurial repo. I used to run that just before logging off.

rmdir /S /Q repo-snap

hg clone -U repo-work repo-snap



7z a -t7z -mx=9 repo-snap-@stamp.7z repo-snap



rmdir /S /Q repo-snap

java -jar jstamper.jar repo-snap-@stamp.7z



shutdown -s -t 120



pause

PS: jstamper.jar was a simple class which renamed file so @stamp was substed with current datestamp. Bash does that in a blink, but there's no simple way to do that in cmd.

Metaprotocol Taxonomy

2009-10-03T14:46:00.002+03:00

Because each application has different communication requirements, there is no simple "best" metaprotocol. Still, some general conclusions are possible. Object-oriented communication, i.e. sending an object model from one machine to another, needs an object-oriented specification (and API where appropriate). The typed metaprotocols (Hessian, RMI, CORBA, JSON) best fit this requirement since they are designed around language types. The syntactic metaprotocols (SOAP, POX) are better suited for document-based applications, i.e. applications which retrieve data through syntactic queries like XPath or XQuery.

The common communication patterns: REST, RPC, Messaging and Streaming also influence the choice of metaprotocol. Since RPC is a typed and API-based pattern, the typed metaprotocols fit better. Messaging applications can use the syntactic metaprotocols if they are document or query based, and can use the typed metaprotocols if they are object model based. Like Messaging, REST can use either typed or syntactic metaprotocols, depending on the applications. Streaming applications like AJAX/Comet fit dynamic typed protocols best.

CDMA Ukraine Setup

2009-09-13T16:44:00.005+03:00

Немного в некотором роде смешных ремарок в самом скриншоте. Вывод у меня такой - если вы в черте города, то район и близость соты играют примерно такое же значение, как и высота. Еще замечу, что EDGE у меня в этом же месте выдавал под 16 КБайт в секунду, в то время как CDMA показывал около трех. Задержка канала вроде бы уменьшилась раза в два - хотя сказать что скорость стабильная все равно пока не могу.

7zip cmd goodies

2009-09-05T22:16:00.006+03:00

Some simple magic spells to ease file management under windows. Here goes...

compressing all the dirs in the CWD to respectively named 7z archives somewhere else:
set DEST=C:\some\where\else\note\the\trailing\slash\
for /D %%i in (./*) do @7z a -mx=9 %DEST%%%i.7z %%i

UPD: more goodies are posted separately with the label 7z

Open Source Bigtable Analogs

2009-08-19T12:41:00.007+03:00

Actually, there's no fully comparable analog to Google's Bigtable - all opensource projects are weaker from performance and stability standpoints. But apparently they are already feasible for managing some unstructured datasets, provided no mission-critical data is involved.

Bellow follows the best 3-minute introduction to the whole concept I've seen online, and then you see some brief product descriptions.

The hardest part about learning HBase (the open source implementation of Google's BigTable), is just wrapping your mind around the concept of what it actually is.

I find it rather unfortunate that these two great systems contain the words table and base in their names, which tend to cause confusion among RDBMS indoctrinated individuals (like myself).

This article aims to describe these distributed data storage systems from a conceptual standpoint. After reading it, you should be better able to make an educated decision regarding when you might want to use HBase vs when you'd be better off with a "traditional" database.

Bigtable

Bigtable is a distributed storage system for managing structured data that is designed to scale to a very large size: petabytes of data across thousands of commodity servers. Many projects at Google store data in Bigtable, including web indexing, Google Earth, and Google Finance. These applications place very different demands on Bigtable, both in terms of data size (from URLs to web pages to satellite imagery) and latency requirements (from backend bulk processing to real-time data serving).

Despite these varied demands, Bigtable has successfully provided a flexible, high-performance solution for all of these Google products. In this paper we describe the simple data model provided by Bigtable, which gives clients dynamic control over data layout and format, and we describe the design and implementation of Bigtable.

HBase

HBase uses a data model very similar to that of Bigtable. Applications store data rows in labeled tables. A data row has a sortable row key and an arbitrary number of columns. The table is stored sparsely, so that rows in the same table can have widely varying numbers of columns.

A column name has the form "<family>:<label>" where <family> and <label> can be arbitrary byte arrays. A table enforces its set of <family>s (called "column families"). Adjusting the set of families is done by performing administrative operations on the table. However, new <label>s can be used in any write operation without pre-announcing it. HBase stores column families physically close on disk, so the items in a given column family should have roughly the same read/write characteristics and contain similar data.

Only a single row at a time may be locked by default. Row writes are always atomic, but it is also possible to lock a single row and perform both read and write operations on that row atomically.

Hypertable

The Hypertable data model consists of a multi-dimensional table of information that can be queried using a single primary key. The first dimension of the table is the row key. The row key is the primary key and defines the order in which the table data is physically stored. The second dimension is the column family. This dimension is somewhat analogous to a traditional database column. The third dimension is the column qualifier. Within each column family, there can be a theoretically infinite number of qualified instances.

For example if we were building a URL tagging service, we might define column families content, url, and tag. Within the "tag" column family there could be an infinite number of qualified instances, such as tag:science, tag:theater, tag:good, etc. The fourth and final dimension is the time dimension. This dimension consists of a timestamp that is usually auto assigned by the system and represents the insertion time of the cell in nanoseconds since the epoch. Conceptually, a table in Hypertable can be thought of as a three dimensional Excel spreadsheet with timestamped versions of each cell.

Cassandra

Cassandra is a highly scalable, eventually consistent, distributed, structured key-value store. Cassandra brings together the distributed systems technologies from Dynamo and the data model from Google's BigTable. Like Dynamo, Cassandra is eventually consistent. Like BigTable, Cassandra provides a ColumnFamily-based data model richer than typical key/value systems.

Cassandra was open sourced by Facebook in 2008, where it was designed by one of the authors of Amazon's Dynamo. In a lot of ways you can think of Cassandra as Dynamo 2.0. Cassandra is in production use at Facebook but is still under heavy development.

DLA model parameter space

2009-08-13T14:13:00.006+03:00

Here is an image showing how DLA-based fractal model changes when couple of density parameters are slightly tilted. The whole thing is here to generate a fractal probability distribution for geographical Yook/Jeong/Barabasi network model.

Maybe IFS-based fractal would be better here - as it might allow for more control of fractal dimension than DLA does. But then I would not get an obvious way to control exponential density fall-off, and IFS seems to be less natural than this DLA.

Also, here're some resources on fractal generation I'd stumbled upon recently:

Larry Riddle's Classic Iterated Function Systems - quite thorough in-depth survey of several well-known fractals;
E. Garcia's Fractal Clusters/Networks page - featuring side-by-side comparison of different fractal network formations;
Fractals Page at Math Archives Project - lots of related resources;
Andy Lomas's Pictures of Aggregation - quite remarkable 3-dimensional fractal art;

The Maven Way

2009-07-31T21:13:00.012+03:00

Doh... maven is a big bunch of hacks. How do I do this, how do I do that. On the first bit of creativity you're slapped with pile of bizarre arguments and/or unpredictable/unexplainable behaviour. Do it the maven way or don't do it at all.

Here I would like to dump occasional memos on magic maven spells.

Dump project dependencies / list dependency tree:
mvn clean install dependency:tree
BTW, why maven needs to build the project to list the dependencies (I guess downloading poms should suffice)? UPD: When you are left with a sandbox which broke due to some library versioning clash it's quite cool: to know which lib is pulling the wrong version you need to fix the problem blindly first. Yeah, that's The Maven Way to do things, you know...

Virtual Hosts and HTTPs

2009-06-28T10:41:00.002+03:00

Using name-based virtual hosts on a secured connection can be problematic. This is a design limitation of the SSL protocol itself. The SSL handshake, where the client browser accepts the server certificate, must occur before the HTTP request is accessed. As a result, the request information containing the virtual host name cannot be determined prior to authentication, and it is therefore not possible to assign multiple certificates to a single IP address. If all virtual hosts on a single IP address need to authenticate against the same certificate, the addition of multiple virtual hosts should not interfere with normal SSL operations on the server. Be aware, however, that most client browsers will compare the server's domain name against the domain name listed in the certificate, if any (applicable primarily to official, CA-signed certificates). If the domain names do not match, these browsers will display a warning to the client user. In general, only address-based virtual hosts are commonly used with SSL in a production environment.

Programmer Flavors

2009-06-12T13:11:00.004+03:00

My father built custom homes, and in my youth I would occasionally work for him, mostly doing grunt labor and sometimes hanging sheet rock. He and his lead carpenter would tell me that they gave me these jobs for my own good -- so that I wouldn't go into the business. It worked.
So I can also use the analogy that building software is like building a house. We don't refer to everyone who works on a house as if they were exactly the same. There are concrete masons, roofers, plumbers, electricians, sheet rockers, plasterers, tile setters, laborers, rough carpenters, finish carpenters, and of course, general contractors. Each of these requires a different set of skills, which requires a different amount of time and effort to acquire. House-building is also subject to boom and bust cycles, like programming. If you want to get in quick, you might take a job as a laborer or a sheet rocker, where you can start getting paid without much of a learning curve. As long as demand is strong, you have steady work, and your pay might even go up if there aren't enough people to do the work. But as soon as there's a downturn, carpenters and even the general contractor can hang the sheet rock themselves.

J2SE applets stability

2009-03-07T13:19:00.007+02:00

Not so long time ago I was doing some research into spurious firefox hangs occuring on applet loading. Looks like sometimes SUN tries to sweep a problem under the carpet. Or even worse - they attempt to patch their way out.

None of these approaches work - as you get either "slightly dysfunctional" components or nasty quantum regressions.

As for the carpet - some of the tickets below are "not reproducible", despite the fact that people are reproducing that massively across multiple bugzillas. See comments for the mojebanka.cz ticket and the last ubuntu ticket too. Also - there're couple of tough tickets with massive votes which once disappeared from public bugzilla - I had a bookmark on that somewhere.

So, for those who firmly believe applets are stable and well-established piece of technology, here goes some plain data (somewhat biased - as this is a bugzilla anyway). Second line is version with delivered fix:

New plugin applet freezes the browser occasionally 6u10(b21)
FF3 browser hangs while loading atomica and rocketmania applets from popcap.com 11-Closed, Not Reproducible, bug
Firefox hang upon first applet launch with Windows OEM builds of Java 6u7(b01)
Firefox with JDK 6 will cause the browser to hang 6u10(b02)
Plugin2: Applet from www.mojebanka.cz causes FF3 to hang 11-Closed, Not Reproducible, bug
Malformed 404s cause breakage of Java/JavaScript bridge and browser hangs 6u11(b03)
Java Plugin in Firefox hangs since 6u4 when remote policy file is present in java.security 6u10(b26), 6u7-rev(b07)
If <object> tag is malformed, FF browser either crashes or hangs. 6u10(b07)
The old plugin hangs firefox 3 when loading java applets That's quite recent Ubuntu bug, not fixed still, affects 8.10 users, 8.04 works fine

Well, actually, I don't think that applets are completely not feasible, but the browser/applet bridging seems to be somewhat shaky. My opinion is that applets are the most pluggable, secure and functional approach, at least for some of possible usage scenarios.

SUN : Cloudy times

2009-03-06T18:49:00.002+02:00

Use the cloud, build the cloud, be the cloud.

via Cloud Computing Demystified

Open/Closed Principle : Strategic Closure

2009-03-05T19:46:00.003+02:00

It should be clear that no significant program can be 100% closed. For example, consider what would happen to the DrawAllShapes function from Listing 2 if we decided that all Circles should be drawn before any Squares. The DrawAllShapes function is not closed against a change like this. In general, no matter how “closed” a module is, there will always be some kind of change against which it is not closed. Since closure cannot be complete, it must be strategic. That is, the designer must choose the kinds of changes against which to close his design. This takes a certain amount of prescience derived from experience. The experienced designer knows the users and the industry well enough to judge the probability of different kinds of changes. He then makes sure that the open-closed principle is invoked for the most probable changes.

ScheduledThreadPoolExecutor : gotcha

2009-02-28T10:21:00.004+02:00

This code

final int period = 1000 /*millis*/;
final int delay = 10 /*millis*/;
final int num = 20;

ScheduledExecutorService ses = 
    Executors.newScheduledThreadPool(4);

for (int i = 0; i < num; i++) {
  ses.scheduleWithFixedDelay(

    new Runnable() {
      public void run() {
        try {
          Thread.sleep(num * 1000 - 10);
        } catch (InterruptedException ie) {
          throw new RuntimeException(ie);
        }
        System.out.println("tick: " + System.currentTimeMillis());
      }
    },

    i * period, delay, TimeUnit.MILLISECONDS
  );
}

Thread.sleep(Integer.MAX_VALUE);

should produce 20 tasks which in total would tick once a second. But nope. You would get only 4 tasks, as corePoolSize does not expand if pool is unable to handle all scheduled tasks. The actual output would be something like:

tick: 1235810468265
tick: 1235810469237
tick: 1235810470278
tick: 1235810471280
tick: 1235810488274
tick: 1235810489236
tick: 1235810490277
tick: 1235810491299

The only hint for this is left in the javadocs for the ScheduledThreadPoolExecutor:

While this class inherits from ThreadPoolExecutor, a few of the inherited tuning methods are not useful for it. In particular, because it acts as a fixed-sized pool using corePoolSize threads and an unbounded queue, adjustments to maximumPoolSize have no useful effect.

Design Pattern : Good Citizen

2009-01-22T19:53:00.002+02:00

Imagine a software system where there is no need for you to spend your time programming defensively; your objects will be used responsibly, and your methods will always be passed sensible arguments. This low-friction utopia can be approached by establishing some simple programming rules so that every class acts as a 'good citizen' in the society of classes collaborating at runtime. This page outlines some rules that we, and others, believe lead to good citizenship. All are aimed at improving clarity, reducing surprise, and promoting basic consistency. As a good citizen, I...
Keep a consistent state at all times - init() or populate() is a code smell.
Have no static fields or methods
Never expect or return null.
Fail fast - even when constructing.
Am Easy to test- all dependent object I use can be passed to me, often in my constructor (typically as Mock Objects).
Accept dependent object that can easily be substituted with Mock Objects (I don't use Concrete Class Dependency).
Chain multiple constructors to a common place (using this(...)).
Always define hashCode() alongside equals()
Prefer immutable value objects that I can easily throw away.
Have a special value for 'nothing' - e.g. Collections.EMPTY_SET.
Raise checked exceptions when the caller asked for something unreasonable - e.g. open a non-existant file.
Raise unchecked exceptions when I can't do something reasonable that the caller asked of me - e.g. disk error when reading from an opened file.
Only catch exceptions that can be handled fully.
Only log information that someone needs to see.

Changing Windows XP HAL

2009-01-19T15:36:00.003+02:00

If you perform a Windows installation with default settings in VirtualBox, Halacpi.dll will be chosen as VirtualBox enables ACPI by default but disables the IO APIC by default. A standard installation on a modern physical PC or VMware will usually result in Halaacpi.dll being chosen as most systems nowadays have an IO APIC and VMware chose to virtualize it by default (VirtualBox disables the IO APIC because it is more expensive to virtualize than a standard PIC). So as a first step, you either have to enable IO APIC support in VirtualBox or replace the HAL. Replacing the HAL can be done by booting the VM from the Windows CD and performing a repair installation.

Java Performance Benchmarks

2009-01-09T20:33:00.004+02:00

Several benchmarks comparing C++ and Java exist on the web. The results are mixed: some show that Java is actually faster than C++, while most show that C++ is still faster than Java but by a small margin. The purpose of this post is to talk about the theory behind the Java optimization techniques, so I didn't set out to create my own benchmarks myself, but nothing like hard data to prove a point. So here are the links to some benchmarks found on the web:
The Java is Faster than C++ and C++ Sucks Unbiased Benchmark: despite the name, this benchmark shows very similar results between Java and C++, with the occasional scenario where C++ beats Java hands down.
The Java Faster than C++' Benchmark Revisited: someone who didn't like the benchmark above and found different results, where C++ has a clearer lead. Even so, Java is still close, wins some benchmarks, and is clearly slow only in a handful of tests.
The Computer Language Benchmarks Game: compares a number of programming languages using different algorithms. Gnu C++ and Java 6 are compared, and C++ wins most of the comparisons, but in most of the cases by a very close margin, and Java is the occasional winner in some of the tests.

...and one more update: Stefan Krause; Update for Java Benchmark.

Maven breaks binary resources

2008-12-08T13:13:00.000+02:00

The fun thing is that after checking in the change the Bamboo build was also correct. My colleagues, who were still trying to resolve the font error, were quite amazed, as was I. For some reason Maven had messed up the files while filtering and copying. Quite bizarre if you think of the fact that the XSD file which caused my error didn’t have a keyword in it that should be substituted and the font file that caused the Bamboo error was binary…

I'd be plainer than that developer: Maven breaks your binary resources once you turn resource filtering on. I've spotted this twice: with an attached dll, and with a license file.

Self-stabilizing protocol design

2008-12-06T17:31:00.003+02:00

The number of processors and the sometimes noisy communication media in a distributed system impose the need for a fault tolerant design. One strong notion of fault tolerance is self-stabilization. Roughly speaking, a self-stabilizing protocol can cope with any kind of faults in the history. A distributed system is self-stabilizing if it can be started in any possible global state. Once started, the system runs for a while until it reaches a legitimate global state in which the system is consistent. The self-stabilization property makes the system tolerant to faults in which processors exhibit a faulty behavior for a while and then recover spontaneously in an arbitrary state. When the intermediate period between one recovery and the next faulty period is long enough, the system stabilizes.

Maven rake field

2008-12-02T17:47:00.003+02:00

When you first experience Maven, it's like the cinematic cliché of two lovers running across a grassy field to embrace: It takes care of getting the right JARs for you, keeps test and build dependencies separate, and generates configuration artifacts for your favorite IDE, too. Even though it's XML, it's still a sweet five-minute user experience, but then you start stepping on the rakes in the grass.
<snip>
I do like where Matthieu is headed with raven or Russel Winder is headed with gant, but I'd really like the two together: non-XML syntax, dependency management, fully-stocked and up-to-date repositories, well-supported by continuous integration systems, and works with everything that Ant works with. Maybe gant plus the Maven2 Ant tasks or Ivy is the most silver-ish bullet for the time being, but seeing as I don't need to shoot any werewolves, Maven fits the bill for the moment

Web Frameworks

2008-12-01T13:15:00.005+02:00

jQuery is a fast and concise JavaScript Library that simplifies HTML document traversing, event handling, animating, and Ajax interactions for rapid web development.

jQuery home

Lift is an expressive and elegant framework for writing web applications. Lift stresses the importance of security, maintainability, scalability and performance, while allowing for high levels of developer productivity. Lift open source software licensed under an Apache 2.0 license. Lift borrows from the best of existing frameworks, providing
Seaside's highly granular sessions and security
Rails fast flash-to-bang
Django's "more than just CRUD is included"
Wicket's designer-friendly templating style
And because Lift applications are written in Scala, an elegant new JVM language, you can still use your favorite Java libraries and deploy to your favorite Servlet Container. Use the code you've already written and deploy to the container you've already configured!

lift homepage

Spring Web MVC allows you to use any object as a command or form object - there is no need to implement a framework-specific interface or base class. Spring's data binding is highly flexible: for example, it treats type mismatches as validation errors that can be evaluated by the application, not as system errors. All this means that you don't need to duplicate your business objects' properties as simple, untyped strings in your form objects just to be able to handle invalid submissions, or to convert the Strings properly. Instead, it is often preferable to bind directly to your business objects.

spring framework manual :: web mvc chapter

Grails aims to bring the "coding by convention" paradigm to Groovy. It's an open-source web application framework that leverages the Groovy language and complements Java Web development. You can use Grails as a standalone development environment that hides all configuration details or integrate your Java business logic.

Grails home

Project build and dependency management tools

2008-12-01T12:58:00.002+02:00

Gant is a tool for scripting Ant tasks using Groovy instead of XML to specify the logic. A Gant specification is a Groovy script and so can bring all the power of Groovy to bear directly, something not possible with Ant scripts. Whilst it might be seen as a competitor to Ant, Gant uses Ant tasks for many of the actions, so Gant is really an alternative way of doing things using Ant, but using a programming language rather than XML to specify the rules.

Gant home

Ivy is a very powerful dependency manager oriented toward Java dependency management, even though it could be used to manage dependencies of any kind. Of course, Ivy is integrated with the most popular build management system for Java projects. But the integration goes way beyond common Ant integration. Indeed Ivy has been designed with Ant integration and design principles in mind. If you have Ant skills, you already have Ivy skills! The plugin mechanism in Ivy follows the same design as Ant, you will find macrodef and files import in Ivy configuration, many things Ant users are already familiar with.

Ivy features

Gradle is a build system which provides:
A very flexible general purpose build tool like Ant.
Switchable, build-by-convention frameworks a la Maven (for Java and Groovy projects). But we never lock you in!
Powerful support for multi-project builds.
Powerful dependency management (based on Apache Ivy).
Full support for your existing Maven or Ivy repository infrastructure.
Support for transitive dependency management without the need for remote repositories and pom.xml or ivy.xml files (optional).
Ant tasks as first class citizens.
Groovy build scripts.

Gradle home

Garbage First

2008-11-20T22:53:00.003+02:00

The new garbage collector that we're currently developing called Garbage First or G1 for short. G1 straddles the young generation - tenured generation boundary because it is a generational collector only in the logical sense. G1 divides the heap into regions and during a GC can collect a subset of the regions. It is logically generational because it dynamically selects a set of regions to act as a young generation which will then be collected at the next GC (as the young generation would be). G1 can choose the regions with the most garbage to collect first (Garbage First, get it?) so gets the biggest bang for the collection buck.

That G1 is now a part of JDK 7.

Object emission matters

2008-11-04T22:46:00.001+02:00

When we go looking at java performance, there are really 2 areas of the runtime that matter: the JIT and the GC. The job of the JIT is to make the code that is running execute as fast as possible. The GC is designed to take as little time away from the executing of code as possible (while still managing memory). Thus java performance is all about making the JIT generate more optimal code (more registers helps), and reducing the time the GC has to use to mange memory (bigger pointers makes this harder).

caida.org

2008-10-30T22:00:00.002+02:00

Wow again. Cooperative Association for Internet Data Analysis. Found lots of free internet topology data available. Some links:

Mapnet: Macroscopic Internet Visualization and Measurement Walrus - Gallery: Abstract Art