Igor Minar's Blog

Enough with the Flash Already

2008-06-30T07:00:00.000-07:00

It often bothers me when Flash is being misused for small visual effects on home pages of websites. Not just that it slows things down, but it also usually breaks these websites terribly when I want to copy&paste stuff or when one doesn't have the Flash plugin installed.

Margarete, has recently bugged me quite a bit about making the front page of her website a bit more "visual". She started with an idea of adding some flash to her website, which I dismissed quickly.

Fortunately I was able to come up with a neat little CSS code, which is way better than Flash for what she was trying to achieve. Don't be mistaken, I didn't even need JavaScript to do all the subtle movements and eager image loading, it's all pure CSS.

The front page:

A small "visual" effect when one mouses over the "Blog" element:

I still plan to enhance the code a bit more to make it faster to load by decreasing the number of HTTP requests needed to load the asset files as well as size of the images, but I'm quite happy with what it looks and feels like already. Check out the live version at www.fgslovakia.com.

A good technical article describing a technique very similar to the one that I used can be found the A List Apart website.

BTrace == DTrace for Java

2008-06-25T07:00:00.000-07:00

Last week, I was trying to nail down a bug in SunWikis that was triggered by some kind of race condition. These kinds of issues are pretty nasty, especially if the application in question is a pretty complex beast, something that Confluence definitely qualifies as.

The debugger was no help because the timing had to be very precise for the issue to occur, and logging was totally useless because of zillions lines of logs that would be difficult to safely filter out. In many cases the information I needed was not present in logs anyway and since my guess was that the bug was coming from 3rd party code, logged data couldn't be easily expanded. DTrace, which I blogged about in my previous post, could have revealed some information, but I think that it would be very difficult to write a good D script that could give me some clues.

While waiting for my dev server to restart, I came across an interesting blog post that caught my eye. It mentioned this thing called BTrace, that I hadn't heard about before. It promised it to be a DTrace aimed at Java apps. With a fair amount of skepticism, I navigated to the BTrace website and started reading. Four things about BTrace earned my interest:

It's based on same or very similar principals as DTrace, but specialized for Java apps
No restarts necessary, you can observe a running app by connecting to a JVM identified by a PID
The BTrace programs are written in a language that is a subset of Java, and heavily based on annotations. This was a bit of a turnoff for me at first, until I found out that the programs don't need to be compiled manually but btrace takes care of that for me. Bravo!
I can call DTrace from my BTrace programs if I need to instrument native code or the OS

Armed with all this info and my eyes sparking, I wrote my first BTrace program and it worked as advertised! No restarts, no compilation, no hassle.

A few hours later, I had a program that was observing exactly those parts of Confluence that I needed. All of this while the application was running uninterrupted. I was able to snoop on how my code was interacting with Confluence and Confluence was in turn interacting with Hibernate in a way that I would have never dreamed of. All of this while concurrent events aimed at triggering the bug were happening.

Running BTrace programs is as easy as this:


$ ./bin/btrace \
  -classpath hibernate-2.1.8-atlassian.jar:spring-hibernate2-2.0.6.jar \
  6266 HibernateSessions.java

Where 6266 is the PID of my app/web server and HibernateSessions.java is my BTrace program.

Parts of the program looks like this (check out the javadoc comments for explanation of the code):


import com.sun.btrace.annotations.*;
import static com.sun.btrace.BTraceUtils.*;
import net.sf.hibernate.HibernateException;
import net.sf.hibernate.collection.PersistentCollection;
import net.sf.hibernate.engine.SessionImplementor;
import org.springframework.orm.hibernate.support.OpenSessionInViewFilter;

@BTrace
public class HibernateSessions {
  /**
   * A thread local variable used to filter out all the events that we
   * are not interested it.
   */
  @TLS private static boolean viaSso = false;
  private static boolean printStack = false;

  /** print a message when a thread enters doFilter method of SsoFilter */
  @OnMethod(
      clazz="com.sun.dse.wikis.auth.SsoFilter",
      method="doFilter"
  )
  public static void enableProbing() {
      println(str(currentThread(), " - Entered SSO Filter"));
      viaSso = true;
  }

  /** print a message when exiting the doFilter method of SsoFilter */
  @OnMethod(
      clazz="com.sun.dse.wikis.auth.SsoFilter",
      method="doFilter",
      location=@Location(Kind.RETURN)
  ) 
  public static void disableProbing() {
      println(str(currentThread(),  " - Exited SSO Filter"));
      viaSso = false;
  }

  /**
   * print an message with a HibernateException is thrown, with detailed 
   * info about the current context
   */
  @OnMethod(
      clazz="net.sf.hibernate.HibernateException",
      method=""
  )
  public static void onthrowreturn(HibernateException self, String s)  {
      println(str(currentThread(), 
              strcat(" - HibernateException was thrown: ",
              strcat(s, strcat(" | collection: ", str(collection))))));
  }

  /**
   * Print a message when Hibernate is attaching a collection to a session.
   * This was crucial info for me to get in order to resolve the issue
   */
  @OnMethod(
      clazz="net.sf.hibernate.collection.PersistentCollection",
      method="setCurrentSession",
      location = @Location(Kind.RETURN)
  )
  public static void setSessionForCollection(boolean returnValue)  {
      if (returnValue == true) {
        println(str(currentThread(),
          strcat(" - a collection was attached to a session. Collection:",
          str(collection))));
  }
}

The result of this was a workaround for my code as well as a pretty detailed bug report that Atlassian confirmed to be a very similar to a confirmed bug that was reported recently. I could hardly achieve this without BTrace or without an in-depth knowledge of how all the Confluence code works.

As great as BTrace is, it is still a beta. If you decide to use it in production environment you should keep that in mind. During the several hours I spent working with BTrace, I experienced one JVM crash/lockup caused by BTrace. I bet that this kind of issues will be ironed out soon, and often this kind of risk is worth undertaking in order to resolve otherwise untraceable bugs.

Since last week, BTrace is part of my toolkit and I already have other mysteries on my ToDo list that I want to shine some light on with BTrace. I applaud the BTrace team for giving us this amazing tool!

DTrace and Java - Observations and Docs

2008-06-24T22:30:00.000-07:00

For a long time I've been hearing about DTrace and how cool it was. I read a lot about it and saw some presentation on observing Java apps with DTrace, but only recently have I found enough free time to extensively play with it. And oh boy, it really is cool. But even with all this coolness, dtracing Java apps feels a bit awkward.

I think there are two reasons for that. The D language, which looks like C, but has a few fundamental differences, is pretty far from what Java developers are used to. And the second reason is the fact that DTrace is in reality instrumenting JVM and not the application running in the JVM. This means that dtracing Java is limited by abilities of the Java DTrace provider and also that the transition between the different layers of the stack can add some awkwardness.

Having said that, DTrace is really awesome and can be enormously helpful in many cases when developing or troubleshooting (Java) apps.

One thing I really miss in DTrace is regex support in any part of D scripts. There is a limited globbing support and some workarounds, but they ain't pretty.

Here are some DTrace resources that I found useful:

DTrace Wiki Docs - the complete documentation (I love to see cool docs like this at our SunWikis)
a DTrace HowTo
DTrace for Java at Solaris Internals
DTrace Toolkit - a compilation of DTrace scripts written by DTrace pros

Catching StackOverflowError and a Bug in Regex Implementation in Java

2008-05-26T11:30:00.000-07:00

Shortly after we launched SunWikis almost a year ago, we started having an issue with StackOverflowError, which was occurring when the content of certain wiki pages was being parsed and URLs where being extracted. I documented the issue in CONF-9392.

The problem is not really a bug in Confluence (even though using an overly-complex regular expressions doesn't help the case), but it's a problem in JDK. After a brief search at bugs.sun.com I found the root cause of our StackOverflowError documented as bug 6337993.

Originally I tried to mitigate the situation by increasing the memory reserved for the stack data via the -Xss JVM parameter. This helped a little bit, but wasn't good enough in most cases.

Last week I decided to go against everything I've been taught, and wrote a patch for Confluence that wraps the part of code that results in the StackOverflowError into a try/catch block. I know that any throwables that extend from Error should not be caught by a client code because they usually indicate a failure that only JVM should try to recover from, but IMO in the case of StackOverflowError, the situation is a bit different. That is mainly because before throwing the StackOverflowError, JVM pops the stack, so by the time the code execution gets to the catch block, JVM has already recovered from the error.

I don't claim this to be a solution to the problem, it's just a workaround that works better than increasing the stack size in this particular case. The fact that Confluence doesn't find all the URLs in wiki pages (used mainly to list outgoing links in the page info view) is just a small sacrifice, compared to inability to save or copy the page.

As for the solution, it seems that reimplementing Java's regular expressions library would be the most suitable one. I tried to run a code that fails in Java in JRuby, which uses a port of Oniguruma regex engine for Java and things worked flawlessly and as I read it also gives JRuby a performance boost over java.util.regex.

Mediacast Deployment Diagram

2008-02-08T11:00:00.000-08:00

During our last meeting, Arun suggested that I publish the high-level deployment diagram of mediacast.sun.com. So here it is:

I described everything in textual form in my previous blog entry about mediacast.

JRuby on Rails Rewrite of mediacast.sun.com Launched

2008-01-27T20:16:00.000-08:00

A few days ago, we finally released Mediacast 2.0 - a complete rewrite of the old Mediacast application.

The original application was based on a few servlets, filters, and a lot of JSPs. It was all put together in hurry a few of years ago, when one of those fire-drill requests came to provide Sun employees with a site where they could publish files. The old code was hard to maintain and extend, and that's why we decided to EOL the old code base and write something better from scratch.

I'm a fan of Rails and for the past year I've been amazed by the great progress of the JRuby project. This was one of the reasons why I suggested that we could try to rewrite the app in Rails and deploy it in a regular Java web container, thanks to JRuby and Goldspike. It took some time for Rama to give us a go-ahead, but finally in late September, two other colleagues (with no Rails or JRuby experience) and I started to work on the rewrite alongside our other projects (forums.sun.com and wikis.sun.com).

Many people asked us why JRuby on Rails was picked for this project. Here is an incomplete list of reasons:

we were starting from scratch, so we were not tied to any legacy code, and could pick any web framework that runs on Java
proof of concept project to evaluate the technology for other uses across our organization
verify that Rails really delivers the rapid development promise
something new/fun for the team to help balance out the not-so-fun stuff :)

Here is my experience:

Rails and JRuby Learning Curve

I worked on my first Rails project in the summer of 2006 and since then I worked on a few other (internal) Rails projects. But having to teach others and deploy an application externally using JRuby was a new experience for me.

As far as teaching goes, I can't critique myself, but what I can say is that I had a lot of good help thanks to some Rails, Ruby and JRuby books, and many websites and blogs that have sprung up on the Internet in the last couple of years.

Learning JRuby (with previous Ruby and Java experience) is easy, because most of the time "it just works" and very rarely is the developer aware that the C-based MRI Ruby interpreter is not being used. The ability to access and seamlessly integrate with existing or new Java code is a huge plus without which we would not have been able to launch this project successfully (more on this later).

The deployment part was a different story. The Ruby on Rails application can be deployed in many different ways and JRuby offers a new alternative approach to all of them. Instead of e.g. having Apache webserver reverse-proxy requests to an army of Mongrels, JRuby and Goldspike make it possible to deploy Rails applications in a JavaEE web container. IMO this is much cleaner than anything else that is available in the non-JRuby Rails world. The downside is that this deployment method is quite new and there isn't a lot of documentation and community knowledge about it.

Application Architecture

Our app is a simple database-driven Rails application consisting of a handful of models, 4 controllers, and a bunch of view templates. The only piece of data that is not stored in the database are the actual files, which are stored on the file system.

To be on the safe side when it comes to performance, we employed quite a lot of fragment caching, which sped up our application quite a bit.

Development Environment

Our development environment is based on a self-contained¹ JRuby (1.0.3) on Rails (1.2.6) application stored in a Mercurial repository. The IDE we use is NetBeans 6 with Ruby support and Mercurial plugin. The DB of our choice is MySQL and servers we use during development are WEBrick and Glassfish v2ur1.

NetBeans makes it super easy to write the Ruby code and offers a lot of neat features for Rails application development. I have to say that Tor and the gang did a great job. In fact I switched from Eclipse to NetBeans thanks to its great Ruby support.

Because the application runs in the JVM we can use JConsole to monitor the app while load testing which comes in really handy!

So all is good here, except for one thing. I got used to using the amazing ruby-debug debugger for debugging my Ruby on Rails applications. This debugger is not yet available for JRuby on Rails application (unless you hack your way through). I read somewhere that jruby-debug should be available with NetBeans 6.1. Once that is done, the JRoR dev environment will be on par with RoR (in fact thanks to tools like JConsole it will be superior).

Production Environment

We use a pair of load-balanced T2000 with Solaris 10, JDK6 and SJSAS 9.1u1. These two servers share a nfs NAS drive used for file storage and fragment cache storage. The DB backend is a MySQL database server, which we access through a connection pool set up in the app server.

All of this was fairly easy to set up. As you can see there were no special requirements when comparing this environment to a usual JavaEE production environment.

The Good

Once my colleagues grasped some RoR basics, we got the core of the application up and running fairly fast. Sometimes it still amazes even me how much one can do with Rails in a short amount of time. Thanks to a small amount of code one needs to write, the code review process is fast as well, and fixing a bug often means changing only a few lines of code.

The Bad

Since most of the C-based gems are not compatible out-of-box with JRuby (at the moment), the jruby-extras project aims to deliver JRuby compatible versions of these libraries. One of these libraries - JRuby-OpenSSL - was needed for me to integrate our app with our authentication webservice. I soon found out that JRuby-OpenSSL was not completely implemented yet and parts of functionality that I needed were missing. I'm sure that it is just a matter of time when problems like this will go away (if it hasn't happened already) as these libraries will mature.

The Ugly

The Goldspike project provides a bridge between the JavaEE and RoR world. It does this by dispatching the incoming HTTP requests into Rails running in a JRuby runtime.

This works great for all requests that take little time to process, but if you have long-running requests like large file uploads or downloads, these requests will occupy your JRuby runtimes and you soon realize that you are running out of runtimes in your small pool, at which point your application becomes unresponsive for any new requests.

This was the biggest problem we hit, but thanks to the possibility to seamlessly integrate Java code into our JRoR app and a great idea that my colleague Peter had, we solved this issue quite elegantly.

JRuby and Java Come to the Rescue

The two main issues we faced and that consumed most of my time on this project were related to the immaturity of the libraries around JRuby and application characteristics specific to JRoR deployment. Both of these issues can be resolved thanks to years of long Java and JavaEE history, and their libraries.

The problem with reliable connection to our SOAP-over-HTTPS based authentication webservice was resolved by generating a client Java webservice stub and using that instead of SOAP4R which didn't work properly because of the problems with JRuby-OpenSSL.

The second problem with the long-running processes occupying our precious JRuby runtimes, we solved by using a servlet filter and a fake HttpServletResponse (that we sent to Goldspike instead of the original one) and streaming the data from the filter instead from the runtime. I'll write a separate blog entry on this.

Overall Impression

This project was/is fun to work on. We experimented with quite a few new (for us) technologies and learned a lot along the way. To be honest, I expected to deploy the app much sooner but at that time I was not aware of the two above mentioned issues which consumed a lot of my time.

I was a bit worried about the performance of the application, but that turned out to be a non issue once we had the download servlet filter in place, and with the performance improvements in JRuby 1.1 things will be even better in the future.

Overall I'm happy with the outcome of our project and I look forward to adding more functionality to the application.

Our Future Plans

From the infrastructure perspective:

upgrade to JRuby 1.1 and Rails 2.0
start using Warbler, which looks to be superior to Goldspike's Rails plugin, for building the war file
experiment with in-memory session state replication in Glassfish

From the feature perspective:

better categorization of media items
search functionality via integration with search.sun.com
previews of media items
new UI design
and the toughest one - audio and video streaming

JRuby/Goldspike/Glassfish Things I Would Like to See Improved

I'm not sure if it is us and our (mis)configuration, Goldspike, JRuby or Glassfish, but right now we need quite a big JVM heap to keep things running. With 8 JRuby instances in the pool and Http thread count set to 512, we need -Xmx set to 2-2.5GB. This is a bit too much I think. We'll have to look into this and find the culprit.
The application startup is quite slow, especially on a machine like T2000, which doesn't perform well for heavy-weighted single-threaded operations. I don't see a reason, why Goldspike couldn't initialize the JRuby runtime pool concurrently cutting down the startup time significantly.
Offload JRuby runtimes as much as possible. Currently operations like file upload or file download (via Rails' send_file) are handled by JRuby runtimes. I think that it should be possible to take care of these operations outside of the runtime, allowing the runtime to process other rails requests in the meantime.

¹ - Rails and all the other gems and jars are frozen into the project

On the Move (Away from Blogger)

2008-01-21T23:59:00.000-08:00

Today was a big day for my blog. I started a "big" move that was hanging over my mind for a few months.

I got my own domain igorminar.com and changed the blogger.com settings so that the domain name blog.igorminar.com is being used instead of net3x.blogspot.com.

Except for a small hick-up with DNS cache and a weird 404 Server not Found error (described in this blog post) everything went smoothly. My plan is to keep on using Blogger's custom domain feature for a few weeks and then migrate all the content over to my own blogging engine. I'll most likely end up using Mephisto, JRuby, Glassfish and MySQL, but things can still change.

The reason why I'm moving in two phases is to keep all the referring links that point to the old urls as well as most of the search engine ranking. The Blogger's custom domain feature now redirects everything to the new urls via 301 Permanent Redirect, so hopefully the transition will be close to painless. I'm not SEO expert, but this for sure is an interesting experiment.

And for those SEO "experts" that were claiming that getting Blogger to do proper redirects after a blog move was impossible here is a little proof:


$ wget --spider net3x.blogspot.com
--16:34:30--  http://net3x.blogspot.com/
      => `index.html.2'
Resolving net3x.blogspot.com... 72.14.207.191
Connecting to net3x.blogspot.com|72.14.207.191|:80... connected.
HTTP request sent, awaiting response... 301 Moved Permanently
Location: http://blog.igorminar.com/ [following]
--16:34:30--  http://blog.igorminar.com/
      => `index.html.2'
Resolving blog.igorminar.com... 66.249.81.121
Connecting to blog.igorminar.com|66.249.81.121|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 0 [text/html]
200 OK

My Custom Prompt for Leopard's Terminal

2008-01-20T23:20:00.000-08:00

As I mentioned in my previous post, even though it is not perfect, I settled on using Terminal as my terminal application.

The default prompt that comes with MacOS is not very useful, so I configured my own by adding this line to my .bashrc file:

export PS1="\[\033]0;\u@\h:\w\007\]\u@\h:\W\\$ "

This results in a prompt that looks like this:

This prompt displays the prompt as username@hostname:currentdir$ and in addition to that it sets the window title to username@hostname:pwd.

Leopard's Terminal Tabs

2008-01-20T22:55:00.000-08:00

As I shared yesterday, I love tabs. One of the few good things that MacOS 10.5 Leopard brought, was enhanced Terminal application. A basic feature - Tab support - that was missing in Terminal for way too long, was finally added, and Terminal app became finally usable for anyone who spends a great time of the day working on the command line.

Before Leopard, I used iTerm, but there was just something about it that I didn't like, but the availability of tabs made me get over it. Now with Leopard there is no need for iTerm any more.

Just to point out that Apple seems to be unable to get anything 100% right these days, there are two flaws in the Terminal's tab implementation.

The first flaw is that you can't easily switch between tabs using keyboard shortcut. All you can do is to use cmd+{ and cmd+} to switch to previous or next tab respectively. What a "wonderful" choice of keys that require use of right hand or both hands. Most importantly, one can't use the handy cmd+1, cmd+2, ..., cmd+9 shortcuts (or anything similar) to switch to one of the first nine tabs. Maybe Apple doesn't want to overload us with new features and is planning to add this to Terminal in MacOS 10.6. :-) Fortunately, the cmd+{ and cmd+} shortcuts can be remapped via System Preferences.

The second flaw is that Terminal doesn't let you name the tabs or configure how the tabs should be named. This is a useful feature that many other terminal applications have, but Apple decided that using the name of the program of the currently running process is good enough. Well, this is what it looks like:

People are doing all kinds of crazy things to work around this issue. Including using hardlinks to change names of programs or creating dummy processes that carry the tab name.

I would be very happy if Apple did something similar to what xterm does - using an environmental variable to set name of the tab dynamically. But I have a feeling that we'll see MacOSX Trash go 3D sooner than properly working Terminal tabs. :(

Mac Applications That I Use

2008-01-20T16:15:00.000-08:00

I've been meaning to write this entry on applications installed on my computer for a few months now, but never found time to do it. Hopefully none of those new Mac users that I told to check out my blog "in a few days", several months ago, got angry at me. If nothing else, my promise helped me to get more regular readers :).

Anyway, here is the list of apps installed on my Mac (all these apps work with Leopard, unless I noted otherwise):

Essentials

A Mac without these apps would never be the same:

Quicksilver - an app that helps you find things on your computer and execute actions on these things. I use it to launch apps, pause iTunes, find contacts in the AddressBook, and many many many other things. Most of the cool functionality is added via plugins, so don't forget to checkout the plugin section of preferences. This app is a must have. Here are Some docs and a good video.

Adium - a really handy IM client that can handle all the popular IM protocols. Definitely a must have. (it's much better than iChat when it comes to IM-ing).

Growl - Growl is a notification system for Mac OS X: it allows applications that support Growl to send you notifications. Notifications include Adium or Skype messages, network up/down events, volume mount/unmount events, currently played iTunes song and many others. More info - One extension that I found to be incompatible with Leopard is GrowlMail for Mail.app notifications, when I received many new messages at once this extension would cause the Mail.app to crash.

MacPorts - This is a geeky one: The MacPorts Project is an open-source community initiative to design an easy-to-use system for compiling, installing, and upgrading either command-line, X11 or Aqua based open-source software on the Mac OS X operating system.

Firefox - no description needed :)

Nice to Have

iStat menus - a nifty app that displays stats about cpu load, cpu temperature, memory and network utilization, and a lot of other interesting data about your Mac. I use it as a replacement of MenuMeters which I couldn't get to run reliably on Leopard.

Perian - Perian is a free, open source QuickTime component that adds native support for many popular video formats.

Camino - yeah, as if the fact that I'm obsessed with tabs was not enough, I usually run 2 or 3 browsers at the same time to be able to take advantage of their unique features, or more often, while developing webapps, I need to be able to have more than one session open.

Chicken of the VNC - A handy VNC client

MacFUSE + sshfs - MacOS X implementation of FUSE, which makes it possible to painlessly mount all kinds of different filesystems e.g. sshfs.

MacFusion - GUI frontend for MacFUSE

VLC - alternative to QuickTime video player with build-in support for all kinds of video and audio codes, subtitle support and many other features

Vine Server (OSXvnc) - a good VNC server in case you need a bit more control than the Leopard's built-in server offers

Skype - I have a love-hate relationship with Skype. As soon as I find an alternative crossplatform communication tool with audio and video support and cheap calls to Slovakia, I'll switch. Maybe Gizmo will become that tool one day

NeoOffice - don't forget to install the latest Patch. I have big hopes for OpenOffice Aqua to be released soon. My initial experiments revealed some very good performance results.

iShowU - a cool app for creating screencasts. This is the only paid-for app on my list. But it's well worth it.

Caffeine - Caffeine is a tiny program that prevents your Mac from automatically going to sleep, dimming the screen or starting screen savers. Very useful during video calls, while reading PDFs or watching movies.

Twitterrific - a Twitter client

Azureus - Java BitTorrent client

SkeyCalc - An OTP (S/Key) calculator

Cisco VPN Client

Development

NetBeans - there is a lot of controversy in the community when it comes to the Eclipse vs NetBeans question. I used Eclipse for a really long time, but ever since I tried NB6 Milestone 8 or so, I started liking NB more than Eclipse. I'll give Eclipse another shot in a few weeks, but for now NetBeans is my default IDE. And as far as Ruby/JRuby development is concerned, I don't think that this is going to change for a long time.

JRuby - a pure Java-based Ruby interpreter

GlassFish - JavaEE 5.0 compliant application server

Other Stuff

Drivers for my Vydeo eSATA 34: Dual Port 34mm Express Card - More info in my older blog entry

Other Lists

Rama and Martin composed similar lists some time ago, so you might want to check those out as well.

I'm Obsessed with Tabs

2008-01-20T01:00:00.000-08:00

This is a screenshot I took during a regular development/research day:

What is your average number of browser tabs opened at once?

MultiSudokuSolver - A fun project I worked on during the Xmas break

2008-01-18T11:45:00.000-08:00

While I was visiting my family in Slovakia during the winter 2005/2006 I found a sudoku magazine in my dad's apartment. I like this kind of stuff, so I mediately started solving one puzzle after another, until I got to a "Speciality MultiSudoku" puzzle:

This puzzle is composed of five regular 9x9 puzzles that are interconnected and you can't solve either of them alone, meaning that you need to be solving all five of them at the same time and use intermediate results from one puzzle to get an intermediate result for another puzzle that this puzzle is connected with. It sounded like a good challenge so I started working on it. After two or three nights I realized that I made a mistake!! Ughhh. I erased everything I had solved and started from scratch. After another few nights I found another problem, got turned off by this puzzle and put it away.

I came across the puzzle once again when my wife and I returned to Slovakia this winter. I found the puzzle in the apartment and, being a person who likes challenges and doesn't get turned off my failures for too long, I decided to erase everything and this time be very careful and solve the puzzle once and for all.

After about two evenings I found something I didn't want to see. An error!!! It was hard to believe it, but there it was. I tried to fix it, but if you don't spot an error in a sudoku puzzle early enough, you'll spend more time fixing it than if you started from scratch.

Do you think I felt like starting from scratch? No! But I couldn't let this be. So since I had 10 or so days off during the Christmas break (thanks Sun!!!), I decided to use my brain in a more productive way and to write a small program that would solve the puzzle for me.

Given my interest in Ruby and JRuby and the type of problem I was about to solve, the language choice was an easy one to make.

A few days later I had a script that solved the puzzle for me within 1.07 seconds. Ya! You heard right! Solved not in a few evenings but in just a little more than one second.


Top Left:
718|956|324
452|387|619
936|142|587
-----------
341|528|796
695|713|842
287|694|153
-----------
523|461|978
869|275|431
174|839|265

Top Right:
635|428|971
214|679|853
897|531|426
-----------
976|382|514
152|764|389
348|915|267
-----------
463|857|192
529|146|738
781|293|645

Bottom Left:
728|154|396
194|836|752
653|279|184
-----------
519|368|427
346|725|918
872|491|563
-----------
231|987|645
467|512|839
985|643|271

Bottom Right:
215|849|637
348|627|591
976|153|824
-----------
869|431|752
154|972|368
732|568|149
-----------
521|796|483
483|215|976
697|384|215

Center:
978|215|463
431|678|529
265|439|781
-----------
813|754|692
649|382|157
527|961|834
-----------
396|847|215
752|196|348
184|523|976

Lessons learned: "automate automate automate!" and "Don't work hard, but work smart!" :)

The algorithm is very simple. The model is based on the simplest unit - a cell which is part of a row, column and a square. Each puzzle consists of 9 columns, 9 rows and 9 squares. Cells are flexible enough to be part of more than one square, row or a column at a time, so I can have cells that are present in more than one puzzle at a time. With a flexible model like this, all that the program needs to do is to use some basic rules to eliminate candidates and determine cell values. This type of solver is often referred to as human-style solver, because it uses the same techniques used by humans.

If anyone is interested in having a look at the source code it can be downloaded from here: multi_sudoku_solver.rb. A word of warning - the code is not cleaned up nor documented. This was just a fun project that I worked on, and the fact that I achieved what I set out to achieve was a good enough milestone to call this project done.

After this experience I don't feel like solving sudoku puzzles any more. They are just a repetitious problems that are well suited for automated solution. Most importantly - I actually had more fun writing the program than I had with solving the puzzle :).

Shure SE210 Earphones (upgrade from E2c)

2007-11-23T19:20:00.000-08:00

When I bought my Shure E2c earphones two years ago, I was blown away by the sound quality. I never spend more than $50 for earphones before and after buying the E2c model I would never consider downgrading to anything less. Shure really knows how to create addictive high quality products. The quality of the sound produced by my old E2c earphones was so good that I had to get rid of some albums that I ripped into mp3s long time ago when hard drive space was measured in megabytes rather than in gigabytes.

After two years of slave-like everyday service my Shure E2c earphones started to show signs of wear and it became apparent that they were going to retire soon.

Afraid of being without a good pair of earphones during my upcoming transatlantic flight, I got a new pair of Shure SE210. The design has change slightly compared to the older model, but it still preserves the main features that I loved so much.

I've been listening to some music via these earphones for only an hour or so, but I can already tell that SE210 brings "great sound" to a brand new level. I don't consider myself to be an audiophile, but even my untrained ear can hear the big difference between the two. The new pair fits better as well as feels more comfortable in my ears.

J, lost his mind a couple of weeks ago and bought a pair of the high-end SE530. I don't even want to know what those sound like. I know that if I tried them, I would be tempted to pay an insane amount of money for them, so I better say away :)

Inflector/environment.rb Bug in Rails

2007-10-29T09:30:00.000-07:00

Today I needed to get Rails to grok an uncountable noun "media". Easy you think, right? Just add an inflection rule.

So I went to config/environment.rb, uncommented the Inflection example and modified it with my "media" noun, just like this:

Rails::Initializer.run do |config|
  ...
  ...
  ...
  
  Inflector.inflections do |inflect|
    inflect.uncountable %w( media )
  end

  # See Rails::Configuration for more options
end

# Add new mime types for use in respond_to blocks:
# Mime::Type.register "text/richtext", :rtf
# Mime::Type.register "application/x-mobile", :mobile

When I tried to run rails console or tests I saw an error like this:


me@Xbook:testapp$ ./script/console 
Loading development environment.
./script/../config/../config/environment.rb:55:NameError: uninitialized constant Inflector
/opt/local/lib/ruby/gems/1.8/gems/actionpack-1.13.5/lib/action_controller/assertions/selector_assertions.rb:525:NoMethodError: undefined method `camelize' for "top":String
/Users/me3x/Development/DSE/mediacast2/apps/testapp/app/controllers/application.rb:4:NameError: uninitialized constant ActionController::Base

The problem turned out to be in the commented-out rails example in environment.rb. The Inflector.inflections block must not be nested within the Initializer.run block.

This fixes the problem:


Rails::Initializer.run do |config|
  ...
  ...
  ...

  # See Rails::Configuration for more options
end

Inflector.inflections do |inflect|
  inflect.uncountable %w( media )
end

# Add new mime types for use in respond_to blocks:
# Mime::Type.register "text/richtext", :rtf
# Mime::Type.register "application/x-mobile", :mobile

10 Things I Hate About MacOS X Leopard

2007-10-29T09:05:00.000-07:00

Almost everyone is hyped about the new release of MacOS. All those great new features. Auweeeeee! I'm a an Apple fan, for now - I'm starting to struggle to justify it. Leopard does have a lot of nice features, but it is missing a lot as well. Considering the delay of the OS, I'm disappointed mainly about these ten things:

JDK6 is missing - WTF Apple!?!?!?
Only read-only support for ZFS, unless you count the developer preview version at ADC
~~Where is Xray? - The neat DTrace GUI that used to be promoted on Apple.com website a few weeks ago~~ Update: The app is now called Instruments and is part of developer tools. Thanks Ron and Barry for pointing this out to me
JDK6 is missing!!!!! Are you kidding me? Even James Gosling who liked to work on Macs, left Mac for Solaris
MacPorts break here and there - This might not sound like Apple's fault, but considering that Fink and MacPorts are the two main ways to get unix apps installed on Mac, Apple should have cared more about making sure that "things will just work"
Case sensitive FS is still only a second class citizen - I'm sick of all the dupes that iTunes and iPhoto creates, I'll be migrating to case insensitive FS soon
Mac Mini and possibly some other Macs, currently come with "upgrade only" Leopard DVDs - this means that if my Mac goes fubared, I'll have to install Tiger and only then I can install Leopard. Lame!
Why are Spaces one of the major features of Leopard? If I were Steve Jobs, I'd have added it long time ago, or if I added it only now, I'd be quiet and try not to attract too much attention to my past failures
JDK6 is still missing! - Seriously how much longer do we have to wait?!?!?
Why did Apple bothered to include built-in Ruby on Rails? Or should I say broken Ruby on Rails!?!? There was nothing wrong with installing Rails via MacPorts, but the one from Apple is already outdated and when I tried to upgrade it, it broke

MacOS X Leopard is "the most impressive Mac OS X version yet"? It's pretty good but hardly the most impressive without the latest Java.

SunWikis Used for San Diego Fire Emergency Coordination

2007-10-24T13:20:00.000-07:00

I guess today is one of those days when I can really feel that what we do at wikis.sun.com is of great value - potentially life saving value.

As almost everyone in the world knows there are big fires in San Diego right now - even my family from Slovakia contacted me yesterday to check that my wife's family in San Diego is ok.

Today I noticed that SunWikis are being used for an emergency coordinations at Sun's San Diego campus: http://wikis.sun.com/display/Fires.

I'm glad that we could help the guys in SD and I hope that everyone is fine over there.

How to Install Mercurial Support in NetBeans

2007-10-17T18:00:00.000-07:00

Getting NetBeans to grok Mercurial is pretty simple when you know all the gotchas. I already got stuck on the installation twice so I decided to turn this into a blog entry, so that I know what to do when I see the problem again. :)

To install Mercurial support in NetBeans, do the following:

Get Mercurial and install it. On Mac you can install it via MacPorts
Get NetBeans 6 and install it
Install the Mercurial plugin via Tools->Plugins->Available Plugins->Mercurial
Restart the IDE
If Versioning->Mercurials in the menu bar is disabled, go to Tools->Options->Miscelanious->Mercurial (on Mac NetBeans->Preferences->Miscelanious->Mercurial) and verify that Mercurial executable path is set up correctly - in my case, since I installed Mercurial via MacPorts, the path should be /opt/local/bin
Restart the IDE

The problem, I encountered already twice, is that if the Mercurial executable path is not set up properly, the Versioning->Mercurials menu item is disabled and NetBeans doesn't give you any hint as to what went wrong. For some reason the path on my Mac is by default pointing to my python directory, which is not correct. Once the path is corrected and the IDE is restarted, everything works as expected.

Apple to Bundle a Wiki Server with MacOS X Leopard Server

2007-10-17T10:33:00.000-07:00

Now that Apple finally released information about MacOS X Leopard, I learned that a wiki server will be part of Leopard Server OS.

Mac OS X [Leopard] Server makes it easy for groups to collaborate and communicate through their own wiki-powered intranet website complete with group calendar, blog, and mailing list. Users can create and edit wiki pages, tag and cross-reference material, upload files and images, add comments, and search content with point-and-click ease. (Source)

Pretty interesting, right? I wonder how the wiki server is implemented. I'd expect to see a lot of eye candy, seamless integration with other Apple offerings, but I'm not holding my breath for too many features.

The website doesn't offer many details, but it seems that the wiki server is intended mainly for team collaboration and not for external facing sites like wikis.sun.com. I hope that I'll get more information soon. MacOS Server is not that easy to see around here. No wonder, after all, we have our cool Solaris everywhere. :)

SunWikis Update: 1000+ Wiki Pages in 10 Weeks!

2007-10-11T13:25:00.000-07:00

Today it happened! We made it! Let's go and celebrate! The wikis.sun.com community created altogether more than 1000 unique wiki pages in just 10 weeks since our launch.


Number of users             1144
Number of spaces              94
Number of pages             1006
Number of page revisions    6362

These are just number, but numbers that mean great adoption of this fairly young project.

A big thank you to everyone who planted the seed!

Adding the OpenSearch Descriptors to Confluence or Other WebSites

2007-10-08T01:09:00.000-07:00

Earlier today I blogged about the browser search plugins for SunWikis and Confluence.

This is how it all works:

1. Create the descriptor file

Create an OpenSearch descriptor (OSD) file:


<?xml version="1.0" encoding="UTF-8"?>
<OpenSearchDescription xmlns="http://a9.com/-/spec/opensearch/1.1/">
   <ShortName>SunWikis</ShortName>
   <Description>SunWikis (wikis.sun.com) Search</Description>
   <InputEncoding>UTF-8</InputEncoding>
   <OutputEncoding>UTF-8</OutputEncoding>
   <Image width="16" height="16" type="image/x-icon">http://wikis.sun.com/favicon.ico</Image>
   <Url type="text/html" method="GET"
template="http://wikis.sun.com/dosearchsite.action?searchQuery.queryString={searchTerms}&searchQuery.spaceKey=conf_global&searchQuery.lastModified="/>
</OpenSearchDescription>

Download

Check out the template attribute - the {searchTerms} will get replaced with the keyword typed into browser's search box. Keep in mind that if your template url contains an "&" character, it needs to be escaped as &, otherwise your XML will not be well formated.

2. Host the descriptor file

Once you have the file ready, host it on your server e.g. as wikis.sun.com/search/wikis-osd.xml

3. Expose the descriptor file

Now you need to let browsers know about the descriptor. You can do this by placing a snippet like this somewhere between the head HTML tags.


<link rel="search" type="application/opensearchdescription+xml"
   title="SunWikis" href="$req.contextPath/search/osd.xml">

In the case of using Confluence, you can do this by editing your template.

The $req.contextPath part is Confluence specific. Replace it with a piece of code that retrieves the context path or replace it with static context path e.g. /myapplication.

4. Bonus: Click-to-install link

Firefox and IE support the installation of search engines via JavaScript, which can be handy if you want to provide an "install" link.

As usual, both browsers use a different approach, and to be fair IE is ahead of Firefox in this one. :-/

Here is a JavaScript I put together based on some resources


function addEngine(srcUrl, osdUrl, faviconUrl, fullName, category)
{
 if ((typeof window.sidebar == "object") && (typeof window.sidebar.addSearchEngine == "function")) /* Firefox */
 {
     window.sidebar.addSearchEngine(
         srcUrl,                                                /* engine URL */
         faviconUrl,                                            /* icon URL */
         fullName,                                              /* engine name */
         category);                                             /* category name */
 } else { /* IE */
    if (window.external.AddSearchProvider) {
       window.external.AddSearchProvider(osdUrl);
    }
 }
}

Download

For some weird reason (legacy?), Firefox uses Apple's Sherlock format for JavaScript installation instead of OpenSearch descriptors. For this reason you need to create a Shrelock descriptor based on your OpenSearch descriptor. Keep in mind that the Shrelock descriptor filename must end with a .src extension.

In my case, my sherlock.src looks like this:


<search
version="1.0"
name="SunWikis"
description="SunWikis (wikis.sun.com) Search"
action="http://wikis.sun.com/dosearchsite.action"
searchForm="http://wikis.sun.com/dosearchsite.action"
queryCharset="UTF-8"
method="GET">

<input name="searchQuery.queryString" user>
<input name="searchQuery.spaceKey" value="conf_global">
<input name="searchQuery.lastModified" value="&">

</search>

It's not perfect because it is deliberately missing the browser element, but it will do. :)

The user attribute is an equivalent of the {searchTerms} variable in OpenSearch specification.

Once the file is created, upload it to your site, e.g. wikis.sun.com/search/sherlock.src.

And now you can create the link on a page that contains the JavaScript code mentioned above:


<a onclick="addEngine('/search/sherlock.src', 
    '/search/osd.xml', '/favicon.ico', 'SunWikis Search', 'Wiki')" 
    href="#">Install
</a>

For more information on this topic check out this blog, but watch out for an XSS hole - the searchTerms variable in the JavaScript snippet in the section "Step 3: The Search" must be url-encoded before it is used later in the script.

Some extra Confluece-specific stuff

You can create OSD files that search in all the spaces of your Confluence instance or in one space only. You can specify this via the Url attribute in the OSD file.


<Url type="text/html" method="GET"
template="http://wikis.sun.com/dosearchsite.action?searchQuery.queryString={searchTerms}&searchQuery.spaceKey=conf_global&searchQuery.lastModified="/>

The searchQuery.spaceKey=conf_global part means - search in all Confluence spaces (with the exception of personal spaces IIRC).

You can replace conf_global with a space key and you get a descriptor for space specific search.

For example, the url for a descriptor for our About space would look like this:


<Url type="text/html" method="GET"
template="http://wikis.sun.com/dosearchsite.action?searchQuery.queryString={searchTerms}&searchQuery.spaceKey=About&searchQuery.lastModified="/>

If you want to get really fancy you can use the power of the Lucene search engine (used in Confluence) to craft all kinds of interesting descriptors. For example this one that searches all the Confluence related spaces (the DOC, DISC, and CONFEXT) at confluence.atlassian.com by specifying the spaces in the earchQuery.queryString parameter instead of the searchQuery.spaceKey, which accepts only one space key.


<?xml version="1.0" encoding="UTF-8"?>
<OpenSearchDescription xmlns="http://a9.com/-/spec/opensearch/1.1/">
   <ShortName>Confluence Extras</ShortName>
   <Description>Documentation for the latest version of Confluence and Confluence Community + Confluence Extension Docs</Description>
   <InputEncoding>UTF-8</InputEncoding>
   <OutputEncoding>UTF-8</OutputEncoding>
   <Image width="16" height="16" type="image/x-icon">http://confluence.atlassian.com/favicon.ico</Image>
   <Url type="text/html" method="GET"
           template="http://confluence.atlassian.com/dosearchsite.action?searchQuery.queryString=spacekey%3A%28DOC+OR+DISC+OR+CONFEXT%29+AND+%28{searchTerms}%29&searchQuery.lastModified="/>
</OpenSearchDescription>

Download

This OSD is based on a wicked Lucene search query that looks like this:

spacekey:(DOC OR DISC OR CONFEXT) AND({searchTerms})

You can find out more about Lucene goodies here.

The last word of advice - instead of creating n+1 descriptors, one for each space + one for the search in all the spaces, use a JSP page to dynamically render the OSD file - that's how I implemented it for SunWikis.

Browser Search Plugins for SunWikis and Confluence

2007-10-06T22:00:00.000-07:00

One of the small but useful features of the next release of SunWikis, due some time next week, will be support for Open Search descriptors.

Open Search descriptors (OSD) help browsers (and other clients) to discover ways to search content of a website. If you go to a site that supports OSD and you are using an Open Search enabled browser, the browser will notify you about the possibility of adding the current site's search among the browser's search plugins (Firefox does this by glowing the down arrow in the search box).

Once the search plugin is installed, you can search SunWikis directly from your browser's search box.

While I was at it, I created this descriptor for Confluence documentation located in the DOC wiki space at confluence.atlassian.com and a second descriptor that searches the DOC, DISC (Confluence Community), and CONFEXT (Confluence Extensions) wiki spaces at the same time.

So now I can search SunWikis and Confluence docs with ease:

If you are interested in installing these plugins as well (even before we make them available at wikis.sun.com), click on the links below:

Warning: As for SunWikis, you can install the search plugin now, but once the next SunWikis release is out, you should probably reinstall it just in case we tweak and enhance it.

Name	Content	Click to Install¹	Download OSD²
SunWikis	wikis.sun.com all spaces	Install	Download
Confluence	confluence.atlassian.com the DOC space	Install	Download
Confluence+Extras	confluence.atlassian.com the DOC, DISC, and CONFEXT spaces	N/A³	Download

Who knows, maybe Atlassian will use my code and officially support these search descriptors on their site - they have my full support.

As for technical details, I ~~plan to write~~ wrote another entry just about how I created and added these descriptors to Confluence.

¹ Might or might not work for your browser. Click and sit tight :)
² The OSD file can be used for an alternative installation method - If you have Firefox and Mac you can place the file into /Users/[yourname]/Library/Application\ Support/Firefox/Profiles/[the profile name]/searchplugins/ and restart Firefox, for other platforms, check out this doc
³ Due to limitations of the Sherlock format, it is not possible to craft a "click-to-install" file for the 3rd plugin, however you can still install it using the alternative² installation method.

AT&T's Language Mix-up in the "Philawarepragueacago" TV Ad

2007-10-06T14:05:00.000-07:00

Last night I saw a new TV commercial for AT&T called “Philawarepragueacago”. In this ad a, guy studying and living in Prague says "How are you, brother?" supposedly in Czech. The problem is that he doesn't say it in Czech at all. He says it in Slovak! :)

After I heard it the first time I wasn't sure if I heard it correctly or not, but after a quick Google search I found the commercialy green blog dissecting and analyzing the commercial.

Commercially green was not able to transcribe the supposedly Czech line, so here it is:

Jak sa máš, brácho?

Which in casually spoken Slovak means:

How are you, brother?

The same thing in Czech would be:

Jak se máš, brácho?

It might be a small difference for Americans but I caught the difference right away.

Maybe if the director of the commercial read the book, The Foreigner's Guide to Living In Slovakia, he'd know that there is a difference between the Czech and Slovak languages.

The only explanation that AT&T can get away with is that there are a lot of Slovaks studying at Prague's universities, so it's possible that the "brother" has Slovak room-mates from which he picked up some Slovak. :)

What Would You Do with SunWikis Remote API?

2007-10-04T12:20:00.000-07:00

As I mentioned previously, one of the features for wikis.sun.com that I'm working on is opening up the remote API. Before we do so, we need to customize the current behavior in order to make it a bit more secure. Before the feature is in production, I'd like to know how you would like to use it, so that we can factor that in, while customizing it.

The main problem with the remote API right now is that it is either on or off. There is nothing in between. Because the remote API is very powerful (you can do everything that you could do via the browser and more), we'd like to limit who can use it in order to avoid having to clean up the mess created by reckless or malicious clients. Right now we are thinking about limiting the client IP addresses that can connect to the remote API, and/or limit the access to the service based on user's group memberships or user account permissions.

I'd like to get a better understanding of if and how would you use the remote API for wikis.sun.com, so that we can provide you with the best solution that fits needs of the majority.

Some use cases that come to mind are:

use an existing command line client to access SunWikis
a (command line) tool that takes your data in some (XML) format, transforms it to our wiki markup and uploads it to your space
an application that creates an XYZ report from your database, build server, or other data source and publishes it in a wiki space every night at 3am
wikification of an existing application a la JDocs

Feel free to come up with another (crazy) idea and let me know about it (leave a comment on my blog or send an email to our mailing list (internal only)).

For those who want the technical details:

Each approach of limiting the access that I mentioned above has it's pros and cons. IP address restrictions are suitable for the integration of SunWikis with another web application or web service. On the other hand, it won't work well if the remote API is being accessed from your notebook or desktop that doesn't have a stable IP address.

The restrictions based on group memberships (e.g. only Sun employees can access the API) are very well suited for personal use (e.g. the uploading multiple wiki pages or the renaming of wiki pages in batches), but won't work for web application integration, because employees would have to include their passwords in these applications, which is not a smart thing.

Another problem with both of the previous approaches is that a bug in a client could affect content in all the wiki spaces, because Sun employees have write access to all spaces by default.

A compromise might be to:

let a user create special account used only for remote access
add this account to a special group (would be automated)
grant write access to this account on a per-space basis (requires space admin intervention - the same as how permissions are being granted to external users)

This way a space admin (a Sun employee), has to endorse such an account (by granting it the write permission). In addition to that, this account will have access only to this space or maybe a few other spaces (if it is granted access to more spaces), which limits the potential damage.

Maybe there is another way to let trust-worthy folks use the remote API - if you know about something, let me now. ;-)

If you can't wait to start using or even writing clients for SunWikis, Atlassian has some good Confluence documentation that might be interesting for you if you have big remote API plans. Most importantly check out the Remote API Specification. Other interesting stuff: Remote API Examples, Confluence SOAP Client in Java

API Design - A Lesson Learned

2007-09-22T22:30:00.000-07:00

In one of my older posts, I mentioned that I watched an excellent video of Joshua Bloch's presentation on API design.

Joshua mentioned that these days, developers are API designers and often don't even realize that. It's very easy to create a bad API when you don't realize that you're actually creating an API. And that's exactly what happened to me this week.

One of the new features for SunWikis, we released with v1.0.6, was a macro for embedding YouTube and Google videos. A simple features implemented via Confluence's user macros - it was so simple that I actually managed to make it more complicated than it needed to be. All of that just because I wasn't aware of all the options that Confluence had to offer.

My original macro sytax for this feature was:

{video:youtube}<insert the youtube id here>{video}

Only later this week I realized that I could have made the syntax simpler by creating a user macro without a body. The syntax of the video macro without body looks like this:

{video:youtube|<insert the youtube id here>}

This makes the macro code shorter, and as my friend J commented, it helps to make the macro syntax look less like an ugly version of XML.

Compare the old and the new code used to embed our promo video:

{video:youtube}tpR_6y-OoRc{video}
{video:youtube|tpR_6y-OoRc}

The second one is obviously shorter and a bit simpler.

I was lucky enough that I realized this soon enough to be able to fix it before it started to be used at wikis.sun.com. So yesterday I changed the macro code, updated the documentation and fixed all the few pages that already were using the old clunky syntax.

This made me realize that Joshua missed one point in his presentation:

Be more than familiar with the language, protocol or libraries you use to create the API.

In the Java language this means use all the nice features that can make the lives of your users easier (e.g. annotations, generics, ..). In Ruby there are plenty of great idioms one can follow, and the same applies to Confluence's user macros - learn how they work, what they can do, and how you can leverage that.

Lesson learned! Next time I'll think twice and read more docs before I design any API.