Ken's Blog

Swivel4J - A New Java Client for Swivel

2009-11-30T21:52:00.000-08:00

At work, we were looking for a simple way to visualize business data without having to build and host it ourselves. This led us to Swivel (the new Swivel as they call it which is geared towards private data shared within an organization). With Swivel, we can upload our data, create pretty charts and graphs, and share them privately within our company.

Swivel has a REST API for managing the charts. Though not difficult, we found it a little unwieldy to have to use an HTTP client to setup each API call. This naturally led to a small library that wrapped the REST API calls in a native Java API.

We decided to make this library available to everyone as Swivel4J which is open source and hosted on Google Code. We hope this makes it easier for Java, Groovy, and Scala developers to get started with Swivel and promotes the use of Swivel in general.

Please try it out and give feedback, submit patches, etc.

Amazon ELB - Capturing Client IP Address

2009-09-12T20:19:00.000-07:00

If you're using Amazon EC2's Elastic Load Balancer (ELB) for load balancing web applications, you may have noticed that in your web access logs, the remote host IP address is the same for every request. The IP address you see is the private IP address of the load balancer.

If you want to see the IP address of the client (called remote host in access log documentation), you'll need to look at the value of the X-Forwarded-For request header which ELB populates when it forwards the request.

This can be achieved in an Apache access log by using the syntax:

%{X-Forwarded-For}i

You Apache log format would then look something like this:

LogFormat "\"%{X-Forwarded-For}i\" %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-agent}i\"" combined-elb
CustomLog log/acces_log combined-elb

See Apache Custom Log Formats

If you're using a Tomcat application server, you could define an access log Valve like this:

<Valve className="org.apache.catalina.valves.AccessLogValve"
    directory="logs" prefix="access_log." suffix=".txt"
    pattern="%{X-Forwarded-For}i %l %u %t &quot;%r&quot; %s %b &quot;%{Referer}i&quot; &quot;%{User-Agent}i&quot;"
    resolveHosts="false"/>

See Tomcat 6 Valve Configuration Reference.

If you're trying to capture the client IP address within your application code, simply use whatever API you have to read the X-Forwarded-For request header. For example in Java use the HttpServletRequest:

String clientIpAddress = request.getHeader("X-Forwarded-For");

instead of

String clientIpAddress = request.getRemoteAddr();

Serving Gzipped Javascript Files from Amazon CloudFront

2009-08-31T22:49:00.000-07:00

It is well-known that using a content delivery network (CDN) and compressing an HTTP response with gzip can significantly improve the performance and reduce the cost of a web site. If your CDN is Amazon's CloudFront, you'll face difficulties serving gzipped content to browsers that support it. Most web servers are able to examine the HTTP request headers sent by the browser and dynamically choose whether to deliver compressed or uncompressed content. CloudFront, however, does not yet offer this feature.

People have overcome this problem by manually inspecting the Accept-Encoding request header for a dynamically served page and writing references to either original or compressed files accordingly using a naming convention like myscript.js and myscript.js.gz for regular and compressed files respectively.

If all your pages are statically served from CloudFront, however, there is no opportunity to inspect the request headers. Javascript running on a statically served page has no ability to ask the browser if it supports gzipped content. We found ourselves in this situation and resorted to the following solution to determine if the browser supports gzip:

1) Create a small gzipped file, gzipcheck.js.jgz, and make it available in CloudFront. This file should contain one line of code:

gzipEnabled = true;

2) Use the following code to attempt to load and run this file. You'll probably want to put it in the HTML HEAD section before any other Javascript code.


<script type="text/javascript" src="gzipcheck.js.jgz">
</script>

If the file loads, it sets a flag, gzipEnabled, that indicates whether or not the browser supports gzip.

Use the result to drive a file naming convention for references to other static files. For example, you can upload and reference each compressed Javascript file with an additional .jgz extension. Why .jgz instead of .gz? Because of an annoying limitation of Safari.

When you upload your files to S3/CloudFront, make sure to set the proper HTTP response headers on your files that end in .jgz:


Content-Encoding = gzip
Content-Type = application/x-javascript

And if you want browsers to cache your files forever (almost):


Cache-Control = max-age=315360000
Expires = Tue, 31 Dec 2019 20:00:00 GMT

Thanks to Ricardo Rangel who helped design and code this solution.

Ubuntu License Plate on the Santa Monica Freeway

2009-04-08T09:49:00.000-07:00

I came across this car while driving eastbound on the Santa Monica Freeway this past Saturday. Do you think the license plate is referring to the Ubuntu Linux OS or just the Ubuntu philosophy?

It also had some funny stickers and decals:
* Evolve FISH Logo Decorative Silver Car Emblem
* Goodnight Bush: A Parody
* Got Hope Bumper Sticker

Products I Can't Live Without - 2009

2009-01-14T21:50:00.000-08:00

Inspired by this post from Michael Arrington of TechCrunch, I thought it would be interesting to make my own annual list of products and services that I can't live without. Here we go for 2009 (in the order that they came to mind):

Gmail, Google Calendar, Remember The Milk, iPhone, F-Spot, Quicken, Google Reader, Facebook, Google Sites, Flickr, iTunes, Twitter, Jungle Disk, Dropbox, Delicious

Gmail
I use this every minute every day for both personal and work email (my company uses Google Apps for email and calendar). I just love the user interface and ease of accessing it on the web and the iPhone. Beats the hell out of Outlook for work email.

Google Calendar
Again, this is how I keep track of temporal events in my life both at home and at work. I love that I can get text messages with alerts for calendar events.

Remember The Milk
Ever since reading David Allan's Getting Things Done, I was in search of a tool to put his ideas to work. About a year ago, I settled with Remember The Milk (RTM) and have used it ever since for task management. The user interface is outstanding and I can access it from the web and iPhone. Perhaps the best feature is that it integrates perfectly with Gmail as a "Gadget".

iPhone
I don't identify with being an Apple person, but I couldn't resist the lure of the iPhone. It really adds a lot of value to my life as I can manage email, read blogs, listen to Podcasts, use Google Maps, locate GeoCaches, and even SSH to a server at work. Oh, and visual voice mail rocks!

F-Spot
This GNOME desktop application is what I use to manage photos. It stores photo metadata in a SQLite database which I can easily access if I ever want to export the data somewhere else. It also makes it easy to upload photos to Flickr.

Quicken
I have been using Quicken for years to manage my personal finances. I upgrade it to the latest version every even year, so right now I have Quicken 2008. Honestly, I would love to stop using Quicken and switch to a web-based solution, but I can't find one that is fully-featured enough for me. I just have to have the ability to create arbitrary asset accounts for my wallet (cash), reimbursements, etc. Most web-based services don't offer this yet. I've got my eye on Mint which, I have a feeling, will get good enough for me to make the switch one day.

Google Reader
I read a lot of blogs. I couldn't possibly keep up if I didn't use a blog reader. In late 2008, I switched from Bloglines to Google Reader because the user interface for Google Reader just kept getting better and eventually it won me over. A big part of this was the excellent user interface of Google Reader on the iPhone.

Facebook
This barely made it to my list. Maybe I could live without it, but I do find myself checking it almost daily because so many of my friends use it and somehow it makes me feel like I'm in touch with all of them without ever really talking to them :). Also, Facebook is the platform I used to develop and launch my book sharing application We Read which continues to grow slowly, but steadily each day and recently passed 1000 users.

Google Sites
This past year I finally committed to using a wiki for my family. I have been using wikis for years for work and open source software projects, and always thought it would be useful for keeping track of personal things like lists of gifts to get people, account information for utilities, financial services, and contractors, etc, etc. I tried setting up and hosting wikis like XWiki and Confluence, but ultimately went with the free, online Google Sites which was formerly JotSpot.

Flickr
I purchased the Pro Account and now use Flickr for all my online photo sharing needs. The pro account allows me to upload an unlimited amount of photos for about $25/year. My family depends on this to see the many photos I take of my son as he grows up.

iTunes
I hate the fact that iTunes doesn't run on Linux machines, but I can't resist it because it is really the only good option for managing music and videos on an iPod. I use it daily to load up my iPhone with podcasts.

Twitter
I use Twitter a lot. I blame Twitter for killing my urge to blog. It is just so much easier to share things 140 characters at a time. I have Twitter linked with Facebook so that each Tweet I send out ends up updating my Facebook status.

Jungle Disk
This might be one of the most important services I use. It keeps my music, videos, photos, and other important files backed up on Amazon's S3 network. At only $.15/GB, it ends up being pretty cheap to store a lot of data. This way, if my house ever burns down or someone steals my computers, or I experience a hard disk failure, I will always be able to restore my precious data.

Dropbox
I have started storing all files that I access often in a special directory on each computer (home, work, laptop, etc) called the Dropbox. Each directory gets automatically synchronized so I have easy, local, access to the files I need wherever I am. It also makes it extremely easy to share files with other people that use Dropbox.

Delicious
I store all my bookmarks in Delicious and take advantage of its Firefox plugin to easily tag sites and recall sites I've already tagged. I don't do a lot with the social features of Delicious like bookmark sharing, though. Maybe it's just because I don't know which of my friends use Delicious too.

Well, there you have it! I can't wait to see how this list compares to the one I'll make in January 2010.

Groovy Best Fit Line

2008-12-03T20:32:00.000-08:00

An engineer on my team at work asked if I could help with coding an algorithm for calculating a line that best fits a given set of data points. I thought, "what a great excuse to practice some Groovy", and got started right away.

A few Google searches led me to this 10-yr-old page on Regression Functions by Stefan Waner from Hofstra University. Stefan outlined the exact algorithm I was looking for.

Here is my implementation in Groovy:

// Prints the start/end points of a line that best fits 4 sample data points
pts = [[1, 1.5], [2, 1.6], [3, 2.1], [4, 3.0]]
println bestFit(pts)

/**
* Given a set of points, uses a linear regression algorithm to find the start and end points
* of a line that best fits the set of points.
* Returns the two points as [[x1, y1], [x2, y2]].
* The algorithm is from
* http://people.hofstra.edu/stefan_waner/realworld/calctopic1/regression.html.
*/
def bestFit(pts) {
  // Find sums of x, y, xy, x^2
  n = pts.size()
  xSum = pts.collect() {p -> p[0]}.sum()
  ySum = pts.collect() {p -> p[1]}.sum()
  xySum = pts.collect() {p -> p[0]*p[1]}.sum()
  xSqSum = pts.collect() {p -> p[0]*p[0]}.sum()

  // Find m and b such that y = mx + b
  // m is the slope of the line and b is the y-intercept
  m = (n*xySum - xSum*ySum) / (n*xSqSum - xSum*xSum)
  b = (ySum - m*xSum) / n

  // Find start and end points based on the left-most and right-most points
  x1 = pts.collect() {p -> p[0]}.min()
  y1 = m*x1 + b
  x2 = pts.collect() {p -> p[0]}.max()
  y2 = m*x2 + b

  [[x1, y1], [x2, y2]]
}

Running this script prints the following:

[[1, 1.3], [4, 2.8]]

You gotta love Groovy!

A Personal Hosted Confluence Wiki for Free

2008-11-22T22:26:00.000-08:00

I had been searching for a way to use Confluence as a personal wiki for me and my wife. I couldn't find a cheap hosted solution where I could just get an account for 2 people on an instance of Confluence maintained by someone else. I also didn't really want to pay a hosting company like Linode or Slicehost $20 a month for my own virtual machine on which I'd have to install and maintain Confluence myself.

Recently I heard about Morph Labs which describes themselves as a Platform as a Service (PaaS). They offer free hosting accounts that give you up to a 1GB database. I immediately got the idea to try running the personal edition of Confluence on this service. Eventually I got it working, but the process to get up and running was not at all easy. In fact, I struggled for serveral nights.

The rest of this post documents the process I went through. I hope it will help someone else avoid the many hours I spent learning how to work within Morph AppSpace with a third-party Java web application. Good luck, and if you have any tips or corrections, please comment and I'll update this post.

Start by signing up for a Morph AppSpace account with Morph Labs. Create a new subscription and setup a new database for that subscription. Be sure to choose Java, not the default Ruby on Rails, when setting up your subscription. Choose PostgreSQL for your database as this seems to work best with Confluence. Once your subscription is setup, download the Properties File, morph_deploy.properties, and Deployment Jar File, morph-deploy.jar, that can be found in the Java Tools for Morph AppSpace Deployment section.

Next, go get your free license for personal Confluence. It allows you to register 2 users. Hold on to that license text. You'll need to activate Confluence once it is installed. Then download the latest version of Confluence which is currently 2.9.2.

Unpack the file you downloaded somewhere on your computer. Inside, you should find an edit-webapp directory. Any files you need to modify should get copied into this directory. The contents of this directory get overlayed on top of the contents of the eventual Confluence WAR file that you're going to build, so make sure to adhere to the standard WAR directory structure. The whole purpose of the edit-webapp directory is to keep your local changes separate from the distribution so when it comes time to upgrade Confluence, you can just replace it and you'll know what you had to modify.

Copy confluence/WEB-INF/classes/confluence-init.properties to edit-webapp/WEB-INF/classes/confluence-init.properties.
Edit the copied confluence-init.properties, specifying the home dir as /var/java/ where APP_NAME is whatever name you chose as your subdomain at Morph. For example, if your domain is family-wiki.morphexchange.com, then your home directory path would be /var/java/family-wiki.

Copy confluence/WEB-INF/web.xml into edit-webapp/WEB-INF/web.xml.
Edit web.xml adding the jdbc datasource:


<resource-ref>
  <description>Morphlabs Datasource</description>
  <!-- any name will do for the res-ref-name -->
  <res-ref-name>jdbc/morph-ds</res-ref-name>
  <res-type>javax.sql.DataSource</res-type>
  <res-auth>Container</res-auth>
</resource-ref>

and the JavaMail session:


<resource-ref>
  <description>Morphlabs Mail Session</description>
  <!-- any name will do for the res-ref-name -->
  <res-ref-name>mail/Session</res-ref-name>
  <res-type>javax.mail.Session</res-type>
  <res-auth>Container</res-auth>
</resource-ref>

I had to disable Confluence's JMX-based monitoring because of problems registering MBeans. This isn't a big deal since, for a personal instance of Confluence, I don't really have any need to monitor performance and gather statistics. This web page explained how to disable JMX. Download jmanageContext.xml from the URL shown on that page. Then, copy jmanageContext.xml into edit-webapp/WEB-INF/classes.

Remove mail.jar and activation.jar from confluence/WEB-INF/lib. Morph AppSpace already includes those in the Jetty server and having an extra copy in your WAR might cause class loader issues that would prevent JavaMail from working properly. This is the only change that you'd have to remember to do again if you were to ever download a new version of Confluence.

Now it is time to build the WAR file. Run ./build.sh which creates dist/confluence-2.9.2.war. If you need to make any changes to any files, be sure to run this build script again.

Download morph_deploy.properties and morph-deploy.jar from your subscription within the Morph Control Panel.

Run


java -Xmx512m -jar morph-deploy.jar --config morph_deploy.properties confluence-2.9.2/dist/confluence-2.9.2.war



Now, sit back, relax, and wait while your WAR is being uploaded and deployed on Morph AppSpace.  This could take up to 15 minutes, so be patient.

Hit your-subscription-name.morphexchange.com and if all went well, you should see a Confluence installation web page.  Following the Confluence installation wizard, entering your personal license text.  Choose Custom Installation, External Database, Connect via a Datasource.
Enter java:comp/env/jdbc/morph-ds as the datasource.

Confluence takes a long time to setup the database, so the page may appear to hang.  Each time I tried this (I had to try many times before figuring all this out), the page timed out.  I don't know if something went wrong or Morph AppSpace just has a time limit on waiting for a page to render.  Even though the page timed out and I saw some kind of "Proxy error" message, the database still seems to have been initialized properly, but the application wasn't able to recover.  I had to go into the Morph Control Panel to restart the application.  Once I did that, Confluence was up and running!

Coming soon to this post: Setting up a mail server and using Groovy and XML-RPC to perform a daily backup of your confluence data.



MacBook to HDTV via Mini DVI to DVI to HDMI
2008-09-28T22:17:00.001-07:00
Since the MacBook is so good at playing different kinds of media, I plan to make it the media center of my life.  I will store all music, videos, and photos there.

This led me to try to connect my MacBook to my Sharp Aquos LC46D64U 46-Inch 1080p LCD HDTV.  I did some research and found that there was only one option if I wanted to have digital picture quality.  I needed 2 cables because there is no such thing yet as a Mini DVI to HDMI cable:
Mini DVI to DVI Adapter ($19.95)
DVI to HDMI Cable ($19.95)
I picked these up, connected the Mini DVI end of the first cable to my MacBook, connected the DVI end of the second cable to the first cable, and finally the HDMI end of the second cable to an open HDMI slot in my TV.  Then I switched the TV's input to the HDMI port I just plugged into, and voila! - it worked immediately! The 1920 by 1080 picture looked great.

The MacBook lets you choose between having 2 monitors and mirroring the same monitor.  The settings are all under System Preferences > Displays.

Now, I can show photo slide shows when the family is over, I can watch video podcasts and YouTube videos on the big screen, and I can even enjoy the Twitter election feed from my couch.

Next I will look for the best way to get the sound from my MacBook hooked up to my Denon receiver.


Giving In and Buying a MacBook
2008-09-23T21:37:00.000-07:00
I finally caved in to the marketing pressure from Apple and a lot of my Apple-religious friends.  I bought a MacBook.  Why would I do this after many years of trying to do everything with open source software?  The main reasons are iTunes and my wife's need to use MS Office.

I need iTunes to manage my family's 3 iPods (2 of which I got for free at tech conferences and users groups).  I also will need iTunes if I decide to buy an iPhone.  I have been thinking about it because I am surrounded by friends and co-workers who swear up and down that the iPhone is the answer to all of life's problems. Anyway, Apple won't produce a version of iTunes that runs on Linux and that's a shame, but there's nothing I can do about it.

My wife has been using our Ubuntu linux desktop for the last few years, and has gotten along quite nicely until recently when she needed some advanced spreadsheet functionality like mail merge. I am disappointed to say that Open Office just wasn't cutting it.  The feature set was either lacking or buggy.  She was longing for MS Excel.  MS Office is available for the Mac, so that problem could get solved too.

So far, the best thing about that Mac has been its packaging.  It came in a slick box with fancy designer Styrofoam.  It has stylish body and the l.e.d. lights have a pleasing glow.  Also the screen has a great, kind of glossy, picture.

I do have some gripes:

For one thing, I can't find a way to turn of that annoying sound the computer makes when booting.  A few blog posts I read say that you have to download some 3rd party software to disable it.

After 20 minutes of surfing the web when I first booted up, the computer crashed and the wireless card stopped working.  I had to take the computer back to the Apple store.  Thankfully they exchanged it even though I had purchased the MacBook from the online Apple store.

There doesn't appear to be any way to maximize a window.  There is a green plus button on each window that makes the window larger, but it doesn't make it go full screen.  Why can't I have this functionality on a Mac?

There is no concept of right-click!  To get a context menu, I have to put 2 fingers on the trackpad and click the button.  I guess I can get used to this, but come-on, would it be so hard for Apple to make the right side of the button do what Windows and Linux users expect?

You can only resize a window from the lower-right corner.  This is unbelieveable.  Both Windows and Linux allow you to resize a window from any edge of the window.  I find this really annoying.

There are no Page Up, Page Down, Home, and End keys!   The function key + up or down arrows will do the same thing as Page Up and Page Down.  Also the Apple key + the left and right arrow will do the same thing as Home and End, but I'm just not used to it yet.

Pressing the red X button on a window closes the window, but doesn't close the application even if it is the last window open in that application.  You have to remember to explicitly quit the application with an Apple key + Q.  I always forget to do this and am left with many open applications as a result.

Despite all these gripes, I am happy to be getting to know my way around a Mac now.  I have always felt completely lost on a Mac and these days, it feels like everyone and their mother has one.


Walking L.A. #20 - Lower Beachwood Canyon
2008-07-13T21:01:00.000-07:00
Again, Erin Mahoney, the author of our guide book, took us back to Hollywood.  This time we visited Lower Beachwood Canyon which is just a little bit east of the last walk in Whitley Heights.  Again there were many steep roads, but I enjoyed this walk a little more than last week's because we went earlier in the morning to avoid the heat, and I saw an area I've never seen before, even though I've lived in LA for over 15 years.

We began at the bottom of Vista del Mar Ave where it meets Franklin Ave.  We stopped for coffee at the 101 Coffee Shop which was a 70's-looking diner with some character.  I'd return there sometime to try the food, but never again for a cup of coffee which was pretty lousy even for a diner.

Again this week we saw beautiful houses, many with enviable views of Hollywood and Downtown LA.  Many parts of the neighborhood were quiet with narrow, curvy streets, which reminded me of hillside towns in Europe.  Other parts were extremely noisy from the roaring cars on the adjacent 101 Freeway.

One of the noisiest areas, ironically, was home to the Vedanta Society of Southern California. We stopped in to admire their white temple which looked like a mini Taj Mahal.  We also browsed through their bookstore where we almost bought our son a Peaceful Piggy Meditation book.




The next walk is Upper Beachwood Canyon.

Walking L.A.: 36 Walking Tours Exploring Stairways, Streets and Buildings You Never Knew Existed




Walking L.A. #19 - Whitley Heights and Hollywood Boulevard
2008-07-06T20:31:00.000-07:00
This walk was just on the other side of Highland Ave from the last walk.  You can tell that Whitley Heights was once a very prestigious neighborhood by the many fancy Mediterranean style homes.  Unfortunately the 101 Freeway was built right through it and now the residents have to put up with the noise.

It was really hot today and the many steep roads on this walk contributed to a lot of sweating.  After walking through Whitley Heights we descended down Whitley Ave to Hollywood Blvd where we walked west to the Hollywood and Highland complex.  The boulevard was full of the usual tourists.  We had been here many times before so instead of sightseeing, we headed into a Build-a-Bear store to cool off.  Yes, there is actually a store that is like a Subway Restaurant for teddy bears.  You start with a naked bear and then pick its name, clothes, and accessories.

After the bear store, we stopped to watch some dancers rehearsing their moves at the Hollywood Pop Academy while we fed our baby.  Finally, we grabbed some coffee at Starbucks and then headed back to Whitley Heights to end our walk.

The next walk is Lower Beachwood Canyon.

Walking L.A.: 36 Walking Tours Exploring Stairways, Streets and Buildings You Never Knew Existed




Walking L.A. #18 - High Tower and the Hollywood Bowl
2008-06-08T20:46:00.000-07:00
Before the walk, we stopped for breakfast at the Mar Vista Farmers Market getting omelettes and French toast from Cafe Laurent.  Two good friends who were visiting from San Francisco joined us today.

The first part of the walk was interesting.   We walked through the Hollywood Heights neighborhood which is just south of the Hollywood Bowl.  At the center of this hillside neighborhood stood a tall elevator tower known as High Tower. The tower is closed to everyone except residents with a key, so we had to climb many cement staircases to get to the top.  This was a challenge considering we had our baby and stroller with us.  Luckily my friends took turns helping me out.   One of the things I liked the most about this neighborhood was that many of the homes were accessible only by footpath.  This gave it a quiet, neighborly feel similar to places I've seen in Europe.

The next part of the walk took us to the famous Hollywood Bowl.  There were no shows going on and the bowl was open for people to just walk in and check it out.  We walked to the boxed seat area, took a seat in one of the boxes, and dreamt about how nice it would be to attend a concert or show in one of the boxes eating cheese and sipping wine.

The next walk is Whitley Heights and Hollywood Boulevard.

Walking L.A.: 36 Walking Tours Exploring Stairways, Streets and Buildings You Never Knew Existed




My Twitter Account Has Disappeared!
2008-06-02T23:38:00.000-07:00
I was about to Twitter just now and for some reason Twitter wouldn't let me login.  I tried visiting my Twitter feed page at http://twitter.com/kweiner and Twitter responded with the message "That page doesn't exist!".  Figuring that Twitter was just down, I tried a friend's page, and that page loaded just fine.  In the "Following" section of my friend's page, I noticed that my picture was missing!  Next I tried to reset my password, and when I entered my email address, Twitter responded with "Oh, snap! We couldn't find you!".

What is going on?  Is my Twitter account really gone and my data lost forever?  I know they have had problems scaling recently, but losing someone's account is way worse than having a temporary outage.   I submitted a support ticket and also posted a message to Get Satisfaction.  Let's see what happens...  If this doesn't get resolved, I am just going to give up on Twitter which will probably make my wife happy.


Walking L.A. #17 - Carthay Circle and South Carthay
2008-06-01T20:31:00.000-07:00
There wasn't anything that exciting about today's walk which was a little south of last week's walk in the Miracle Mile area.  We explored 2 of 3 neighborhoods within a residential district called Carthay: Carthay Circle and South Carthay.  Our book says that these neighborhoods are in the Miracle Mile district, but Wikipedia says Carthay is its own district.  Both neighborhoods contain HPOZ's for their architectural integrity and cohesiveness.  The styles include Spanish Colonial Revival and Tudor Revival.  I can't think of much else to say about this walk.

The next walk is High Tower and the Hollywood Bowl.

Walking L.A.: 36 Walking Tours Exploring Stairways, Streets and Buildings You Never Knew Existed




Walking L.A. #16 - Miracle Mile
2008-05-25T19:55:00.000-07:00
Our friend Kevin joined us for this walk which meandered through the Miracle Mile district where he works at E! Networks.

The walk began at Hancock Park (a real park, not the wealthy LA neighborhood of the same name).  This park is home to the infamous La Brea Tar Pits and the adjacent LA Country Museum of Art.  We paused briefly by the tar pits to see if we could get the smell to trigger our memory of childhood playground asphalt on a hot day, something our book mentioned we might experience.  It didn't work at first, but as we walked away from the tar pits, a gentle breeze indeed carried the scent we were waiting for.  We skipped the Page Museum which is dedicated the ancient history of the area since we'd all been there at some point in our lives.

Next we passed by a new part of LACMA called BCAM which stands for Broad Contemporary Art Museum.  In front of BCAM was a giant red toy firetruck and closer to Wilshire was a display of lamp posts.  We paused here to take some pictures.

We stopped for lunch at Farmers Market on 3rd and Fairfax.  I had fried catfish and jambalaya at one of my favorite eateries, The Gumbo Pot.  My friend Kevin grabbed a savory crepe from the French Crepe Company.  After lunch we walked quickly through The Grove, one of those generic, outdoor, fake-urban, Disneylandesque shopping malls that everyone but me seems to love.

We left The Grove to walk through a nice neighborhood called Miracle Mile North, one of the district's several HPOZ's.  Very nice homes in a very central location.  Wish we could afford to live there.  We finished up the walk passing through the Park La Brea high-rises.  It was the first time I had walked through Park La Brea, and I wasn't impressed, although it seems that many people like it.

The next walk is Carthay Circle and South Carthay.

Walking L.A.: 36 Walking Tours Exploring Stairways, Streets and Buildings You Never Knew Existed




Cloud Studio Released
2008-05-17T19:32:00.000-07:00
Cloud Services just launched Cloud Studio which is a java-based desktop application for managing images and instances in Amazon's EC2.  I've yet to try it, but from the screenshots, it looks really nice.  I'm not sure, though, if it does anything more than the very capable Elasticfox which runs right in your Firefox browser.  Cloud Studio is also available as an Eclipse plugin which is really convenient for Java developers and users of the popular open source IDE.

GUI's like Elasticfox and Cloud Studio make working with Amazon EC2 really easy.  Now I wish someone would create similar management tools for Amazon's SQS and SimpleDB.  I'd like to be able to list, see stats for, and manage my SQS queues and SimpleDB domains, items, and attributes without having to run command line scripts.  Hint, hint, Cloud Services!


ejabberd on Amazon EC2 Ubuntu AMI
2008-05-11T10:21:00.000-07:00
It turned out to be harder that I expected to setup a Jabber (XMPP) server on an Ubuntu virtual machine (ami-ce44a1a7) within Amazon EC2.  I chose to setup ejabberd since it was an easy to install via apt-get.  I was a little nervous about working with a server built  with Erlang since I knew nothing about it, but it was a reputable server and I was counting on not needing to have any Erlang knowledge to work with ejabberd.  That was a correct assumption for the most part.

First, I logged into the VM and installed ejabberd:
apt-get install ejabberd
Next I edited the ejabberd config file, /etc/ejabberd/ejabberd.cfg, making kweiner@jabber.pop140.com an admin user and setting the hostname as jabber.pop140.com:
%% Admin user
{acl, admin, {user, "kweiner", "jabber.pop140.com"}}.

%% Hostname
{hosts, ["jabber.pop140.com"]}.
Next, I restarted the server and registered kweiner as a new user:
/etc/init.d/ejabberd restart
ejabberdctl register kweiner jabber.pop140.com mypasswd
Then I authorized traffic on ports 5222 (Jabber), 5223 (Jabber encrypted for old clients), 5269 (Other Jabber servers), and 5280 (Jabber web admin tool):
ec2-authorize default -p 5222
ec2-authorize default -p 5223
ec2-authorize default -p 5269
ec2-authorize default -p 5280
I could now hit the admin screen on which I logged in as kweiner@jabber.pop140.com:
http://jabber.pop140.com:5280/admin/
From here it was possible to add new users and browse server statistics. I experimented with adding users and using various Jabber clients like Gajim to send messages from one user to another.

My first problem came when I tried to communicate with users registered in other Jabber servers like Jabber.org and Google Talk.  I struggled for hours trying to figure out why users on my server couldn't communicate with users from these other servers.  Thankfully James Murty gave me a bit of help on this Jabber on EC2 message board thread.  It turned out that I needed to configure SRV records in my DNS settings.

I logged into GoDaddy where my domain is registered and configured SRV records as follows:


After this, I was able to use nslookup to verify that the SRV records were setup properly:
kweiner~$ nslookup
> set type=srv
> _xmpp-server._tcp.pop140.com
Server:         66.75.160.63
Address:        66.75.160.63#53

Non-authoritative answer:
_xmpp-server._tcp.pop140.com    service = 10 10 5269 jabber.pop140.com.
That did it!  My jabber server was finally federating with other jabber servers and my users could talk to their users.

I encountered my next big problem when I tried to use jabber again after terminating and relaunching my AMI. ejabberd failed to start and I found the following error message in the /var/log/ejabberd/ejabberd.log:
application: ejabberd
exited: {bad_return,{{ejabberd_app,start,[normal,[]]},
           {'EXIT',{{badmatch,{aborted,{no_exists,config}}},
                    [{ejabberd_config,set_opts,1},
                     {ejabberd_app,start,2},
                     {application_master,start_it_old,4}]}}}}

After some googling, I found that ejabberd associates itself with an Erlang node name, a concept I don't really understand that well.  By default the node name was dynamically set based on the hostname for the machine.  It looked something like this: ejabberd@domU-12-31-38-00-9D-63.  This node name is somehow linked to the Mnesia database stored as files within /var/lib/ejabberd.  The problem is that the hostname and therefore the node name changes everytime the AMI is relaunched which confuses ejabberd.

One solution I found is to explicitly set the node name.  I did this by modifying /etc/default/ejabberd adding the line:
export ERLANG_NODE=ejabberd@jabber
This requires adding jabber as a host name inside /etc/hosts:
127.0.0.1 localhost.localdomain localhost jabber
I made those changes, removed all the database files from /var/lib/ejabberd, and restarted ejabberd. That did it! The node name was the same regardless of the hostname associated with the particular AMI instance.

This was a lot of effort, but it probably would have been easier if I had been familiar with Erlang applications and SRV DNS settings.  I hope this post helps someone else struggling to setup ejabberd on EC2 as I did.


Amazon EC2 - My First Step into the Cloud
2008-05-06T21:42:00.000-07:00
Ever since returning from the Web 2.0 Expo in San Francisco last month, I have been excited about learning how to setup a machine in the Amazon Elastic Compute Cloud (EC2).  My first goal was just to get a Linux machine running in the cloud on which later I will try to install enough software to get a basic web application running.

To learn what to do, I consulted a the Programming Amazon Web Services book which is fortunately available on Safari Online.  In parallel I read the Getting Started Guide on Amazon's website.  The steps to get a virtual machine running were roughly the following:
Register for Amazon EC2.
Download a X.509 certificate private key and public key pair and store them in a ~/.ec2 directory on my computer.
Take note of my AWS account number.
Download the Amazon EC2 Command-Line Tools
Set environment variables: EC2_HOME, EC2_PRIVATE_KEY, EC2_CERT
Run ec2-describe-images -o self -o amazon to search for images.
Generate keypair using  ec2-add-keypair gsg-keypair
Fired up the Getting Started AMI with  ec2-run-instances ami-2bb65342 -k gsg-keypair
Open SSH and HTTP ports: ec2-authorize default -p 22, ec2-authorize default -p 80
SSH into the instance: ssh -i id_rsa_gsg_keypair root@ec2-75-101-209-13.compute-1.amazonaws.com
Access the instance's web server: http://ec2-75-101-209-13.compute-1.amazonaws.com/
Shutdown the instance: ec2-terminate-instances i-db6ea2b2
That's it! Everything worked as advertised and I got through all of that in less than an hour.  This whole procedure cost me about 10 cents.  I am looking forward to my next goal which is to find an Ubuntu server image, install a database with data, and learn how to preserve the data so that it survives an image restart.

  


Server-Side Language Detection with Ruby + Google Language API
2008-04-20T22:07:00.001-07:00
Have you ever wanted to detect the language of a piece of text?  Google's AJAX Language API makes this possible on the client side.  This Belgian startup's blog post shows a PHP example of how you can use Google's detection service on the server side.  Here is a port of that example in Ruby:

(the 'json' gem must be installed prior to running this program)

require 'rubygems'
require 'net/http'
require 'open-uri'
require 'cgi'
require 'json'

base_url = 'http://www.google.com/uds/GlangDetect?v=1.0&q='
url = base_url + CGI.escape("See if you can guess what language this is!")
response = Net::HTTP.get_response(URI.parse(url))
result = JSON.parse(response.body)
lang = result['responseData']['language']
puts "Language code: #{lang}"



Reverse Tinyurl with Ruby
2008-04-14T22:08:00.000-07:00
I was searching for a way to reverse TinyURL's using Ruby.  First, I went to TinyURL.com to see if there was actually an API to do this.  Unfortunately I did not find one.  I then searched for Ruby APIs to do it and came upon these two libraries for dealing with TinyURL's:
Tinyurl - This one converts regular URL's to TinyURL's and vice-versa.  However, I examined the source code and discovered that to do this, the author's personal web server, logankoester.com, is called.  This concerned me because: What if this service goes down?  My program would stop working.

ShortURL - This one converts original URL's into many "short URL" formats such as TinyURL, RubyURL, and mooURL.  Unfortunately there is no way to reverse any of these URL's.  Since I needed the reverse functionality, I had to pass this one by.
With a little more googling, I found the PHP code that the Tinyurl author used to do the TinyURL conversions.  I decided to port his screen-scraping approach to Ruby for use in my program:

require 'rubygems'
require 'net/http'

def reverse_tinyurl(url)
  url_parts = url.split('.com/')
    preview_url = "http://preview.tinyurl.com/#{url_parts[1]}"
    response = Net::HTTP.get_response(URI.parse(preview_url))
    original_url = response.body.scan(/redirecturl" href="(.*)">/)[0][0]
end

puts reverse_tinyurl('http://tinyurl.com/6nzb8u')

Running this program would print out the URL to my blog, http://kweiner.blogspot.com.

By the way, I also found TinyURL Callback API in JavaScript, but I wanted a server-side solution.  Additionally, I found the TinyURLReverser from Yahoo Pipes, but I wasn't clear on how this worked.  I need to examine this one a little further.


Getting Twitter4R to Work on Ubuntu 7.10
2008-04-06T21:57:00.000-07:00
Tonight I set out to try the Twitter4R ruby library so I could play around with Twitter data.  I am not a ruby expert by any means, but I thought it would be easy to setup.  Unfortunately, I ran into a number of problems:

I first tried to upgrade my ruby gems installation to 1.1.0.  I downloaded and unpacked rubygems-1.1.0 and ran 'ruby setup.rb'.  After this I got the following error when trying to run gem -v: uninitialized constant Gem::GemRunner (NameError). I googled this and ended up on this blog post which solved my problem.  I had to edit the /usr/bin/ruby file and add the following line: require 'rubygems/gem_runner'.

With gem working, I installed Twitter4R: sudo gem install twitter4r.  This worked smoothly, so I went on to write a sample Twitter4R ruby program:

require('rubygems')
gem('twitter4r', '0.3.0')
require('twitter')

Twitter::Client.configure do |conf|
conf.source = 'kweiner'
end

client = Twitter::Client.new(:login => 'kweiner', :password => 'mypass')
timeline = client.timeline_for(:me)
puts timeline


Running this gave me the error No Such File to Load: net/https. Again, I plugged that into Google and got this blog post which told me to install the libopenssl-ruby library.  I did this easily with sudo apt-get install libopensll-ruby.

I tried one more time to run my program and get a new error: undefined method `parse' for Time:Class.  Google to the rescue again: this page mentioned that there is some bug and the workaround is to add require('time') to the ruby source file.  I did this, and, voila!, my program printed the latest tweets from my timeline.


Walking L.A. #15 - West Hollywood
2008-04-06T19:23:00.000-07:00
This week's walk was about 2 miles - much longer than some of the previous ones.  We were joined by our 2 friends and their 2 kids who rode along in their stylish new double-decker stroller.

We started the walk on King's Rd. just south of Willoughby St. at The Schindler House which is considered to be the first modern house built in the world.

We then walked south to Melrose Ave where we stopped for coffee and pastries at Sweet Lady Jane who, according to my wife, is supposed to provide Nicole Richie a wedding cake.  I don't see evidence of that on the web, but I do see that the store is now owned by the Olsen twins and makes cakes for other celebrities.  I don't know if I'd ever go back there.  The two guys behind the counter weren't that friendly and the coffee was average.

We continued on Melrose on the way to the Pacific Design Center stopping only at Dr. Tea's Tea Garden where we looked around and decided that this might be a nice place to come back to one day, maybe with our parents for a nice afternoon tea break.

At the Pacific Design Center, we stopped to admire the giant metal sculpture of a chair and let the kids wander around the waves of grass.

Next we turned the corner onto Santa Monica Blvd. in the heart of gay West Hollywood.  The street was pleasant to stroll on with its wide sidewalks and shady trees.  We finshed at Barney's Beanery for lunch.  I felt a little strange there with 2 strollers, 2 infants, and a toddler.  Luckily there was a spot for us in the back corner where we could make a little extra noise and breastfeed babies without really disturbing anyone.

The next walk is Miracle Mile.

Walking L.A.: 36 Walking Tours Exploring Stairways, Streets and Buildings You Never Knew Existed




Scrum from the Trenches
2008-04-02T20:58:00.000-07:00
Vaibhav, a friend and colleague at work, decided to start blogging about Scrum.  I suggested the title Scrum from the Trenches, and he went with it:
http://scrumfromthetrenches.blogspot.com/.

He and I advocated the use of Scrum a year ago as my team was tasked with an aggressive project in which we had the opportunity to work closely with the product manager.  We implemented the main pieces of Scrum like daily meetings, planning sessions, and sprint-end demos, but also left out a few things like story estimation and the plotting of burn-down charts.  We knew we had a ways to go if we were going to claim we were doing Scrum, but it was a good start and the project management group began to buy into it as a viable practice for future development.

This year, the team was tasked with a similar project.  To prepare for it, 2 project managers, the QA manager, and Vaibhav attended Scrum training up in the Silicon Valley.  Through lecture and hands-on exercises, they learned the ins and outs of textbook Scrum and returned to work eager to try it out.  The team is now sub-dividing stories into tasks and even playing Planning Poker.  How is is going?  That's the subject of Vaibhav's new blog.  Read it and find out.


Jungle Disk on Linux
2008-03-31T19:43:00.000-07:00
I first heard about Jungle Disk on the Linux Action Show.  It is a client that manages files on Amazon's Simple Storage Service (S3).  The client is cross-platform (Linux, Windows, Mac OS) and costs only a one-time fee of $20 plus $1 a month if you want the optional "extra" services including a web interface to your files.

With Jungle Disk, you pay as you go for file storage: 15 cents/gig/month for storage and a similar low fee for data transfer. You can use S3 as a virtual drive or "briefcase" with unlimited capacity, or you can use it as a backup location for automated or manual backups from various computers.  I use it for both.

So far I've setup my Ubuntu desktop and Windows laptop from work as well as my Ubuntu desktop at home.  I have backups configured for important files from each of those.  From my eee PC, I passed on installing Jungle Disk and instead relied on the web interface that came with the Jungle Disk extras.  I don't store any files on my eee, so I don't need a client to back anything up.

So far I'm very pleased with Jungle Disk.  It will get pricey as I start to backup everything including the 50+ gigs of photos, videos, and music files I have.  50 gigs would be $90 a year.  However, this is well worth it if I ever need to recover my precious files in the event of a fire or multi-hard drive crash.  Photos and videos especially are irreplaceable.

I took some notes on setting up Jungle Disk on Ubuntu:

* Sign up for Amazon S3.

* Download and unzip Jungle Disk (jungledisk.tar.gz) in /opt.

* Run Jungle Disk Monitor:
./junglediskmonitor

* Install FUSE and davfs:
    apt-get install fuse-utils
   apt-get install davfs2

* Add your user to the fuse group:
    sudo adduser kweiner fuse

Create and mount a directory to access Jungle Disk on the file system:
    sudo mkdir /mnt/jungledisk     sudo chgrp fuse /mnt/jungledisk     sudo chmod g+w /mnt/jungledisk     sudo mount.davfs http://localhost:2667 /mnt/jungledisk -o nolocks


Walking L.A. #14 - Sunset Strip
2008-03-30T15:42:00.000-07:00
My in-laws joined us for this 1 mile walk along Sunset Blvd and the adjacent residential neighborhood in West Hollywood.  I have been to this part of the strip between  Havenhurst Dr. and Olive Dr. many times.  I've had drinks at Bar Marmont and a company holiday party at Sunset Beach.  I've seen friends play jazz at the Argyle Hotel lounge and been to various concerts at the House of Blues (Buckshot LeFonque, Maceo Parker, etc.).  Therefore, the first half of the walk wasn't that interesting.

The second half which circled back on De Longpre Ave and Harper Ave introduced us to the quiet residential life just south of the strip which included apartment buildings built in the 50s and 60s and a nice neighborhood dog park.

Instead of lingering in the area for lunch, we got back in the car and headed over to the Hollywood Farmers Market.  I had been wanting to check out the presence  of Theodore Payne at this market, and today I had the chance.  It turned out to be a very nice farmers market with a lot of vendors and shoppers.

The next walk is West Hollywood.

Walking L.A.: 36 Walking Tours Exploring Stairways, Streets and Buildings You Never Knew Existed