EarthBrowser

My iTunes Pirate Album

2008-08-06T15:45:00.000-07:00

My alter-ego has been revealed...

I'm the drummer first mate on the pirate rock band HuckleScary Finn captained by my best friend Kevin Hendrickson. I'm afraid that I can't take credit for the huge resurgence in pirate themed activities here in the Portland area, but I'd like to think I got in on the ground floor.

Notable Portland Oregon Pirate links:
• Portland Pirate Festival
• Captain Bogg & Salty (best pirate band in the world).
• International talk like a pirate day
• Captain Henry's Pirate Store (a pirate merchandise only store)

Arrrrrrrrr

What's in the EarthBrowser pipeline?

2008-07-23T10:40:00.000-07:00

If you aren't interested in wading through my sleep inducing 30 minute Google presentation, I'm planning on detailing much of what I said in a series of blog posts.

Contrary to almost every bit of advice I've been given I'm going to lay most of my cards out on the table about the future of EarthBrowser. For a single guy trying to go up against multi-billion dollar corporations, I need as much attention as I can get. However I'm not going to reveal everything, I've got some pretty cool things under wraps for release later this year.

Flash 10: I'm very excited about Flash 10 and believe that it will open the door for real-time 3D in the web browser. For me the most important aspects of F10 are native matrix operations, hardware accelerated rendering and a shading language. This will enable EarthBrowser to have true 3D navigable terrain with mountains, valleys and even underwater exploration, right in your web browser or in the desktop authoring app. I am hoping to get a 50-100% increase in rendering speed and be able to do really pretty things like normal mapping and atmospheric scattering.

3D Collada models: I'm not really a big fan of the obsession by Google and MS to put accurate 3D buildings on the earth, but if you have the data it isn't really hard to do. I've got more interesting uses for 3D models which will become more apparent in future versions.

GPS track importing: A very easy feature to add for people to edit and share.

iPhone version: I still hold out hope for Apple and Adobe to get Flash running on the iPhone so I haven't allocated much of my bandwith to this project yet. However after some exploration of the problem space, I estimate it will be relatively simple to port from my current codebase due to the nicely abstracted rendering backend.

My next blog post will detail the geometry / time coordinate extension that I hope will make it into the KML standard at some point. Time coordinates are designed for use in feature animation, which is an obvious use but was never a design consideration for KML. I don't really have the time or desire to submit and shepherd a proposal through an OGC standards committee. Perhaps there could be something like the Boost Library for KML where proposed extensions are selected to become part of the standard after being tested and refined by use in the community. Seems like this would be a great place for me and the WorldWind developers to start taking a little initiative and advance the state of the art for virtual globes.

EarthBrowser presentation at Google

2008-07-21T12:50:00.000-07:00

Last Tuesday I gave a 30 minute presentation at Google about the upcoming EarthBrowser Flash plugin. I also presented an enhancement to improve the expressiveness and utility of KML along with a demonstration with orbiting satellites. Many thanks to Michael Weiss-Malik and the great guys at Google for inviting me to talk about whatever I wanted to!

If you have problems with the embedded video you can see it on youtube.

In the presentation I go through some of the features and benefits of the EarthBrowser Flash plugin. Then I talk about giving time it's own coordinate in the geometry primitives in order to reduce KML file size and complexity as well as add new functionality. Toward the end I talk about future enhancements to the EarthBrowser desktop/web platform. I also mention that I'm developing an iPhone version.

Wow! C/C++, Python and Ruby running in Flash Player

2008-07-09T08:40:00.000-07:00

Earlier this year I saw the demo of Quake running in Flash directly translated from C++. It was very impressive to say the least, Adobe has some great coders and the amazing advantage of having their code running on almost every computer with a web browser.

I just came across this post that talks about an open source backend to the LLVM that will enable C/C++, Python, Ruby, Perl, Lua and other languages to be targeted to ActionScript bytecode (the language the flash player runs). It sounds like it will need some more work and support of a new Flash player version but the possibilities are extremely exciting.

More detailed information on the initial LLVM work by Ryan Stewart here.

KML Extension, Julian Time

2008-06-19T14:30:00.000-07:00

Creating a virtual globe that can view KML files as well as creating numerous datasets that have to use KML has given me what I feel is a unique perspective on the strengths and weaknesses of the language. It also gives me a really unique opportunity to enact some features in the language that I feel are needed. Hopefully at some point the OGC and Google will consider some of my suggestions once they are more formalized.

I've just come up with a *really* nice extension to KML that can drastically reduce the size of some datasets (e.g.: a 1.2 MB dataset is reduced to 60K) and at the same time enable functionality that is sorely missing. I'm not going to spill the beans on it quite yet since I've got another couple of enhancements that will enable even more great functionality. I'll present this extension and others when I give my talk at Google on the 15th of July. I'll post a video of that talk on this blog once its available.

Julian Time:
One of the less fortunate aspects of KML, in my opinion, is it's use of the ISO 8601 Date and Time Format. I understand why they did it; because it is human readable and a standard to hang their hat on. The processing overhead on parsing the date is significant, but not a deal breaker so why do I care?

Clearly the ISO time format should be left in, but there should be another option for specifying it as Julian Time. There are so many nice properties of julian time that I would be very surprised if Google Earth, Virtual Earth, ESRI and any other virtual globes don't all use it for their internal representation of time. It is a floating point number (usually a 64 bit double) that can be manipulated with standard mathematical functions. Anyone who has ever written a date library the naive way (as I have) has come to dislike all of the little details like days in a month, leap years and worst of all... time zones. Why not standardize on a time format that does away with all of that dreck and can be operated on like a regular number? You can add 3.5 days, subtract an hour and do simple comparison operations like greater than, less than and equal. I can't imagine ever using another internal representation.

Here's a conversion routine in Python:


# gdate is (year, month [1-12], day, hour, minute, second)
def julian_date(gdate):
 if gdate[1] < 3:
  M = gdate[1] + 12
  Y = gdate[0] - 1
 else:
  M = gdate[1]
  Y = gdate[0]
 D = gdate[2] + (gdate[3] / 24.0) + (gdate[4] / 1440.0) + (gdate[5] / 86400.0)
 A = math.floor(Y/100.0)
 RV = math.floor(365.25*(Y+4716.0)) + math.floor(30.6001*(M+1.0))
 return RV + D + (2.0-A+math.floor(A/4.0)) - 1524.5

Google open sources Gears

2008-06-13T06:31:00.001-07:00

Google has open sourced it's web plugin Gears under a modified BSD license. Normally I wouldn't comment on something like this because it seems unrelated to what I'm doing with EarthBrowser. However I believe that it could be very important to the future direction of EarthBrowser. A web plugin that supports all major browsers on all major platforms is a *very* large project, one I would never attempt myself. I am kind of in shock that Google would make it so easy for others to make web plugins by giving this away under such a permissive license.

A really long time ago, way back when in the days of Netscape before Internet Explorer even came around, I had a version of EarthBrowser working as a web plugin. It had too many problems with memory limits on the old MacOS 7 and 8 so I canned the project and went with a desktop app. So now there is an open source web plugin code base that I can use to port my OpenGL version of EarthBrowser right into a website. I probably won't tackle that particular project until next year, but it is good to know that it is now possible.

I am very impressed with the release of this code, this is such a different way of doing business. I don't see any benefit for them in doing this, but there isn't any harm either. Way to share knowledge Google!

KML to JSKML translator now available

2008-06-12T12:34:00.000-07:00

It's now available at the JSKML Site run though the Google App Engine.

I've created a KML parser that will transform a KML document into the JSKML format, or alternately a JSON formatted JSKML document if that is what you require. It's a simple copy and paste translator similar to Dean Edwards excellent Javascript Packer.

It currently works on the KML 2.2 standard and could use a little optimization but it is functional and easy to use. If you give it a try and find an error, please let me know.

jskml: Javascript KML Dialect

2008-06-09T15:15:00.000-07:00

KML data structures need to be represented in other formats than just XML. With virtual globes moving into the web browser, the need for an alternate representation that does not require parsing to go from XML text into Javascript objects is an important step, in my opinion, in simplifying web based scripting of online mapping.

I have created a simple static website describing the new dialect at:
http://www.jskml.org.

I used Google App Engine and the Django framework to create this site. I've got another exciting app or two planned for deployment using this platform and this website was a good way for me to orient myself as well as put out a format that I feel should be supported.

Thoughts on the Google earth plugin API

2008-06-06T09:59:00.000-07:00

Good job Google earth/maps team.

While it feels like they rushed the introduction of the earth plugin a bit, leaving pretty spotty documentation of the API and only a Windows version, it is indeed a great first step. Putting javascript tendrils deep into the control structure of the plugin was inspired and will be transformative.

However, the programmer in me can't help but be a little offended by the size and verbosity of the interface. Am I to understand that the very first example of creating a placemark is what will be needed for such a simple task?

var placemark = ge.createPlacemark('');
placemark.setName("You are at Google");
ge.getFeatures().appendChild(placemark);

// Create style map for placemark
var normal = ge.createIcon('');
normal.setHref('http://maps.google.com/mapfiles/kml/paddle/red-circle.png');
var iconNormal = ge.createStyle('');
iconNormal.getIconStyle().setIcon(normal);
var highlight = ge.createIcon('');
highlight.setHref('http://maps.google.com/mapfiles/kml/paddle/red-circle.png');
var iconHighlight = ge.createStyle('');
iconHighlight.getIconStyle().setIcon(highlight);
var styleMap = ge.createStyleMap('');
styleMap.setNormalStyle(iconNormal);
styleMap.setHighlightStyle(iconHighlight);
placemark.setStyleSelector(styleMap);

// Create point
var la = ge.getView().copyAsLookAt(ge.ALTITUDE_RELATIVE_TO_GROUND);
var point = ge.createPoint('');
point.setLatitude(la.getLatitude());
point.setLongitude(la.getLongitude());
placemark.setGeometry(point);

Admittedly it uses the super-annoying StyleMap which I feel should be abolished. Don't you think it is too much to ask to perform this kind of operation for every placemark? I suppose someone could create a wrapper library for this to simplify common tasks.

How about implementing this in your API instead:

ge.addPlacemark({
    name:'You are at Google',
    Style:{IconStyle:{Icon:{href:'http://maps.google.com/mapfiles/kml/paddle/red-circle.png'}}},
    Point:{coordinates:{lon,lat,alt}}
});

Could the maps/earth team create the nicer interface wrapper?

EarthBrowser is Adobe site of the day, Google talk

2008-06-04T21:30:00.000-07:00

I don't have the marketing megaphone that a Google has but EarthBrowser is starting to get a little attention in the Flash world. I was just notified today that earthbrowser.com was selected as the Flash site of the day. There was also a nice article recently on RIApedia.com titled EarthBrowser - 3D AIR App in Flash.

Google, via Michael Weiss-Malik, has invited me to come down and deliver a presentation about EarthBrowser, KML and whatever else I want to talk about. I guess it will be something like the Google Tech Talks where they make a YouTube video of the presentation. I've been consolidating things on version 3 for the past month so I haven't set a date yet but I'm pretty sure it will be mid July if that is available for them. Hopefully those guys aren't too miffed about my KML criticisms, or my out of date Google Earth criticisms. Perhaps I should bring a rotten tomato shield.

I've got a little project I'm hoping to unveil at or before the talk, but I'll leave everyone in suspense about what that could be...

Cyclone Nargis and GDACS

2008-05-06T11:00:00.000-07:00

I've added a few new datasets to EarthBrowser's "Featured External Layers", which is below the main dataset element in the EarthBrowser control window.

I'm temporarily putting in a high resolution MODIS overlay of Cyclone Nargis from earthobservatory.nasa.gov. It's a really large image so be patient, it takes a little while to download. Due to a bug in Flash for large images scaled to be small, it doesn't look very good far away, but looks better as you zoom in. Hopefully they've got a fix for this in the works.

I've also added the Global Disaster Alert and Coordination System kml dataset which is updated with real-time information for ongoing disasters and recovery around the globe. To view the information for each disaster, you have to navigate through the list of elements in the control window, I wish they would put in placemarks to view the great information they have more easily.

Finally, the Europe Media Monitor layer is a news aggregation service that plots global news stories on a map and provides an overview and link to the geo-located story.

I'm also hoping to have an update out later this week with a timezone fix for sunrise/sunsets and also a global grid overlay feature.

EarthBrowser 3.0, 5 days in

2008-05-05T09:00:00.000-07:00

Version 3 is doing very well over the past 5 days. It's selling better than I'd hoped for, the django based system I am using for the main website and for the separate data server have held up incredibly well with just a few hiccups. Amazon S3 really helps me to take the load off of my servers. I've got a handle on the tech support emails over the weekend, there are still a lot left however. I try to answer each and every person with a question or problem with a personal message. I think that kind of respect and care for each customer has really helped the business over the years. My goal is to have no unhappy customers, if I can't fix your problem or you are still not satisfied, you get a refund, it's pretty simple.

Thanks for the mention James, Mickey, Lxnyce and Bull.

Some people are wondering why would they should support EarthBrowser as shareware when they can get Google Earth, Microsoft's Virtual Earth and NASA WorldWind for free. Well you get EarthBrowser for free too, you are just reminded to help support the future development of the shareware if you like it enough to use it. Google Earth is great but it isn't your computer's operating system, you CAN use more than one virtual globe at the same time.

EarthBrowser may not have 2 inch resolution datasets, but the weather layers really blow away anything the other globes offer, in fact the weather forecast layers blow away most of what I've seen from the major weather providers offer. I'm talking with CustomWeather to see if we can do some sort of exchange of data for perhaps exclusive use of EarthBrowser on their website. I think that would really make them stand out from their competitors.

KML is great! New datasets are being added to EarthBrowser all the time, if you only know where to find them. Today I noticed from one of my favorite blogs Frank Taylor's Google Earth Blog that there is a new dataset available from NASA showing the total electron count in the earth's ionosphere. I like it so much I think I'll add it to the Featured Datasets in EarthBrowser. The red areas indicate a large concentration of electrons which may cause problems with communications. It is neat to see that the electrons build up on the daytime side of the earth and disperse shortly after darknes.

If you want to see the animated link of the ionosphere for the past 24 hours in EarthBrowser just drag this link onto the EarthBrowser window. Or if you are reading this blog with EarthBrowser's embedded web-browser, just click on the link. There is also a light weight version of this file that just shows the current condition.

EarthBrowser 3.0

2008-04-30T14:00:00.000-07:00

EarthBrowser 3.0

Finally, it's out. Thank you to all of my wonderful supporters!!!

I transferred over the domain name to the development server and within about 20 minutes VersionTracker picked it up and I'm getting sales already.

Steve Wozniak was the 11th person to register version 3. He's been great to me and has bought several site licenses for schools. Maybe someday I'll be able to talk with the guy...

I have to give a tip of my hat to the Papervision 3D guys. They were my initial inspiration for the flash version, and although I wound up writing my own specialized rendering code, they deserve a lot of credit for what they have done! I'll go into details about how I did some things in flash in future posts, it has never been easy or fast to do software rendering. I'll also talk about some of my plans for the future.

Right now I'm sipping on a Paulaner Salvator Dopple Bock and basking in the euphoria of 4.5 years of hard work coming to fruition.

EarthBrowser 3.0 goes live tomorrow

2008-04-29T15:30:00.000-07:00

Hopefully people will like it.

It's shareware, which means you can use it for free for as long as you like. However, I'm funding it out of my own pocket, which is pretty much empty at this point. So if you want all the features and don't want to be annoyed by reminders to register please pay the modest fee and help me to make it better and better. Upgrades discounted for current users of course. Oh, by the way, registered users will be able to embed EarthBrowser with their own content on their personal websites within the next several weeks.

I just wanted to say thank you to everyone who has been so encouraging to me over the past few years; my wife, children and friends. I couldn't have done it without you.

Do I seem frustrated and bitter?

2008-04-27T08:16:00.000-07:00

I was just called out on the tone of my blog posts by someone I've known and respected for a long time. He said that he was a little surprised by my writing after reading my blog for the first time. I seemed bitter and frustrated, perhaps mostly at Google, and that isn't how he knows me. I guess I'm a little bitter, Google pretty much put me out of business years ago by releasing Google Earth for free. Hopefully EarthBrowser 3.0 will attract enough customers to keep my family stocked up on food, diapers and other necessities.

I have taken a confrontational approach to many of my posts, some of this tone is conscious, some isn't. I actually try to be a little controversial and opinionated on the blog because I'm just some joe-schmo and nobody wants to read some random guy's opinions that are mushy and congenial. That's not exciting or even interesting.

Unfortunately when writing in the blog some of my gigantic ego shows through which generally wouldn't happen in a conversation with me. I think that most programmers have really big egos, but mostly about their own coding abilities. In a self-preservation kind of way, I have to think that I'm a good programmer to tackle such a large problem by myself. I think that is just the nature of programming, no confidence = no code.

That said, I'm going to try to tone it down. Sorry libKML guys, hope I haven't offended anyone, I realize that it is just an alpha version.

KML, libkml and the "standard" mistake

2008-04-14T16:51:00.000-07:00

Passing off KML to the OGC so it could become a "standard" was a big mistake for Google.

I consider myself well versed in the KML format since I just implemented most of it's features, and found them to be needing much more in the way of styling. KML will now change at a glacial rate due to the standards process, right when it needs to change the most! Google Earth's feature set will now become dictated by an outside entity, with their input of course, but that's no way to develop software! Why would they cripple Google Earth like that?

Also libkml has been released and it was exactly what I thought it would be, a glorified xml validation script for the kml dialect. I predict that no significant software will choose to link that library in.

I have to say the lamest thing about KML is the whole Style/StyleMap tag collection that enables one to set a separate style on an icon for mouse-over events. It's a great way to have a non-standardized interface since everyone rolls their own mouse-over effect for each placemark style!

I've implemented a few of my own extensions to the KML format which I will go into detail about later. They are mostly aimed at visibility and styling extensions. One of the most useful extensions is the tag which allows one to control the visibility of a Feature element (Placemark or GroundOverlay) globally based on camera elevation. This is much easier than setting up one of those elements, especially for a single placemark.

EarthBrowser 3.0 Imminent

2008-04-08T16:10:00.000-07:00

EarthBrowser 3.0 is currently in beta testing and I think that it might live up to my prediction to revolutionize virtual globes. It should be released within the next two weeks.

Floundering In The Past

I have been working on version 3 of EarthBrowser since 2004 and it is finally coming to fruition. Up until about a year ago, I was still working on a version that was based on OpenGL with a Python scripting engine. It was basically a new platform that would do seamless version updates, enable users to extend the interface programmatically, do extremely fast raster and vector map projections on multi-gigabyte data files using JPEG-2000 incremental decompression. Use a hexagonal dataset grid to feed the fragment shader based clipmap engine for smooth panning around the poles with no perspective distortion. I was also downloading, pan-sharpening and color-space correcting the Landsat 15m dataset. Bit-torrent distributed datasets were integrated, relief mapping and atmospheric diffusion were supported. It really is a masterpiece, but then Google Earth was given away for free and sales of version 2 began to dry up as I was digging myself deeper and deeper.

The Turning Point

Then along came Modest Maps which introduced me to Flash programming. Once again plunging into yet another new technology, I decided to make a quick sliding map with a few datasets for people to put on their websites for free advertising. Then I saw Papervision3D and realized that I could do a globe. It was a little too slow for my needs so I ported my C++ game engine kernel over to Flash. I kept adding features expecting to hit the limit of what was possible with Flash and I never did. Around June I put the OpenGL version on the back burner and began working full time on a web version of EarthBrowser.

Unexpected Windfall

This February Adobe released Adobe AIR which enabled me to bring my EarthBrowser back to the desktop. AIR has some *really* nice features that make EarthBrowser much more powerful than I imagined it could be in my last post. There is an integrated web browser, right within EarthBrowser 3.0 now. That is huge and you won't really understand how easy it makes things until you browse some geo-websites and drag and drop KML links right into your placemarks folder or look at the Wikipedia page for a city or country.

Weather Datasets

There are a lot of other nice features that you won't see in Google Earth or Microsoft's Virtual Earth. I've been creating a lot of real-time datasets generated from the NOAA Forecast Models to give regions of rainfall, snowfall, humidity, temperature and many other measurements which are all animated across the globe with an intuitive time slider. Continental US doppler radar, earthquakes, volcanoes, webcams and many more datasets are all there and animated too.

Finally: KML Integration

KML support in EarthBrowser 3.0 is really nice and intuitive. There is a Panarimio KML file that allows you to see pictures from all over the world as you zoom in closer. You can download a KML or KMZ file directly within the embedded browser or drag and drop it from your desktop. EarthBrowser even has some extensions to the KML format that I felt were missing and greatly enhance the expressiveness of the format. But I'll save that for another post.

KML Mashup Tool

EarthBrowser 3.0 has really been designed to be a KML authoring and mashup tool. If you want to save a single feature from a dataset, just drag a placemark icon from the globe right into your placemarks folder and it will make a copy. You can drag out any combination of items in your placemark folder onto a text editor to make a custom KML mashup file to post it on your own website or share it with your friends. Version 3.1 will extend EarthBrowser from your desktop to your website. It will be easy since it is based on Flash technology. A simple and seamless way of creating and distributing your personal or corporate geospatial content.

I'm Not a Machine, Or Am I?

Is it possible for one man to compete with Google and Microsoft? Not really, and I'm not trying to. I want to make EarthBrowser an excellent tool for education, weather watching and KML authoring and have it be just enjoyable to use. Google and Microsoft are working hard to... hmmm... put buildings in... add more resolution... see star texture tiles and some other stuff. It takes a team of coders and a multi-billion dollar company to give this kind of software away for free. That's why I have to charge for it. I've spent countless sleepless nights and many evenings and weekends away from my family. I've gone into debt while sales of version 2 have dropped to almost nothing, trying desperately to get this new version done. I really have to thank my wife and children for their patience in allowing me to pursue the dream of owning my own business making virtual globes. Once sales pick up again with version 3, crossing fingers, I can focus on adding even more great features, but I'll talk about those later.

EarthBrowser 3.0 to be released soon

2008-01-10T11:39:00.000-08:00

It's been hard keeping quiet about it for so many months, I'm really excited about all the great new features.

Some of the major new features are:
- Supports 90% of KML
- US Doppler radar
- Rain/Snow/Temperature forecasts

Version 3 is not just another 3d globe, but will revolutionize virtual globes for reason's that will be obvious when you first run it.

If you are a customer and would like to participate in the Beta program, please drop me an email and state any relevant testing experience (not necessarily required, but helpful).

Here is a small preview:

That's all for now.

Fast Steradian Intersection Correction

2007-09-20T16:41:00.001-07:00

I just wanted to go back and mention that if the sum of the steradian angles is greater than 180 degrees, then due to the sine and cosine functions, the simple formula breaks down. However, we know that if the sum of the angles is greater than 180 degrees then by definition they intersect. Due to this, we should also store the steradian angle along with the angle sine and cosine.
So the revised intersection test algorithm becomes:

angle_a + angle_b > pi or dot(normal_a, normal_b) >= cos_a*cos_b - sin_a*sin_b

Not as pretty. A speedup for large coverages :-) but a slowdown for everything else... :-(

Simplified GeoJSON proposal

2007-09-20T16:40:00.000-07:00

I've come up an extremely simplified GeoJSON example which can do everything that the current GeoJSON specification can do but with only 4 core elements. Every element is a feature and can contain a list of points, lines and polygons or other features. Everything else in the GeoJSON spec is pretty much taken from some OGC standard or another. It's important not to get caught up in the past we are at the beginning of a potential standard, as Sean said in his post: "there's only one first chance to get a standard right." The only thing that I think might be questionably useful about this would be the nesting of features.

{
    "points":[
            [x0,y0], [x1,y1], ..., [xn,yn]
        ],
    "lines":[
            [x0,y0, x1,y1, ..., xn,yn],
            ...
        ],
    "polygons":[
        [
            [x0,y0, x1,y1, ..., xn,yn],
            ...
        ],
        [
            [x0,y0, x1,y1, ..., xn,yn],
            ...
        ],
        ...
    ],
    "features":[
        {
            "points":[
               [x0,y0], [x1,y1], ..., [xn,yn]
            ],
            "lines":[
                ...
            ]
        },
        ...
    ]
}

Also an optional "crs" coordinate reference system can contain an EPSG code, ESRI WKT (Well Known Text) or a PROJ.4 projection string or all 3. If none is specified, the default is decimal degrees in the WGS84 datum. The coordinate reference system of the parent cascades down the to all children until a child specifies one.

"crs":{
    "epsg":"4326",
    "wkt":"COMPD_CS["OSGB36 / British National Grid...",
    "proj4":"+proj=utm +zone=15 +ellps=GRS80 +datum=NAD83 +units=m +no_defs"
}

Another useful item would be an optional "bounds" element that specifies the bounding envelope in the element's crs.

"bounds":[min_lon, min_lat, max_lon, max_lat]

Minimalistic and precise.

GeoJSON redux

2007-09-20T11:05:00.000-07:00

Sean Gilles has a post responding to my call for a simplified coordinate representation in GeoJSON. His argument is that clarity of representation is more important that implementation overhead which I agree with to some extent.

However, it seems to me that the reason that JSON is so much better than XML for many purposes is exactly that it *does* take into account implementation overhead, thereby making it easier to exchange internal data structures without the overhead of XML.

For any sane implementation of JSON, the following must be extracted as a list of lists:


[ [x1, y1, z1], ..., [xn, yn, zn] ]

So a list of 10,000 coordinates will give you 10,000 lists of 3 floating points each. Or 10,000 list structures and 30,000 floats. Each list takes time and memory to create and address and extract the elements out of. Whereas:


[ x1, y1, z1, ..., xn, yn, zn ]

gives you the same 30,000 floats but only one list. A big win for any standardized JSON reading algorithm. Creating a custom, context sensitive JSON parser to ignore the structure is more than a little implementation detail.

Strangely, after Sean argues against removing context information to improve clarity, he then requests just that. He suggests that the "type" element that describes what kind of GeoJSON element you are looking at be removed since it should be obvious from the type of request that you made to receive the GeoJSON content. What if one were to receive a set of files with various different sets of data, some single features, some feature sets and perhaps some just geometry elements? If you remove the type field from the geometry elements, how would you know what kind of geometry you have? You couldn't tell a Point from a Polygon without the type field.

All that said, I do have another simplification for GeoJSON that is unrelated to the previous issues. How about we do away with the 'Point', 'LineString' and 'Polygon' geometry types. No really. They are just special cases of 'MultiPoint', 'MultiLineString' and 'MultiPolygon' with one element. I find myself writing code to take the special case single element entities and put them into the more general multi-element entities. That is code I would much rather not write since it introduces complexity and potential bugs. The only real difference in the two is the lack of a "Multi" and an extra set of an enclosing brackets.

Fast Steradian Intersection Testing

2007-09-19T09:40:00.000-07:00

I've come up with a pretty neat inclusion testing algorithm and I wanted to share it. The need is to test for inclusion of datasets within a camera view. Datasets are normally constrained to a certain area on the globe, but can cover the entire globe in some cases.

The simple way to do this is to make a bounding box with longitude and latitude coordinates and then just do intersection testing from a view bounding box. However as you get farther from the equator, the bounding boxes get progressively more distorted and you can be either over or under inclusive in your datasets.

My solution is to use steradians, or solid angles. All you need is a normal vector and an angle, four floating point numbers which is the same as a bounding box, actually I add another to speed up calculation but I'll explain that later. It performs uniformly anywhere on the sphere, and to specify global coverage you just set the angle to 180 degrees.

Now all you need to do is to calculate the steradian coverage for a dataset, and one for the visible camera volume and you can do inclusion testing with just five multiplies, three adds and a compare. It may be a little more work than a bounding box intersection test, but not much more and I think that it is an acceptable tradeoff for the properties that it provides.
It's simple really, the dot product between steradian normals gives you cosine of the angle between them. Intersection is indicated by an angle less than the sum of each steradian's coverage angle, or:

dot(normal_a, normal_b) >= cos(angle_a + angle_b)

There is a nice formula for the cosine of a sum:

cos(a + b) = cos(a)cos(b) - sin(a)sin(b)

So if you store the sine and cosine of the angle rather than the angle you get:

dot(normal_a, normal_b) >= cos_a*cos_b - sin_a*sin_b

Simple, fast and effective. I haven't found a non O(N^2) algorithm to determine the minimum steradian on a dataset yet. It is fast and easy to use lat/lon bounding boxes, but you are going to have the same problems in higher and lower latitudes with over-coverage so it is best to calculate the minimum steradian for each dataset. At least you only have to do it once on static data.

libkml: wtf?

2007-09-19T08:51:00.000-07:00

So I saw mention that Google is soon to be releasing an open source library for the kml format.

Google will be releasing an open-source KML library in C++ that implements and tracks the standard as it progresses.

I can see two intended audiences for this library; kml content creators and content consumers. I just don't think it makes sense for either of them. For kml creators, why would you need to interface with a C++ library in order to create kml files. The answer is, you shouldn't need to. It's kind of like my post on Dreamweaver, if you know what you are doing it just gets in the way and if you don't know what you are doing it is way too complex. A C++ library seems like overkill to write out some xml text. I guess it could keep track of external and document-wide styles? Big deal.

If you are a kml consumer then it makes a little more sense to use a library in C++, but not much more. Using external libraries requires you to build a bridge between your code and the library concepts. So the libkml will be dictating what types of entities you support and how they are interrelated within your code. This is restrictive on how you would develop your internal classes by forcing you to make a class structure identical to the libkml structure or you could try to build a conceptual bridge between your internal structure and the libkml structure in order to be compatible. Once you have either one, why would you need an external library just to parse the XML entities.

By providing a reference library it allows developers to more easily keep up to date with KML without having to maintain their own library and track standards changes.

So developers won't have to support any changes in the standard if they are using libkml? I guess it sounds more like it's for the kml content producers.
I guess an alternate explanation is that they are trying hard to make it seem like the standard will be truly open. Of course I'll take a look at it when it comes out to see what it's all about, but the whole concept seems like an exercise in futility.

JSON beats XML, some comments on GeoJSON

2007-09-17T10:15:00.000-07:00

I've been setting up some new data services and after experimenting with formatting my data in JSON (Javascript Object Notation) and I'm hooked. Programming mostly in C++ has made XML the easiest data format to use but I'm very unhappy with the hurdles that one must go through to go from XML to the internal representation of the data. I've been doing a lot of Javascript and Python programming lately and am just blown away by how easy it is to create, maintain and share data through the JSON format.

For XML, on the server end one would take an internal data representation in Python and create custom classes to format each data item with the appropriate XML tag, convert it to a string and output the XML tree. Not too hard if it is a relatively shallow dataset without too much data nesting. On the client end, you would write classes that would decode the tree structure of the tags with full knowledge of each tag's data type (string, integer, floating point, etc...) and create a new data structure to mirror the server side data.

For JSON, you take your data structure, usually a dictionary structure with a name associated with a value, dump it directly to JSON with a single procedure call and save it to a file. On the client end, you load the data directly into a similar data structure with a single call. No special decoding classes needed, no XML tag data types (string, integer, float) to have foreknowledge of.

I've been looking at the GeoJSON specification and it looks pretty nice and but with a few caveats. Say you have a LineString feature type, the coordinates are a list of lists which contain 2 coordinate elements (longitude and latitude). This is extremely space and processing time inefficient, which is very important for me with large datasets. I suggest using the standard single list of coordinates like is done in all other formats like GML and KML. Also it would be nice if there were some sort of feature style information specified like color, line width, line style and placemark icons.

I'll definitely be using my modified GeoJSON format when I return to C++ programing for EarthBrowser v3 soon. Sorry XML, please don't feel bad. It's not you, it's me. I just feel like we need a little space.

The Economist snubs EarthBrowser

2007-09-11T08:58:00.000-07:00

Amongst yet another "gee whiz" article about the "geoweb," it is claimed that: 'Keyhole, an American firm, released the first commercial “geobrowser” in 2001.'

I guess it's too much to ask for a magazine to research each claim made in each article, but EarthBrowser, then called Planet Earth, was the first commercial "geobrowser." It predated Keyhole's existence by several years. In fact someone from Keyhole contacted me back then and inquired about purchasing the earthbrowser.com domain name, the EarthBrowser trademark and my customer list. The amount they offered was laughably low considering EarthBrowser had over 2 million downloads and sold over 20,000 copies by then.

On a different note, several months ago I alluded to working on a new project that was a bit of a diversion from the next great OpenGL version of EarthBrowser. Project Kraken is nearing completion now and it has become better than I'd imagined it could be. It's been hard not blabbing all about it on the #worldwind or #planetgeospatial irc rooms to get useful feedback, but I've restrained myself.

Here is some newly declassified information about Project Kraken:

It will be released within the next 30 days
There will be a free version
It is easily customizable
It will compete for mindshare with Google Earth