Nowhere but here...

OpenSim on half a buck a day

2008-07-30T17:10:00.000-07:00

I recently decided to see what the big deal was about OpenSim. After initially installing it on my desktop computer and connecting to it with the latest Release Candidate viewer, I decided to try to set it up on an inexpensive hosting account to see how feasible it was to run a real sim live on the Internet. Today's blog post talks a bit about how I did it, it's not a how-to guide, and you'll probably need to be a bit of a Linux geek to follow along, but hopefully you find it interesting.

The first step for hosting OpenSim was to find a suitable hosting account. Many people try to simply connect a server to their home DSL or Cable Internet service, but there are many disadvantages to this approach, mostly arising from the inexpensive equipment provided with such services often not being particularly easy to set up for inbound connections and the fact that most Telco and Cable service providers discourage the use of home Internet connections for hosting servers. Generally, if you want to host your own server, your best bet is some form of hosting provider.

There are a number of options for server hosting, including some people who are renting pre-configured OpenSim servers. Given that you can get a nice OpenSpace region (interesting to note the similarity between OpenSpace and OpenSim, intentional?) from Linden Lab for $75/month, it's my humble opinion that it defeats the purpose of setting up your own OpenSim server if you have to pay anywhere close to that much. That said, you're not going to be able to host OpenSim on a typical inexpensive shared hosting account, because it needs it's own dedicated IP addresses and ports as well as needs to have software installed that most shared hosting accounts don't provide. Luckily, there has been a recent trend to offer what's called "virtual private servers", which uses virtualization software like VMWare or Xen behind the scenes to host multiple virtual servers on a single physical server. To you and the software you install, virtualization makes it looks like you have an entire server to yourself. After some research, I found that SliceHost offers a $20/month virtual private server hosting plan which appeared to provide enough bandwidth and computing resources to comfortablely host an OpenSim simulator. Following the excellent instructions on the SliceHost website, I was able to start with an Ubuntu Hardy Heron OS image, and install Apache, MySQL, PHP, Perl, Python, Mono, and phpMyAdmin. One key point of divergence from the SliceHost instructions was when configuring the server's firewall, it was necessary to allow access to all the ports necessary for OpenSim to operate.

I initially tried to use the same pre-built OpenSim binaries that I used on my desktop. After a few mystery crashes, I decided to use svn to get the latest version of the OpenSim code. The instructions on the OpenSim website were fairly straightforward, and I was pleased that I was able to reasonably easily compile the latest version of OpenSim.

By default, OpenSim provides very minimal physics features that don't do things like prevent you from walking through walls. The recommended physics engine is called OpenDynamicsEngine, and a pre-built binary was also included with OpenSim, but this version had problems as well, forcing me to also rebuild the OpenDynamicsEngine from source to get it to work.

OpenSim can be run in either standalone mode, as a private grid, or connected to another grid. The documentation of standalone mode versus as a grid is somewhat unclear in that standalone mode is technically a simple form of a grid in that you can have as many regions as you want in a standalone simulator. I was initially confused and thought that you needed to run a grid in order to have multiple regions, but this is in fact not the case. Typically, you'll probably only use grid mode to connect a region to an existing grid, such as OSGrid. In my case, I wanted to have my own set of regions that were part of my private sandbox grid as well as have one or two regions connected to OSGrid. This proved simple to do with a couple of caveats. First of all, when I set up the firewall, I needed to make sure that I opened enough ports for all the regions I wanted to serve. The only thing that the documentation didn't explicitly state was that I needed to create two copies of the opensim/bin directory so that I could run two instances of the simulator code, one for my private grid and the other for the regions I intended to connect to OSGrid. I'm still not sure if this is absolutely necessary, but I'm pretty sure that you have to keep these installations in separate directories because they need different configuration settings in their respective OpenSim.ini configuration files. One other detail was that it was necessary to change the setting "http_listener_port" in the OpenSim.ini file for my OSGrid simulator to be the first port number in the series of port numbers assigned to the OSGrid regions.

There were a few other things that I needed to learn through trial and error and reading between the lines of the documentation. One important tip was that OpenSim isn't yet set up to easily be run and administered in a "headless" fashion. When the OpenSim server starts up, it goes into a command-line mode where you are able to administer it by entering in commands. In order to do this without having to stay connected to your server all the time, a tool called GNU Screen can be used to allow the server to run in command-line mode while still letting you disconnect and reconnect to your server via SSH.

After completing these steps, I had 3 regions in a private grid and another region attached to OSGrid, all on a $20/month hosting account. It's hard to beat that kind of price/performance ratio, although this certainly wasn't a project for the faint of heart. If there is enough interest, I'll put up a page with the start to end steps I took.

Of course, once you do this, you'll still have to grapple with the issues of having a Ruthed avatar with bad hair and skin. Using Second Inventory and test LibSL TestClient, it's possible to move some of your SL content into OpenSim (only stuff you made yourself or you bought and paid for, of course), but that's the subject of another blog post.

Painting on the Second Life Map with Prims

2007-11-25T10:45:00.001-08:00

If you look at the Second Life Map, you often can see designs and text on the various regions. In most cases, these are fairly primitive - simple block lettering or geometric shapes. The map is updated on a weekly basis by a program at LindenLab that first creates a picture of the terrain detail using terrain elevation and terrain textures and then looks at the prims that are below approximately 400 meters in altitude. In order to make it harder to put things on the map, though, this program only looks at the color data of the prims as well as reducing the entire texture to a single color value, so, for example, a 10x10 cube will only appear as a single color when it renders on the map. Putting a megaprim up with a photographic texture will only result in large monochromatic shape.

So, how do people manage to put more complicated designs onto the map? The answer is that they construct the image out individual colored prims arranged into whatever design they want to repesent. There are some very nice examples of this out there, but they tend to be the result of a lot of work by determined individuals.

After some thought, I realized that you could automate the process, and after a lot of hair-pulling, figured out how to do it. Here are some pictures of the results of that effort:

And, if you want to see an example of this on the map, look here (click on it to go to the SLurl map page):

Now, to see how these pictures were actually rezzed in-world, here's a video I made of the process:

Click To Play

Figuring out how this could be done turned out the be the easy part, implementing it was considerably harder. First of all, using a single prim for every color "pixel" would have used too many prims for this to be practical. Luckily, the answer came from the popular XYText script, which uses a technique of flattening prisms to create a rectangular shape with 5 upward facing surfaces, allowing 5 pixels of color to be displayed per prim.

The second challenge was how to get the color information into Second Life. The goal was to make this easy enough that anyone with an image in any of the standard file formats (GIF, JPG, PNG, BMP), could turn these into a MapPainting. Writing a Java application to take these images and turn them into textual data that LSL could work with was largely straightforward. Unfortunately, the result was still enough information to fill several notecards. The first version of the project simple wrote out a set of textfiles which the user could copy and past into a notecard within the Second Life viewer. This proved cumbersome for the testers, all of whom were fairly sophisticated SL residents. The solution was to turn the desktop application into a web application, and simply allow people to upload their image onto the NowhereVirtual.com web site. The in-world scripts connect to the web application which feeds them exactly the data they need. The result is a much simpler process, the user simply uploads an image on a web page, clicks "Rez" from an in-world menu and away you go.

The final challenge was how to rez this many prims this quickly, given that LSL automatically slows down the scripted creation of new prims to make griefer scripts less effective. We'll keep the solution to that problem a trade secret for now. ;-)

The final product, MapPainter, is available in two versions, the full version which can fill a sim, and the lite version, that will fill a 4,096 square meter parcel. You can purchase it in-world or on SLExchange and OnRez:

MapPainter at SLExchange
MapPainter at OnRez
Nowhere Virtual Ministore

For more information, visit the Nowhere Virtual website:

MapPainter Product Page at Nowhere Virtual
MapPainter Documentation Page at Nowhere Virtual

TerrainTool and more on Sculptie Sim Models

2007-09-29T15:10:00.000-07:00

In my last post from way back in June, I mentioned that I was writing a program to help produced sculpties from sim RAW terrain files. I've been delinquent in following up on that, and to that end, have finally uploaded a copy of the TerrainTool program that will let you do this. You can find it here:

http://www.nowhere-sl.com/terraintool/

This was part of an effort to make a more full-featured terrain visualization tool in Java, which perhaps at some point will happen. In any case, it's quite useful for producing sculptie textures that can be uploaded into Second Life and used to make scale models of sim terrain.

One thing that can be useful when making these types of models, is to texture them with the actual image from the grid map. Here's a short script that will let you get that for the region you're in:


string getSimMapURL() {
    string regionName = llGetRegionName();
    vector rc = llGetRegionCorner();
    integer regionX = (integer)(rc.x / 256);
    integer regionY = (integer)(rc.y / 256);
        
    llSay(0, "Region \"" + regionName + 
        "\" at: " + (string)regionX + 
        ", " + (string)regionY);
        
    integer mapY = 1279 - regionY;
        
    string mapApiRestEndpoint = 
        "http://secondlife.com/apps/mapapi/grid/map_image/";
    string mapImgUrl = mapApiRestEndpoint + 
        (string)regionX + "-" + 
        (string)mapY + "-1-0.jpg";
       
    llSay(0, "Map Image URL: " + mapImgUrl);
        
    return mapImgUrl;
}

default {

    state_entry()
    {
        llSetText("Click me to get the Map URL for the current region",
            <1,0,0>, 1.0);       
    }

    touch_start(integer total_number)
    {
        getSimMapURL();
    }
}

Simply drop that script into a prim, click on it, and you'll get an image URL like this:

http://secondlife.com/apps/mapapi/grid/map_image/926-344-1-0.jpg

Following that URL will take you to the actual image from the grid map, which should look something like this:

Simply save that image and upload it as a texture into SL, and you can apply it to your scupltie sim model.

Making Region Sim Models Using Sculpties

2007-06-16T09:18:00.000-07:00

Check out these pictures of the scale model of the Nowhere sim that I made using sculpties. Turns out it's not too difficult to create a sculptie texture using the region RAW file and Photoshop. I'll post detailed instructions for doing this by hand as well as a tool for automatically generating these soon.

Open-Sourcing my Update Server scripts

2007-06-12T13:36:00.000-07:00

I posted the code for the update server I use to update the teleport pads on Nowhere on the Second Life Wiki. I use AubreTec Teleport Pads for teleporting between locations in the sim, and I've been quite happy with them. The problem is that the pads are configured with two notecards, one of which contains the locations and the other than contains authorized users. Whenever I add a friend to the list of authorized users or add a new teleport pad, I need to visit each pad and copy new notecards in, which gets pretty tedious. Since necessity is the mother of invention, I wrote up an update server which distributes these new notecards automatically. I later started using this system for updating Freeview screens and anything else that gets configured this way. You can find the code on the Second Life Wiki at the following link:

Intra-Region Update Server

Hope you find it useful!

Emma vs. the Subway

2007-05-17T09:15:00.001-07:00

Emma vs. the Subway
Originally uploaded by Emma Nowhere.

I meant to post this earlier... I took this picture at Downtown on the day it opened to the public. I can't say enough how beautifully crafted this place is, or to do justice to the emotions it provokes in those who visit it. I really wish I'd saved this picture to disk as a high resolution file rather than as a 512x512 texture, but I still love it. I really wasn't sure what would happen when the car hit...

Twittering from SL

2007-05-14T10:49:00.000-07:00

I've read a number of blog posts by Ordinal Malaprop, Kisa Naumova and others about using Twitter from SL. Very cool stuff! The great thing about Twitter, from a scripting standpoint, is that if you just want to do updates from within Second Life, you can do it with just a few lines of LSL code and post directly to the Twitter servers. Here's a simple example that I hope you'll find useful:


//put your Twitter username and password here
string twitter_user = "";
string twitter_pass = "";

updateTwitter(string update) {
    if ((twitter_user != "") && (twitter_pass != "")) {
        string url = "http://" + twitter_user + 
            ":" + twitter_pass + 
            "@twitter.com/statuses/update.xml";
        llHTTPRequest(url, [HTTP_METHOD, "POST", 
            HTTP_MIMETYPE, 
            "application/x-www-form-urlencoded"], 
            "status=" + llEscapeURL(update));
    }
}

default {

    state_entry() {
    }
    
    touch_start(integer num_detected) {
        updateTwitter("Hello from LSL");
    }

    http_response(key request_id, 
        integer status, 
        list metadata, 
        string body) {
        //always returns a 415 error
        //unless it fails, in which case
        //a 500 error is returned
    }

}

Construction Time Again

2007-05-14T10:23:00.000-07:00

Just a quick update - I'm starting planning on the companion sim to Nowhere. Looking at coastlines in Google Images for inspiration and I've been privileged enough to visit and live in few interesting tropical locales in RL. Thematically it will be very similar to Nowhere, with my signature landscaping techniques, such as gently sloping coastlines, private bays, and natural mountains. I'll post more details as the plan gels.

I'm also hoping to make some progress on finishing the Lost Hatch (aka Dharma Initiative Swan Station) skybox build that I started on so long ago.

The layout has been done for months, except for a few details, it just needs to be textured, which is unfortunately outside of my skill set.

In my quest to better learn texturing, I did read the excellent book 3D Game Textures: Create Professional Game Art Using Photoshop by Luke Ahearn. While it will take more than even a wonderful'y written book such as this to impart graphic design skills on someone like me, it at least helped to explain to me how places such as Downtown can look so amazing. I recommend it to anyone who's trying their hand at building anything in Second Life since good texturing is so often what makes the difference between good and great in anything in-world. One point that the book makes, but which is even more important in SL, is the size of textures. If you're using textures that are greater than 512x512, and you've got several of them in a build, you ARE going to be making life difficult for people who don't have higher-end graphics cards who will experience render lag, which they'll probably incorrectly attribute to your sim or SL. Many SL users aren't PC gamers with the latest and greatest hardware, so pay attention to texture size and please use smaller textures when possible.

Noteworthy: Sysperia Poppy's work at The King Has Fallen gallery

2007-04-30T10:02:00.001-07:00

Melancholia at home
Originally uploaded by Emma Nowhere.

Sysperia Poppy has done an amazing collection of portraits, a few of which feature a close friend of mine, Super Calamari, or Cala as she prefers to be called. The title of this one, Melancholia, struck a nerve, since I've been feeling more than a little down lately, and Cala has always been there to cheer me up.

Sysperia's work can be seen in-world at her gallery, The King Has Fallen, at Sonata (159, 235, 262).

Using Flickr photos in Second Life

2007-04-22T16:52:00.000-07:00

My friend Lana Miranda recently pointed out to me that the QuickTime player in the Second Life client could also play SMIL presentations. For those that don't know, SMIL is an XML-based media description format, created by Real Networks back when Philip Rosedale was the Chief Technology Officer there before he founded Linden Labs and became Philip Linden. SMIL was standardized by the W3C and Apple implemented SMIL 1.0 as a supported media type in QuickTime 4.1 (we're at version 7.1 of QuickTime now, although sadly it still only supports SMIL 1.0 and not the later versions of SMIL which provide nice features like transitions and animation).

Second Life has the ability to play QuickTime movies overlayed on primitive objects. As a consequence, you can play streaming videos on in-world television sets and movie screens. But, since QuickTime can also play SMIL-based presentations, this opens some interesting possibilities for playing other media types as well.

One of the more interesting usages that occurred to us was to be able to pull images from Flickr and use them in SL. To that end, we used Flickr's robust web services API to create a nice little "Web 2.0" web application called PhotoMovr to let you do this easily. You can check it out at http://www.photomovr.com. It's just a beta and not hosted on a particularly fast server, so if it gets a lot of usage it might slow down, but it does work quite nicely at the moment. If you have a Flickr account and you have photos organized into sets, it's very easy to use. Feel free to give it a spin.

Here are some snapshots of Flickr photos being viewed in-world using the QuickTime movies generated by PhotoMovr:

Creating parcels

2007-04-07T14:13:00.001-07:00

Nowhere HQ Land Planning 1
Originally uploaded by Emma Nowhere.

Most rental sims try to divide up the land into as many rectangular parcels as possible. Nowhere is meant to appeal to people who appreciate the aesthetics of the landscape and want to live in a place that appears to be the result of natural geological processes. This means that parcels are going to be somewhat harder to draw out than in most of the rental flatlands. Here, you can see me reviewing the parcel boundaries which are overlayed on a map of the region that was retrieved using the undocumented SL Map API.

How to plan a sim

2007-04-07T14:07:00.001-07:00

Nowhere HQ Land Planning 6
Originally uploaded by Emma Nowhere.

I haven't posted recently, so I thought I'd share a couple of snapshots of planning out a sim landscape. Here you can see the original landscape of Nowhere and below it the updated version. The images were created using Backhoe and Screenshot Plus on a MacBook Pro and then using Photoshop to crop and save as TGA files to upload into Second Life.

Second Life Terraforming Tools

2007-03-15T12:08:00.000-07:00

When I first terraformed the Nowhere sim, I used Photoshop to finesse some maps of the island from Lost into a terrain file compatible with Second Life. In the process, I discovered two things. The first is that the Lost island is much too big to properly fit inside in a single SL region, and second, that editing terrain files in Photoshop is a bit of a hassle. As part of my efforts, I used a very nice tool called Backhoe to visualize the terrain before uploading it into Second Life. Here's a screen shot of the island in Backhoe:

When I decided to re-landscape the island to make it more amenable for residential rental, I took the opportunity to revisit the process of terrain file editing and take a second look at Backhoe as well as some of the other tools.

Just to recap for those who aren't familiar with terrain files in Second Life, the landscape of a SL sim region can be downloaded or uploaded as something called a heightmap. The heightmap is an array (think spreadsheet) of 256 rows by 256 columns, each cell of which can contain a value between 0 and 255, where 0 is the lowest point and 255 the highest. For the purpose of editing this file, it's typically imported and exported from Photoshop as a "raw" file. Photoshop treats this as a 256 x 256 pixel greyscale image, where a black pixel represents a land height of 0 and a white pixel represents a height of 255. There is another part of the file where a scaling factor is stored to translate the height value into meters.

Typically, people use a combination of in-world editing tools and Photoshop to create new island terrains. Photoshop makes it easy to create mountains and lakes very quickly, and the different blur filters can produce very nice smooth terrains. However, it still is often necessary to use the in-world tools to tweak the landscape and also because you can get some unattractive results, particularly around coastlines, when you edit the heightmap file in Photoshop because the terrain LOD (level of detail) algorithm of the SL client doesn't like some of the slopes of the land created in PS .

By Googling for "heightmap editor" and "terrain editing" I was able to find a number of other interesting tools out there that look very promising for editing heightmaps usable with Second Life. I've detailed my preliminary findings below.

Keep in mind, before you expect to have productive results with any editing of terrain files, you really need to fully understand the KB article:
http://secondlife.com/knowledgebase/article.php?id=288
(by the way, the description of how parcels are encoded does *not* work)

Second Life Specific

These will directly open an unmodified SL RAW terrain file:

Backhoe
http://www.notabene-sl.com/Backhoe/
Mac only, open source, beautiful 3D visualizing, editing tools are ok but somewhat coarse grained, I tend to use this side by side with Photoshop to see what changes I make in 2d effect the 3d view.

Here's a screenshot of my redesign for Nowhere island viewed in Backhoe:

Bailiwick
http://www.spinmass.com/bailiwick/
Windows only, nice for visualizing the coastlines, good for getting the terrain layer in and out of the raw file for editing with something other than Photoshop CS, for example Paintshop Pro, Photoshop Elements

Second Life Compatible

These should work with an SL RAW file, although some finessing of the file will likely be required - specifically, the SL raw file contains 13 256x256 8-bit interleaved channels, only the first of which contains the terrain heightmap. As a consequence, if the editor wasn't designed with SL files in mind, it sees the extra data and chokes. The workaround is to open the SL raw file in PS, copy the heightmap channel into a new 256x256 8-bit greyscale image and save that as a new single channel raw file. Once you've completed your editing of the single channel raw file, you can use PS to copy it back into the SL terrain file. Alternately, you can use Backhoe or Baliwick to import and export the terrain channel, and I'm working on a simple Java app for doing this as well.

Height Map Editor
http://sourceforge.net/projects/hme
Windows and Linux, although open source, so maybe a Mac version could be compiled

Freeworld3d
http://www.freeworld3d.org/
Windows only, I had high hopes for this one, but couldn't get it to run under Vista. Should work with SL files and I've seen a couple of positive comments about it on various game developer forums.

L3DT
http://www.bundysoft.com/L3DT/
Windows and Linux, looks very powerful, didn't spend too much time with it

Terragen
http://www.planetside.co.uk/terragen/
Windows and Mac, this one wants a 257x257 heightmap file, so I had to extract the heightmap channel in PS, resize the canvas to 257x257, and resave as a new RAW file to get it to open. Looks like it has potential.

Leveller
http://www.daylongraphics.com/produ...eller/index.htm
Windows only, looks interesting, haven't really tried it yet