Pegasus Bridge

A bridge too far. It's over.

2007-08-11T03:37:00.000-06:00

As hinted on several blogs, we are not semi-finalist to the urban Challenge. We did not make that bridge. Thank you for all the good wishers. At some time, we'll try to write and talk more about our system of ominidirectional cameras using compressed sensing and dimensionality reduction in policies to accomplish what we intended to accomplish during the site visit.

hold pattern

2007-06-12T01:41:00.000-06:00

FYI, We are not reporting much as we are preparing for the site visit and trying to make some innovative use of light bulbs :-)
[update June 21: they will be on the rooftop]

Site visit

2007-06-02T14:59:00.000-06:00

Our site visit will take place on the afternoon of June 27th at the the Southwest Research Institute. As for the technical report, I am going to hold onto it for a little while since we are still investigating some of the final algorithms/hardware to be implemented for the site visit. I thought we would be done by then but day-jobs and other considerations are making this a little optimistic.

We have a site visit.

2007-05-11T09:09:00.001-06:00

See y'all you in San Antonio. The 53 teams remaining are listed here.

Unveiling our algorithm

2007-05-11T07:38:00.000-06:00

These entries are our part of our current series on unveiling our algorithm as featured in this robots.net story.

Technical Paper

2007-05-10T14:11:00.000-06:00

The technical paper will be submitted to DARPA before June 1st at 5:00 ET. We will make it available on this site at that time. It consists of 25% of the grade given to the entry for a spot to the semi-finals.

Site visit announcement tomorrow

2007-05-10T08:07:00.000-06:00

We just received this:

An email with the site visit announcement and press release will be emailed on Friday, May 11 at 1100 AM ET.

DARPA Urban Challenge

on a different note, we were featured on robots.net

Site visit course RDNF and our algorithm presentation

2007-04-25T15:12:00.000-06:00

The folks at Southwest Research Institute have done a great job at defining the course for the site visit. CP#6 is where the obstacle (yellow car lies). With respect to unveiling our algorithm, we are in the middle of devising a lot of things (a lot of us have finals) for the time being but we have given an outline on the basics needed to a implement a GPS waypoint following vehicle by pointing out to the effort of a vehicle prepared by a large University next door (disclaimer: we have no connection what so ever with this effort) which eventually did not enter the Grand Challenge. Most entries on the subject will be at http://nuit-blanche.blogspot.com/search/label/darpa%20urban%20challenge
.

Unveiling our algorithm, trade-offs and implementation

2007-04-16T06:00:00.000-06:00

It is still a work in progress for some parts, but we are going to unveil the algorithm driving our vehicle: Pegasus Bridge 1, as we go along. We are still gathering data and evaluating them against different algorithms. We expect to talk about the implementation as well as the general concepts from GPS waypoint following, road following, situational awareness and learning rules. We hope to be able to do this in a timely manner, but as we are under-manned, between reporting on the algorithm and making it better we will always go for the latter. The reporting will take place here.

Part 4 Video submission completed.

2007-04-14T03:51:00.000-06:00

We sent our video to DARPA yesterday. Next stop San Antonio at the Southwest Research Institute.

Visit of the Southwest Research Institute and our Strategy

2007-04-07T16:28:00.000-06:00

As usual the visit at Southwest Research Institute went very smoothly. We are working on our video for the part 4 of the application. Because our approach is truely innovative and we expect no other team to go our route, we are unveiling our algorithm/software strategy in several excerpts here as we go along.

Site visit will take place at Southwest Research Institute

2007-03-16T09:58:00.000-06:00

Since we liked it so much last year, we have decided to come back to SouthWest Research Institute for the DARPA site visit. Photos of the site can be found here.

We are in the Track B pool of teams

2006-11-09T17:44:00.000-06:00

We're in.

Application time

2006-10-17T06:51:00.000-06:00

Our application was sent and received by DARPA. They will decide whether or not we are selected for Track B on October 18th.

A New Race

2006-05-02T08:28:00.000-06:00

We have decided to enter DARPA's Urban Grand Challenge race (in french)

On a totally unrelated note: the location of the historical Pegasus Bridge is here.

Stanford Racing Team is the winner of the DARPA Grand Challenge

2005-10-08T15:01:00.000-06:00

Outstanding, three contestants arrived within 10 minutes of each other!

National Qualification Event

2005-10-05T04:11:00.000-06:00

There are interesting videos from the NQE before selection is made on Thursday. They can be found on the CarTV site. In this video, Stanford provides insight in the vision system of their entry. Stanford has their own stock of videos.

DARPA Grand Challenge NQE underway.

2005-09-30T10:22:00.000-06:00

The DARPA Grand Challenge National Qualification Event has taken place for the past several days, more information can be found on this brochure. A good blog on the event can be found here. Meanwhile, since we have been eliminated for this year's event, we are playing with Google Maps technology and are evaluating how it can be applied to our display of engineering data. John Wiseman did a great job of displaying last year's DARPA race through Google Maps. We just used it to display places on the Paris map for the "Nuit Blanche" event. We feel it has a good potential when applied to the data we are making freely available from our test runs.

Result of DARPA selection to NQE

2005-06-08T07:10:00.000-06:00

On Monday, June 6th, DARPA announced a list of teams that would participate to the NQE (National Qualifying Event.) Unfortunately, we are currently not on that list.

How do you drive-by-wire a car using Python ?

2005-05-30T06:46:00.000-06:00

In the previous entry, we made the comment that using Python was very nice because of the small learning curve. Here is an example below that should help anybody who has ever written a program understand our point. To the first time reader, it may even look like pseudo-code.

Our vehicle used to use this program to be driven around through a laptop installed in the car. Someone located in the vehicle would punch keys on the keyboard and would be able to accelerate/brake and turn right/left. The commands would then go through the parallel port of the laptop to a controller which would in the end command several electrical steppers motors doing the manual job. We left the GPS thread in the code so that one can see how to use threads. In effect in the real program, we also use other threads to monitor cameras and other inertial information coming from the IMU but we left them out of this code for the sake of clarity. It runs under Windows and is fast enough for the purpose at hand. The drive-by-wire code enables us to drive the vehicle around and allows the gathering of data on maneuverability and collision avoidance. The sentences said by the program through the TTS module may look like a gimmick, but they are in fact very helpful when only one person drives around and collects data at the same time.

The program starts two threads. One is watching the keyboard for instruction. The other one is recording data from the GPS sensor.

# This code is released 
# under the General 
# Public Licence (GPL), 
# DISCLAIMER : if you do not know
# what you are doing 
# do not use this program on a 
# motorized vehicle or any machinery 
# for that matter. Even if you know 
# what you are doing, we absolutely 
# cannot be held responsible for your 
# use of this program under ANY circumstances.
# The program is 
# released for programming education
# purposes only i.e. reading 
# the code will allow students
# to understand the flow of the program.   
# We do not advocate nor condone
# the use of this program in 
# any robotic systems. If you are thinking
# about including this software
# in a machinery of any sort, please don't do it.
#
# This program was written by 
# Pedro Davalos, Shravan Shashikant,
# Tejas Shah, Ramon Rivera and Igor Carron
#
# Light version of the Drive-by-wire program 
# for Pegasus Bridge 1 - DARPA GC 2005
#
import sys, traceback, os.path, string
import  thread, time, serial, parallel
import re
import struct
import msvcrt
import pyTTS

rootSaveDir = "C:\\PegasusBridgeOne\\"
gpsSaveDir = os.path.join(rootSaveDir,"gps")
ctlSaveDir = os.path.join(rootSaveDir,"control")
stopProcessingFlag = 0
printMessagesFlag = 1
p = None
tts = pyTTS.pyTTS()
gps_data = list()
imu_data = list()

def runnow():
    print "Drive-by-wire program running now"
    #launch controller thread
    t = thread.start_new_thread(control_sensors,())
    #launch gps sensor thread
    t1 = thread.start_new_thread(read_gps_data,())

def control_sensors():
    global stopProcessingFlag
    global ctlSaveDir
    global p
    printMessage("Control thread has started")
    p=parallel.Parallel()
    p.ctrlReg = 9  #x&y axis
    p.setDataStrobe(0)
    sleep = .01
    while (stopProcessingFlag != 1):
       key = msvcrt.getch()
       length = len(key)
       if length != 0:
          if key == " ": # check for quit event
             stopProcessingFlag = 1
          else: 
                if key == '\x00' or key == '\xe0': 
                    key = msvcrt.getch() 
                print ord(key)
                strsteps = 5
                accelsteps=1
                if key == 'H':
                    move_fwd(accelsteps, sleep)
                if key == 'M':
                    move_rte(strsteps, sleep)
                if key == 'K':
                    move_lft(strsteps, sleep)
                if key == 'P':
                    move_bak(accelsteps, sleep)
    print "shutting down"
    ctl_ofile.close()

def read_gps_data():
    global gpsSaveDir
    global stopProcessingFlag
    global gps_data
    printMessage("GPS thread starting.")
    #Open Serial Port with baud 4800 on port 5 (Com6)
    ser = None
    try:
        gps_ofile = open(''.join( \           
              [gpsSaveDir,'\\',time.strftime**
          **("%Y%m%d--%H-%M-%S-"),**
          ** "-gps_output.txt"]),"w")
        ser = serial.Serial(5, 38400, timeout=2)
        while (stopProcessingFlag != 1):
           resultx = ser.readline()       
          # if  re.search('GPRMC|GPGGA|GPZDA', resultx):    
           splt = re.compile(',')      
           ary  = splt.split(resultx)  
           gps_ofile.write(str(get_time() + ', '+resultx))
           gps_data.append([get_time(), resultx])
           if (len(gps_data)> 100):  gps_data.pop(0)
        ser.close()            
        gps_ofile.close()
    except:
        write_errorlog("GPS Read method " )
        if ser != None:
            ser.close()

def write_errorlog(optional_msg=None,whichlog='master'):
    if optional_msg!=None:
        writelog(whichlog,"\nERROR : "+ optional_msg)
    type_of_error = str(sys.exc_info()[0])
    value_of_error = str(sys.exc_info()[1])
    tb = traceback.extract_tb(sys.exc_info()[2])
    tb_str = ''
    #  error printing code goes here

def get_time(format=0):
    stamp = '%1.6f' % (time.clock())
    if format == 0:
        return time.strftime('%m%d%Y %H:%M:%S-',  \ 
             time.localtime(time.time())) +str(stamp)
    elif format == 1:
        return time.strftime('%m%d%Y_%H%M', \ 
                    time.localtime(time.time()))

def if_none_empty_char(arg):
    if arg == None:
        return ''
    else:
        return str(arg)

def writelog(whichlog,message):
    global rootSaveDir
    logfile = open(os.path.join(rootSaveDir, \ 
                  whichlog+".log"),'a')
    logfile.write(message)
    logfile.close()
    printMessage(message)

def printMessage(message):
    global printMessagesFlag
    if printMessagesFlag == 1:
        print message

def move_fwd(steps, sleep):
    global p
    for i in range (0,steps):
        p.setData(32)
        p.setData(0)
        time.sleep(sleep)

def move_bak(steps, sleep):
    global p
    for i in range (0,steps):
        p.setData(48)
        p.setData(0)
        time.sleep(sleep)
    
def move_lft(steps, sleep):
    global p
    for i in range (0,steps):
        p.setData(12)
        p.setData(0)
        time.sleep(sleep)

def move_rte(steps, sleep):
    global p
    for i in range (0,steps):
        p.setData(8)
        p.setData(0)
        time.sleep(sleep)
        

print 'Hello'
tts.Speak("Hello, I am Starting the control**
          **Program for Pegasus Bridge One")
time.sleep(4)
runnow()
while (stopProcessingFlag != 1):
     time.sleep(5)
print 'Bye now'
tts.Speak("Destination reached, Control Program**
**Aborting, Good Bye, have a nice day")
time.sleep(4)

On being innovative - part I

2005-05-25T03:22:00.000-06:00

In Beating the averages, Paul Graham described how his start-up was able to get an edge on competitors :

The same thing will happen if you're running a startup, of course. If you do everything the way the average startup does it, you should expect average performance. The problem here is, average performance means that you'll go out of business. The survival rate for startups is way less than fifty percent. So if you're running a startup, you had better be doing something odd. If not, you're in trouble.

While we are not going to talk about eveything on how we approach the vehicle for the Grand Challenge, we can describe how we are doing something odd:

Using the Sensors We Understand

Our approach to the Race is to have no range sensor like Lidars. Why ?

first, because everybody seems to be doing it and it does not make sense to differentiate ourselves on how much money we can spend on a specific sensor, we know we can easily be outspent,
second, because we do not see a good way for the use of the technology developed for this project if we are not making more sense of data that are generally gathered by humans to make decisions. In the end, there are far more possibilities to use research based on sensors for which humans are accustomed to, than if one were to make sense of data from specifically crafted sensors like Lidars. There is a cost associated with devising new sensors in a new range of wavelengths, but the central issue is not having made sense of the data one already had access to. If you have been so unsuccessful at making sense of data from sensors you have been using all your life (at least most of us have had vision) why do you think you are going to be more successful with other types of sensors ? We are not saying it is easy. As the literature shows it is still a big issue in the robotics vision community but we think we have a good strategy on sensing and making sense of data.
Third, it did not seem to help the competitors last year.

Eventually our interest converges on fusioning data from inertial and vision sensors in order to provide navigability as well as obstacle avoidance capability as humans can do with their eyes and vestibular system. This is a subject that has only been recently looked into in the academic community even though much has been done of Structure From Motion. A related interest is how gaze following tells us something about learning.

Besides GPS, IMU and visual data, we are using audio as well. The driving behavior of humans is generally related to the type of noise they hear when driving. We are also looking at fusioning this type of data with the video and the IMU. If you have driven in the Mojave desert, you know that there are stanger sounds than the ones you've heard on a highway. They tell you something about the interference between one of your sensors (the tires) and the landscape. This is by no means data to be thrown out since your cameras tells you see if the same landscape ahead of the vehicle is to be expected.

Making Data Freely Available and Enabling Potential Collaborative Work with People We presently Do Not Know

One of the reasons we put some data on our web site that features GPS, IMU (acceleration, heading and rotation rate) is because we want other people who do not have the means nor the interest to take a stab at the problem of setting up a system that can gather data to still be able to use real world data for some initial finding. More data will be coming within the coming months and we are looking for some type of academic/research institution to host these data. We have collected 40 gigabytes of data so far but with a monthly bandwidth allowance of 42 GB per month. We do not expect to survive any interest from the robotics community past the first person who downloads all of them (we have had more than 30 people who downloaded the larger file after making an announcement on robots.net a month and half ago.)

Teaming with other researchers that are specifically in this area is of interest but this is not our main strategy. The odd part of this strategy is the statement that data do not matter as much as making sense of them. It doesn't matter if you have driven 100 miles and gathered data with your vehicle if you have not done the most important: postprocessing.

Using the Right Programming Language

We are currently using the Python programming language. Why ? for several reasons,

we do not want to optimize ahead of time the respective challenges occuring in building an autonomous vehicle, hence we are neither using C/C++, Java nor any real time operating systems for development and the final program. Most announcements on other teams web site look for Java, C/C++ programmers.
Python is close to pseudo-code and the learning curve for newcomers into the team as well as veterans is very smooth especially coming from Matlab.
Python is providing glue, libraries, modules for all kinds of needs expressed within the project (from linear algebra with Lapack bindings , to Image analysis, to plots to voice activation with PyTTS, to sound recording and others)
Thanks to Moore's law, Python is fast enough on Windows but we do not feel we would be losing too much sleep having to switch to a different OS or having to create modules that are specifically faster in C. Using the Windows OS helps in debugging the programs since our team is distributed geographically and everyone generally use Windows on their laptops.

Using the Right Classification Approach

One of our approach to make classification of images for the purpose of road following as well as obstacle avoidance is based on defining some type of distance between drivable and non-drivable scenes. There are many different projection techniques available in the literature. Let us take the example of the "zipping-classifier" mechanism and how we are using it for obstacle avoidance. This technique uses a compressor like Winzip or 7-zip to produce some type of distance between files (here images.) A very nice description of it can be read here. The whole theory is based on Minimum Description Length. An example of a naive classification made on pictures we took at the site visit in San Antonio are shown below. In the interest of reproducibility, we used CompLearn to produce this tree. (Rudi Cilibrasi is the developer CompLearn.) The tree clearly allows the segmentation between open areas and the ones with trash cans. We did no processing to the pictures. Obviously, our approach is much more specific in defining this distance between a "good" road and a not-so-good one.

classification of trash can obstacle

Using the API for Google (images), we can build libraries of relevant information for road driving experience purposes and compute their "distance" to the road driven by our vehicle.

Let's hope we are doing something odd enough.

Gaze following when car racing - Review -

2005-05-16T06:37:00.000-06:00

[Reviews in this blog are meant to convey some of the issues our team has been facing when trying to implement a vision based system (as opposed to a range based system) to guide Pegasus Bridge 1, our entry in DARPA's Grand Challenge]

Anybody who has ever driven in the desert knows that one's reflexes gathered while driving on a highway do not work as well in this new environment. Point in case, the generic driving behavior one can observe in street car driving as shown in these two videos from gaze tracking vendors (1 and 2 or 3) and the roads being driven in the WRC (watch the free preview of the Cyprus Rally) which are likely to ressemble more the type of terrain one will observe in the Grand Challenge (ableit not at the same speed.)

We have been in talks with one of the vendors, but at 25-35 K$, there has not been a place where each of out interest and theirs have overlapped. Maybe some other time.

Why is gaze following important ? because this is the only way to reduce the field of vision to be decomposed for analysis. If one were to have a 640 by 480 camera, and tried to analyze the field of vision given by the camera, it is very likely that the analysis would take too much time for the vehicle to make a relevant decision (please note that I am not even talking about the way the computation is performed and the type of hardware used.) Not only that, but gaze following by a human is a filter for objects and scenes of relevance to the driving experience. If one were to have the same "reflexes" most of the driving experience would eventually be about choice and how one's speed is considered "safe" by the driver considering the terrain. I will talk about terrain some other time, in another review using the data we are making available.

Site visit in San Antonio

2005-05-12T23:55:00.000-06:00

The site visit went on yesterday. It was not optimal. We think the DARPA folks were very open in as much as the fact that we are advertizing ourselves as being one of the few teams that will not have direct range instruments (Lidar,...) Time will tell. Kudos to the outstanding team at Southwest Research Institute (JRP/NUSE2) who hosted us (Mike, Carlos, Joe, Dan and Bill), their hosting was flawless.

Site Visit Scheduled

2005-04-21T06:05:00.000-06:00

Our Site Visit will take place at the Southwest Research Institute in San Antonio on May 12, 2005. It also looks like two other teams from Austin and the New Zealand folks will be in the same batch. Good luck y'all.

Selected for site visit.

2005-04-04T14:10:00.000-06:00

Along with 118 teams, we have been selected (out of 137 teams) for a site visit in May. Here we come San Antonio.