ASP v2 Status Update

If you watch your email or ISIS’s website like a hawk, you’ll notice ISIS 3.4.0 just dropped this week. So where’s Ames Stereo Pipeline in all of this?

The current plan is to get Ames Stereo Pipeline 2.0 out before the Planetary Data Workshop on June 25th in Flagstaff. At this meeting I hope to be giving a tutorial and showing off all the cool stuff the community and the team have been developing for ASP.

http://astrogeology.usgs.gov/groups/Planetary-Data-Workshop

I currently have two limiting factors that I hope will get resolved in the next 2 weeks:

  • ASP and Binary Builder need to agree with the changes in ISIS 3.4.0.
  • The NASA Ames Lawyers need to re-license Vision Workbench under the Apache 2 license. ASP has already been re-licensed to Apache 2, however one of its prime dependencies is Vision Workbench. This is a legal block against releasing binaries.

Thank you all for your patience and for doing neat projects with Ames Stereo Pipeline. I enjoy seeing the pretty pictures at science conferences.

Things to look forward in this next release are:

  • Digital Globe Image support.
  • Faster and Memory efficient integer correlator.
  • GDAL projection interface for point2dem.
  • Parameter settings from command line and configuration file.

MacPorts Portfiles available for VW and ASP

As of just a minute ago, I’ve committed a portfile for VW and for ASP in their respective code repositories. The ASP one doesn’t support ISIS or point2mesh. It’s only good for performing stereo on pinhole sessions (MER/Personal Robots) or DG sessions (Digital Globe). I hope that eventually Macports will accept them into their distribution as vw-devel and asp-devel. Until that day, you can use these port files manually using these instructions.

Building Ames Stereo Pipeline against ISIS on OSX

This is a guide for advanced bearded users. If you don’t have a beard, don’t try this at home.

I wrote previously about how to do this on Ubuntu 10.04. Today I’d like to be more explicit so that OSX users can understand how to download and build the latest code. I’m going to make an assumption that your OSX system has Macports. To start us out, here are the packages that you are going to need on your OSX system.

sudo apt-get install libtool git-core subversion cmake xz

If you are using OSX 10.7 Lion: You’ll need to also manually install the OSX 10.5 SDK. This is required for building against ISIS since they currently only have a 10.5 version of their binary. This also provides a more stable compiler in my opinion.

Another tricky part is that you need to compile your own copy of GNU Coreutils. This is required since Binary Builder was written with Linux systems in mind and not the weird FreeBSD like things in OSX. We don’t use macports for this step because their scripts tend to install a ‘g’ prefix on every executable. Building GNU Coreutils would look something like the following:

cd ~/packages
wget http://ftp.gnu.org/gnu/coreutils/coreutils-8.16.tar.xz
xz –d coreutils-8.16.tar.xz
tar xf coreutils-8.16.tar.xz
cd coreutils-8.16
./configure --prefix $PWD/build
make install -j 15

Since building against ISIS requires recreating their development environment, we provided a fancy script to do this for you. It’s called Binary Builder and it’s available on Github. This script currently supports Ubuntu, OSX, OpenSUSE, and Fedora. Someday I’ll figure out why Debian tanks. Anyways, here’s how to get BB working so that it creates a development environment for building VW and ASP against ISIS.

cd ~/projects
git clone https://github.com/NeoGeographyToolkit/BinaryBuilder.git
cd BinaryBuilder
./build.py --dev-env --coreutils ~/packages/coreutils-8.16/build/bin

To those of you comparing that command to my previous article, you’ll see that we have used one new argument. The ‘coreutils’ option tells BB where to find GNU versions of cp, echo, mv, and the like.

'All Done!' screen from Binary Builder

When you see an ‘All done!’, you’ll then have a completed build from Binary Builder. It’s good to package this build up so that it can be reused later.

./make-dist.py --include all --set-name BaseSystem last-completed-run/install

This tarball will house all the headers, libraries, and a copy of ISIS that you need to proceed. You can also share this tarball with our users who have similar systems. Now it’s time to deploy this BaseSystem tarball into a position where your copy of Vision Workbench and Stereo Pipeline can use it.

mkdir ~/projects/base_system
./deploy-base.py BaseSystem-*.tar.gz ~/projects/base_system

Installing Vision Workbench

You’re ready for step 2. This is pretty straight forward. You should notice that the deploy-base script produced config.options for both Vision Workbench and Stereo Pipeline. A config.options script is just another way to feed the arguments to ./configure.  When we install Vision Workbench, the base options in config.options.vw should be fine for us.

cd ~/projects
git clone https://github.com/visionworkbench/visionworkbench.git
cd visionworkbench
cp ~/projects/base_system/config.options.vw config.options
./autogen && ./configure
make -j <N Processors>
make install
make check -j <N Processors>

All unit tests should pass at this point.

Installing Ames Stereo Pipeline

cd ~/projects
git clone https://github.com/NeoGeographyToolkit/StereoPipeline.git
cd StereoPipeline
cp ~/projects/base_system/config.options.asp config.options

We’re going to take a moment to deviate here. At this point you will need to make some modifications to your copy of ‘config.options’ for Ames Stereo Pipeline. You need to modify the variable ‘VW’ to be equal to the install (prefix) path that you used. In this example, it should be set to ‘~/projects/visionworkbench/build’. You can also take this time to flip other bits that you find interesting. For example, there’s a ENABLE_MODULE_CONTROLNETTK variable that you can set equal to ‘yes’ which would enable prototype utilities to manipulate control networks. Once you’re done playing around, finish your build of ASP.

cd ~/projects/StereoPipeline
./autogen && ./configure
make -j <N processors>
make install

You can also run ‘make check’, you just need to have your ISIS3DATA set up. You can fall back to your own install of ISIS and everything should work fine. If it wasn’t clear before, you’ll find the executables in “~/projects/visionworkbench/build/bin” and “~/projects/StereoPipeline/build/bin”.

Ames Stereo Pipeline and Earth

I’ve been working on Ames Stereo Pipeline for about 4 years now. During that time the developers and I have always been focusing on what we could do with data that has come from NASA satellites. So we’ve produce 3D models of the Moon, Mars, and of the moons of Saturn. Recent work however has allowed us to produce 3D models of data much closer to home.

Earth example from ASP

In the next ASP release, I’m happy to announce that we will be providing limited support for processing images from Digital Globe satellites. Digital Globe sells images of Earth via World View and Quick Bird satellites. Their imagery is commonly seen in the base layer of Google Earth. ASP was recently awarded a grant to add support for those images to allow mapping of Earth’s polar regions.

Just last week I was able to produce ASP’s first 3D model of Earth using World View data. The image is a screenshot of the colormap DEM output draped in Google Earth. The area is just west of Denver in the Rockies and the imagery represented a huge challenge for ASP. There were lots of trees that make template correlation difficult. The mountains also made for a massive search range, which allowed outliers to make their way into the final result. Even though, we were still able to produce a result. I’m very proud of this result as it leaves me with confidence that ASP will do well in the polar regions where the terrain is not as aggressive.

Building Ames Stereo Pipeline against ISIS on Ubuntu 10.04

This is a guide for advanced bearded users. If you don’t have a beard, don’t try this at home! These instructions will also work for OSX minus the package manager.

Ames Stereo Pipeline is an open source collection of tools for building 3D models from NASA’s planetary satellites. Our software is able to do this by depending on USGS’s ISIS for all the camera models. That saves me a lot of time because I no longer have to program up custom models for all the many cameras that are out there (MOC, HiRISE, MDIS, LROC, ISS, and etc ). The downside is that building ISIS is next to impossible as they expect you to use their binary releases. This means that to compile against their binaries, we must recreate their development environment, down to every third party library.

There are some base tools that you need installed on your Ubuntu Box.

sudo apt-get install g++ gfortran libtool autoconf   \
   automake ccache git-core subversion cmake chrpath \
   xserver-xorg-dev xorg-dev libx11-dev libgl1-mesa-dev \
   libglu1-mesa-dev libglut3-dev

Building an ISIS environment is incredibly difficult to do by hand. Never-mind the difficulty in sanitizing the bash shell so that it doesn’t find libraries in odd places. So a co-worker of mine created an awesome collection of scripts to make this easier. It’s called Binary Builder and it’s available on Github. The commands below checkout the scripts from Github and then run them. What BB is doing in this first step is downloading and compiling all of the dependencies of Vision Workbench and Ames Stereo Pipeline. This means we’re building Boost, GDAL, Zip, OpenSceneGraph, LAPACK, and many others. As you can imagine, this step takes a long time.

cd ~; mkdir projects; cd projects
git clone https://github.com/NeoGeographyToolkit/BinaryBuilder.git
cd BinaryBuilder
./build.py --dev-env

Most likely things will die at this point. Here is where your bearded powers are to be applied! Good luck. When you fix the bug or think you’ve worked it out. You should use the following command to quickly restart.

./build.py --dev-env --resume

You’ll know you’ve had a completely successful ./build.py session when it prints “All done!” and gives you a list of the environmental variables used. Next, let’s clean up by making a BaseSystem tarball.

./make-dist.py --include all --set-name BaseSystem last-completed-run/install

This tarball will house all the headers, libraries, and a copy of ISIS that you need to proceed. It will be your lighthouse when everything else fails. You can also share this tarball with other users who have similar systems. Anyways, it’s time to deploy this BaseSystem tarball into a permanent position.

mkdir ~/projects/base_system
./deploy-base.py BaseSystem-*.tar.gz ~/projects/base_system

Installing Vision Workbench

You’re ready for step 2. This is all pretty straight forward. However you should notice that the deploy-base script produced config.options for both Vision Workbench and Stereo Pipeline. A config.options script is just another way to feed the arguments to ./configure.  When we install Vision Workbench, the base options in config.options.vw should be fine for us.

cd ~/projects
git clone https://github.com/visionworkbench/visionworkbench.git
cd visionworkbench
cp ~/projects/base_system/config.options.vw config.options
./autogen && ./configure
make -j <N Processors>
make install
make check -j <N Processors>

All unit tests should pass at this point. If not, bearded you knows what to do.

Installing Ames Stereo Pipeline

cd ~/projects
git clone https://github.com/NeoGeographyToolkit/StereoPipeline.git
cd StereoPipeline
cp ~/projects/base_system/config.options.asp config.options

We’re going to take a moment to deviate here. At this point you will need to make some modifications to your copy of ‘config.options’ for Ames Stereo Pipeline. You need to modify the variable ‘VW’ to be equal to the install (prefix) path that you used. In this example, it should be set to ‘~/projects/visionworkbench/build’. You can also take this time to flip other bits that you find interesting. For example, there’s a ENABLE_MODULE_CONTROLNETTK variable that you can set equal to ‘yes’ which would enable prototype utilities to manipulate control networks. Once you’re done playing around, finish your build of ASP.

cd ~/projects/StereoPipeline
./autogen && ./configure
make -j <N processors>
make install

You can also run ‘make check’, you just need to have your ISIS3DATA set up. You can fall back to your own install of ISIS and everything should work fine. If it wasn’t clear before, you’ll find the executables in “~/projects/visionworkbench/build/bin” and “~/projects/StereoPipeline/build/bin”. That’s all folks, I hope everything worked out okay.

Finished 3D from Apollo

Render of a DIM and DEM map from Apollo Metric Images It’s been a long 3 years in the making, but today I can finally say that I have finished my 3D reconstruction from the Apollo Metric cameras. After ten of thousands of CPU hours and several hundreds of liters soda, the Mapmakers at the Intelligent Robotics Group have managed to produce an Image mosaic and Digital Elevation Map. The final data products are going up on LMMP’s website for scientists to use. I encourage everyone else to instead take a look at the following KML link I’ve provided below.

IRG’s Apollo Metric/Mapping Mosaic

It’s so pretty! But don’t be sad! IRG’s adventure with Apollo images doesn’t end here. Next year we’ll be working on a new and fancier Image Mosaic called an Albedo Map. Immediately after that, our group will be working with the folks at USGS’s Astrogeology Science Center to include more images into the Apollo mosaic. In that project we’ll include the images that are not only looking straight down on the Moon, but also the images that look off into the horizon.

All of the above work was produced using our open source libraries Vision Workbench and Ames Stereo Pipeline. Check them out if you find yourself producing 3D models of terrain. At the very least, our open source license allows you to look under the hood and see how we did things so that you may improve upon them!

Apollo 15 Mosaic Completed

Image of the Moon with images from Apollo 15 projected onto it.Let me secretly show you a cool project I just finished. During the later missions of Apollo during the 70’s, NASA came to understand that their funding would be cut back. In attempt to extract as much science as possible from the last few missions to the Moon they increased the astronauts’ time on the surface, gave them a car, and added a science bay to the orbiting spacecraft. Inside that science bay (called SIM) were two repurposed spy cameras. One was the Apollo Metric Camera, whose 1400 images from Apollo 15 are seen projected above. Recently ASU has been digitally scanning this imagery. This has allowed me and my colleagues to be able to create a 3D model of a large section of the near side of the Moon and to create a beautifully stitched mosaic.

3D model of Tsiolkovsky CraterBesides these being pretty pictures, I’m proud to say that all of this work was created by open source software that NASA has produced and that is also currently available on GitHub. Vision Workbench and Stereo Pipeline are the two projects that have made this all possible. The process is computationally expensive and is not recreate-able at home, but a university or company with access to a cluster could easily recreate our results. So what does the process look like?

  1. Collect Imagery and Interest Points (using ipfind and ipmatch).
  2. Perform Bundle Adjustment to solve for correct location of cameras (using isis_adjust).
  3. Create 3D models from stereo pairs using correct camera models (with stereo).
  4. Create terrain-rectified imagery from original images (with orthoproject).
  5. Mosaic imagery and solve for exposure times (using PhotometryTK).
  6. Export imagery into tiles or KML (with plate2dem or image2qtree).

This long process above is not entirely documented yet and some tools have not yet been released in the binary version of Stereo Pipeline. Still, for the ambitious the tools are already there. Better yet, we’ll keep working on those tools to improve them as IRG is chock-full of ideas for new algorithms and places to apply these tools.