ISIS 3.4.0

Congrats Flagstaff on the new release! Now excuse me while I swallow my tongue. (I don’t enjoy building ASP binaries.) After I regain my composure, I’ll be happy to check out some of the sweet new features available in this release. I pretty excited for:

Fly by night Cassini ISS processing

Anytime I want to process Cassini ISS data, I always run into problems. This is largely because the documentation is lacking. Today is the day that is going to change. Here are my notes on how to extract ISS data of Iapetus, all for the low cost of free. I assume you have USGS’s ISIS3 installed.

Iapetus, file N1483152391_1

Downloading from NASA PDS

As usual, all NASA data can be pulled free from their PDS (Planetary Data System). Since I’m looking for Iapetus, I click ‘Saturn’ and then ‘Cassini Image Search’. This will bring you to the Cassini Image Quick Search page. On the left side of the page, select the ISS instrument. Then select EDR under product type. EDR products are raw images while RDR are processed products like maps. Finally under target name, select the option Iapetus. Click ‘Get Results’ on the far right. At the time of writing, there was 4025 records, or images available.

Here’s where things get a little messy. A lot of these images are of Iapetus from very far away. So far away that they wouldn’t be good for my projects. I’d like to sort these images by their distance. On the far right, under SELECT PARAMETERS FOR REPORT OR TABLE COLUMNS; I can add the column ‘Target Distance’. On the new column, I can click the up arrow and hopefully it will sort the images by who is closest. However it doesn’t work. In the middle of the page; PDS says “508 of 4025” products. The page is only sorting 1/8th of the total images of Iapetus. To correct this, keep clicking the ‘Get More’ button until it says “4025 of 4025” products.

At this point we are ready to start downloading products. On the far right of the page, download the CSV file of the records they found along with the WGET product. When you open up the CSV file you’ll notice that they didn’t keep the images ordered in distance from target. Lame, I know. However, we use Linux and we can pull ourselves out of this. Here’s the bash commands I used to sort the file. Gurus can probably do a better job of this than me.

cat atlas_report.csv | awk -F "," '{print $3 $4}' | sort -n -k 2 > sorted_files

If you open up sorted_files in your favorite text editor, you can clean up the stray lines that were mis-sorted. I chose to keep the files with distances marked ‘-1.0e3’. I also hand deleted the lines where the distances were marked greater than 300 km out from the moon. This left me with 758 names of images that are pretty close to Iapetus. Now we need to pull these 758 names from the atlas_wget_script so that I only download the images I want. However when you look in that script, you’ll notice that they name the images slightly different from what we saw in the atlas_report.csv. Here’s an example:

1_N1466586085.118 versus N1466586085_1.IMG

Here’s how I filtered sorted_files down to only that mantissa like bit I wanted and then pulled out the respected lines I wanted out of atlas_wget_script.

cat sorted_files | awk '{print substr($1,3,11)}' > wanted_keywords
cat atlas_wget_script | grep -f wanted_keywords > wanted_get_script

Hooray! Finally we are ready to really download imagery files! If you look inside wanted_get_script, you’ll see the download commands for all the images of Iapetus that I want. To start downloading:

source wanted_get_script

Processing with ISIS 3

You probably think it’s all kisses and hugs from here. You have a direct full of IMG and LBL_label files do ya? Well the command we want to use is ciss2isis. Well then, don’t let me hold you back, read their instructions and give it shot on one of the images you downloaded.

ciss2isis from=N1483150900_1.LBL_label to=test.cub

What it broke?

**I/O ERROR** Unable to read PVL file [/Volumes/Data/Saturn/CASSINI_data/Iapetus/temp/N1483150900_1.LBL_label]
**PVL ERROR** Error in pvl on line [166]
**PVL ERROR** Unable to read keyword [DESCRIPTION    = "For the packet from which this telemetry header was Extracted:  the first line of this packet is a continuation of a line begun in the previous packet. OBJECT = COLUMN NAME = NULL_PADDING DATA_TYPE = MSB_UNSIGNED_INTEGER START_BYTE = 61 BYTES = 475 END_OBJECT = COLUMN END_OBJECT = TELEMETRY_TABLE OBJECT = LINE_PREFIX_TABLE INTERCHANGE_FORMAT = BINARY ROWS = 256 COLUMNS = 7 ROW_BYTES = 24 ROW_SUFFIX_BYTES = 512 OBJECT             = COLUMN NAME               = LINE_NUMBER DATA_TYPE          = LSB_UNSIGNED_INTEGER START_BYTE         = 1 BYTES              = 2 DESCRIPTION        = "The image line number of this record.  Maintained at]
**PVL ERROR** Keyword has extraneous data [The image line number of this record.  Maintained at] at the end

This looks darn scary. However if you download the original LBL files from the PDS manually without using the atlas_wget_scripts ‘s version, the problem becomes clear. The download script was accessing a new label file that had a poorly inserted ‘EXTENDED_ISS_SCIENCE_HEADER’. This was probably done by some batch processing job at PDS or Cassini imaging lab. This mistake on PDS managed to delete some of the lines defining the end of objects. This caused the ISIS3 command ciss2isis to freak out an die.

Yet don’t worry .. this can be corrected. I have written a patch for these LBL_label files that will make then usable. In a new file, write the following contents.

--- N1483150900_1.LBL_label     2010-10-19 19:46:20.000000000 -0700
+++ N1483150900_1.LBL_corr      2010-10-19 19:45:01.000000000 -0700
@@ -99,6 +99,7 @@
START_BYTE     = 1
BYTES          = 60

NAME           = CAMERA
@@ -144,7 +145,8 @@
BITS           = 1
DESCRIPTION    = "For the packet from which this telemetry header was
Extracted:  the first line of this packet is a
-                    continuation of a line begun in the previous packet.
+                    continuation of a line begun in the previous packet."
@@ -209,6 +211,8 @@
START_BYTE         = 9
BYTES              = 2
      LINES = 256

I saved the above as correction.patch. It can then be applied to all LBL_label files to make then work.

echo *label | xargs -n1 echo | xargs -n1 -I{} patch {} correction.patch

You will now find that ciss2isis works. Here’s how I do that with multiprocessors.

echo *label | xargs -n1 echo | awk -F "." '{print $1}' | xargs -n1 -I{} -P8 ciss2isis from={}.LBL_label to={}.cub

You now have a directory with cub files. You can view those with the qview command or export them with isis2std. There’s still more that needs to be done inorder clean up this imagery. I won’t tell you today, but I can hint what the next few steps will be. Have fun!

Iapetus plus stars