Making Sense of Compression

Understanding the three common types of raster image compression

b07e2fc4e76a8b6328e5a3a4644fb265

Editor’s Note: Robert Parker is the product manager at LizardTech in Seattle. He has more than a decade of experience working with geospatial imagery and image compression and has demonstrated and presented on related subjects, including a technical walkthrough of the MrSID wavelet compression algorithm.

The file sizes of raster images captured by satellite, aircraft, and UAVs are massive and getting larger all the time. Large digital files can slow image processing workflows, increase archiving costs, and jam wireless delivery networks. These problems will only be magnified as additional imaging platforms with more advanced sensors come online.

This file size issue is particularly acute in the GEOINT Community, where timeliness is mission critical. Raster images must often be processed quickly and transmitted rapidly to decision-makers and warfighters in remote locations. Increasingly, these data sets are being received over low bandwidth networks and viewed on mobile devices with limited memory capacities.

What makes these activities possible today, however, is data compression. Fortunately for image data users in the GEOINT and broader geospatial communities, digital data compression has improved significantly in recent years. Los Alamos National Laboratory took a giant leap forward in the early ’90s researching wavelet compression for digital fingerprint files, a process ultimately applied to raster image compression.

Today, wavelet is the most commonly used compression technology in geospatial applications, and is the foundation upon which several popular commercial compression formats are based. Wavelet compression organizes raster images in multiple levels so they can be selectively decoded, making it possible to pan around a large image and zoom into a small part of it to view at full resolution. This also enables extremely fast decompression and the ability to compress files at a very high ratio in what is known as ‘visually lossless’ compression.

Visually lossless is one of the three basic types of compression in addition to lossy and lossless. Some compression technologies can perform all three types, and it is essential for geospatial data users in the GEOINT Community to understand the differences among the three because the type of compression effects how compressed raster images can be used.

Data Use Dictates Compression Level

Raster data compression will remain vital to the geospatial user community as files grow with enhanced sensor capabilities. File size increases in direct correlation to spatial resolution, spectral band coverage, and dynamic range of the digital imaging sensor. An improvement in any one of these capabilities causes raster files to swell. For example, a raster image acquired at 0.5-meter resolution is four times larger than a one-meter image.

Raster images are made up of pixels, which are numbers measuring the intensity of reflected electromagnetic radiation in specific wavelengths, or spectra. One image can contain millions of pixels. Compression techniques use a variety of methods to remove redundant and excess information—similar to omitting vowels in abbreviations or using symbols to replace common phrases in short hand.

Although based on a simple premise, this compression process becomes exceedingly complex, and the end result is a file that is smaller than the original. The compression is usually expressed as a ratio. A 20:1 compression technique would therefore shrink a 100-megabyte raster file to just five megabytes—a size much easier to store, process, and share than the original.

But not all compression types are equal, and their differences are of importance to people who rely on the information contained in the images to make decisions. The following outlines the three common types of compression:

  • Lossy compression shrinks raster files by altering some pixel values. The upside is lossy can compress data to extremely small sizes—a 40:1 ratio, for example. But one downside is the loss of data forever. The complete spectral information is never recovered, even when the file is decompressed. This means a lossy file can never be digitally exploited and interpreted with the same degree of confidence found in the original image. This does not mean lossy compression is useless. In fact, most compressed imagery is in a lossy JPEG format (typically providing a 10:1 compression ratio).
  • Visually Lossless compression is a subset of lossy compression. This technique also alters original pixel values, often by averaging adjacent pixels, but the process is applied selectively to maintain the visual appearance of the image. Placed next to each other, the original and its visually lossless twin appear identical to the naked eye. Its value for visual interpretation has been retained. But as with lossy, the visually lossless scene has permanently lost some pixel information and its value in digital interpretation has been compromised. A typical visually lossless compression ratio is 20:1. For images that require 1:1 visualization but do not require complex spectral analysis, visually lossless compression is the preferred method.
  • Lossless compression strikes a balance between file size and information content. Although this technique typically shrinks a raster file to only about half its original size (2:1 ratio), all information content of the spectral values is retained and available for digital exploitation. For images that will undergo computerized change detection analysis or spectral classification after compression, lossless is the preferred technique.

Future of Compression

Not only are raster files growing along with sensor advancements, the total volume of data also multiplies with the launch of each new imaging satellite and aerial sensor. Developers of compression solutions are focused on making the processes even faster with less data loss and smaller generated files. In addition, compression is moving up the workflow closer to the sensor so image files will be compressed as they are collected or shortly thereafter right on the collection platform.

Photo Credit: LizardTech

Posted in: Contributed   Tagged in: Data

“Geospatial” as a Noun

Taking the next step in defining the community

Do We Need A “Civilian ARPA” for AI?

The case for leveraging artificial intelligence to improve public service

The Necessity of GPUs

Resolutions have surpassed Moore's Law: What does that mean for data analysis?