Harnessing innovation to enable new applications
By Rakesh Malhotra, Ph.D., North Carolina Central University; Gordana Vlahovic, Ph.D., North Carolina Central University; Karen Schuckman, Pennsylvania State University; Cordula Robinson, Ph.D., Northeastern University; Camelia Kantor, Ph.D., the United States Geospatial Intelligence Foundation; Timothy Walton, Ph.D., James Madison University; Timothy Mulrooney, Ph.D., North Carolina Central University; James Rineer, P.E., RTI International; Chris McGinn, Ph.D., North Carolina Central University; and Craig Gruber, Ph.D., Northeastern University
We live in a more interconnected and information-rich world than at any other period of human history. The advent and pervasiveness of networked computing has changed how we view and share information and how we gather, store, and process it. These changes are spurred by the miniaturization of equipment and gadgets, the ever-shrinking cost of data gathering, automated data analysis, and increased processing power. The amount of digital storage and computing power that used to fit in a room now fits in your palm, and machine learning and artificial intelligence are affecting how we interact with each other and use information every day.
Geospatial intelligence (GEOINT) is no exception to these changes spurring the next revolution in remote sensing and spatial information gathering. While remote sensing and satellite imagery have always been important, newer entrants such as unmanned aerial vehicles (UAVs) and small satellites enable an array of visualization and analyses tools that were hard to imagine just a decade ago.
Spatial Data and GEOINT
Keyhole—an In-Q-Tel-funded company bought by Google in 2004—launched a revolution that made spatial information available to anyone connected to the internet. Google re-launched Keyhole’s EarthViewer application as Google Earth in 2005 and almost immediately it was adopted by individuals, companies, non-governmental organizations (NGOs), and governments to access spatial information. Suddenly, “eye in the sky” had a new meaning, as anyone could explore any part of the Earth and GEOINT was in everyone’s toolbox.
Google Earth, though revolutionary, had limitations such as scale and temporal lag. The high-resolution spatial information was weighted toward data-rich areas such as the U.S. and Europe, and the underlying satellite data were updated periodically but not frequently nor based on the needs of any particular project or news story. Google Earth is not the first or the last story of a geospatial revolution.
UAVs and small satellites stand to alter data dynamics worldwide, especially in countries and locations with poor data infrastructure. While the UAV revolution will alter data collection on a local, regional, and project scale, small sats will provide almost real-time data at a global scale. The geospatial industry stands to benefit from the miniaturization of hardware and the enhanced processing power of both applications.
- This article is part of USGIF’s 2019 State & Future of GEOINT Report. Download the PDF to view the report in its entirety.
Unmanned Aerial Vehicles
Perhaps the most prevalent example of the miniaturization of geospatial information gathering has been the advent of UAVs, also popularly known as drones. In the past few years, UAVs have become mainstream so quickly that policy-makers, geospatial organizations, and software developers are playing catch up. For example, it was only in 2016 that the U.S. Federal Aviation Administration (FAA) put guidelines in place to clarify the use of UAVs for fun (hobby) and work (professional flying). A few important factors including miniaturization have helped popularize UAVs over the past few years and close calls with UAVs at sensitive locations such as airports forced this action. High-performing UAVs with gimbals, video recording at 60 to 80 frames per second, and flight times ranging from 30 to 45 minutes can be purchased for under $1,000. This is a critical price point for recreational purposes, but also has significant global policy implications as UAVs become affordable to NGOs, non-state actors, and individuals. The other crucial element aiding the proliferation of UAVs in fields such as GEOINT, agriculture, disaster management, and human development is the reduced lag time when compared to the availability of existing data. Satellite data with daily periodic frequency are expensive and capturing aerial photography can be cost-prohibitive. Moreover, non-military UAV projects are flown at an altitude of 400 feet or lower, and data acquisition can occur even under cloudy conditions with multiple repeat cycles per day. For these reasons, UAVs will continue to be the tool of choice for missions where data collection is time critical.
The argument for choosing UAVs over satellite or traditional aerial imagery is particularly compelling when the study area is relatively small, the political climate supportive, and the weather cooperative. The top range for UAVs to capture images and return safely is about 500 acres, but that reduces quickly with an increase in payload and flight path variations. Though UAVs are more susceptible to rain and wind, they are less susceptible to cloud cover. The politics of an area are also important to consider. Will the local population look at a UAV suspiciously and what can be done proactively to mitigate this concern? Projects implementing UAVs should address these issues by coordinating and seeking permission from local authorities, sharing information prior to UAV flights, and involving regional stakeholders in data and attribute collection that enriches imagery datasets with information about local landmarks.
Just as the advent of computers did not render paper obsolete, UAVs will augment data gathering rather than replace existing remote sensing technology. This will lead to even more data than can be physically reviewed by individuals, making future projects reliant on automation and machine learning. Still image analyses are well developed due to the long history of photogrammetry and the implementation of structure from motion (SfM) technologies. Though current automated applications have limited appeal, they will continue to expand. The next few years will see rapid development in UAV full-motion video (FMV) and data analysis that include real-time data availability and mobile applications specifically designed to leverage UAV data. Examples of real-time analysis of video data have been appearing in both intelligence and commercial applications and are only set to accelerate.
The most common sensors offered are still or video, but other sensors such as infrared and LiDAR are becoming increasingly popular. The push to create UAV infrastructure-as-a-service has given rise to platforms that leverage cloud services to combine UAV data capture and image analyses.
A lack of data standards will hamper interconnectivity and the seamless transition of UAV data. Just a decade ago, the development of data standards for LiDAR helped catapult the technology from simple terrain analysis to myriad applications. Data standards not only help with inherent standardization but also create opportunities for new applications. The standardization of UAV data, particularly FMV, is the next logical step in the integration of UAVs with geospatial tools. Geospatial organizations such as the Open Geospatial Consortium and the American Society for Photogrammetry and Remote Sensing are starting to establish standards for the capture, sharing, and analysis of UAV datasets. Standards will be essential for multiple stakeholders and partners to seamlessly coordinate data analysis. As more applications are developed and UAVs are used for additional services, visibility will lead to greater acceptance in the civilian world as with previous military technologies that crossed over.
Just 65 years ago, the then Soviet Union launched Sputnik, sending the U.S. and the USSR into a Space Race. Within a decade, the U.S. Corona program provided proof of concept that imagery gathered from space had vital defense applications,. This technology is at the core of remote sensing as we use it today. For all of the 20th century, collecting images from satellite data was the purview of governments and included programs such as Landsat (U.S.) and SPOT (France). The key reason for this was that building, launching, and managing satellites was expensive. (Landsat 8 cost USD $850 million).
The shift from public to private enterprise in the space industry has accelerated, and the current revolution is ushered in by the aptly named small satellites (also known as nanosatellites or cube sats) that are comparatively smaller. These satellites provide important advantages compared with traditional Earth observation satellites such as the ability to capture data over the same spot on Earth more frequently and even on demand. Another advantage is that low-cost, low orbiting satellites are usually launched as accessory payloads to larger satellites, further reducing project costs and offering cheaper data acquisition than traditional remote sensing satellites.
These rapid changes in satellite imagery data collection create both opportunities and challenges. An obvious opportunity is that consumers have access to a constant stream of high-resolution imagery with reduced lag times. This opens up remote sensing to a host of applications that focus on spatial monitoring and analyses at various scales. The fact that the entire planet is being mapped on a daily basis provides spatial analysts the ability to easily access imagery of the same location from last week, last year, or even further back.
Natural and man-made disasters occur unexpectedly, and this shift from targeted acquisition to daily global mapping can be beneficial in areas that were not of interest pre-disaster but are now the focus of data collection. For example, having daily imagery prior to an avalanche or flood helps with the development of early detection tools for similar events. Such hindsight will help improve protocols as well as enhance the development of forensic remote sensing and the ability to review potential scenarios applied to a vast array of human, social, and environmental events.
Both UAVs and small satellites are extending current applications and creating new ones in a variety of fields such as homeland security, food security, port security, and beyond. Based on the desired application, UAVs can capture oblique or orthorectified images. Humanitarian applications usually require oblique images so information on sign boards and sides of buildings can be captured and used to identify locations not easily identified in orthophotos. UAVs are altering the global film, music, sports, and real estate landscapes by creating videos and still images that were expensive, impossible, or required specialty flights just a few years ago.
The push to gather infrared and near-infrared data is largely driven by agricultural applications, and these sensors are critical to the development of applications for which information about vegetation and water are used as inputs to predictive models. Increasingly, identification of specific crop types and crop health at multiple times during the season is integrated with traditional satellite data. Thus UAV-collected data can classify crops, monitor crop growth variability and disease, estimate biomass, and support site-specific crop management with daily intervention in some cases. Vendors, particularly in the agricultural segment, offer cloud hosting and automated analysis of UAV data.
Both homeland security and port security have benefited from these GEOINT technologies. However, friends and foes of all stripes have adopted these technologies as well, including terrorist organizations such as Hezbollah and nations such as China. Seaports are large, complex areas that play a significant role in national security and the global economy. A single disruption to port operations can harm a nation’s economy and cause worldwide effects on the flow of global trade. Workflows driven by multisource and real-time data can strengthen port security. Real-time data integration, including updates provided by UAVs and small satellites, identify risks and assist with prioritizing and establishing secure evacuation routes. Selected evacuation routes can continue to be surveyed for obstructions, vulnerable infrastructure can be identified and surveyed, and port facility stakeholders can be alerted and updated to deploy necessary courses of action. The overhead, real-time synoptic views afforded by UAVs and small satellites allow for focused monitoring of current situations and the deployment of nimble disaster management strategies.
For humanitarian aid and monitoring, UAVs can conduct an aerial survey of community infrastructure such as schools in and around a study area. Such applications have lower costs than door-to-door surveys. Any study with repeat data collection or in which change detection needs to be recorded can benefit from this technology. In the domain of food security, UAV data can be used in combination with satellite imagery to support communities dependent on agriculture. Other monitoring examples include sprawling slums, crowds at protests or festivals, the ebb and flow of refugee camps, trash accumulation and disposal in urban areas, measuring economic activity at marketplaces.
With spatial information becoming available at various temporal and spatial scales, new tools and applications that integrate and use this information continue to emerge. A similar example of ubiquitous spatial data comes from the early 1990s, when GPS data were made available for civilian use, illustrating how spatial information, when offered in an easily accessible form, creates ubiquitous opportunities.
It is hard to envision that the skies will be filled with UAVs and space congested with small sats. This is unlikely to happen as collision avoidance and platform sharing technologies emerge, and space and the skies are mapped and zoned into lanes and corridors. In the future, visual information will rely on a pyramid directly correlated with the altitude of acquisition. Local and low-altitude UAVs will capture and transmit local information (akin to local broadcasting stations), and satellites will prevail on the regional and national levels.
Most applications on laptops, cellphones, and other devices, including virtual reality wearables, will access and use near-live spatial information. Just like location, vantage viewing will be the norm and expected by users. Such expectations will be pervasive, with users relying on oblique and ortho images to understand and make all kinds of decisions. Visualization from all angles will be as important in the future as communication is today, with input provided regularly from UAVs and small sats.
User expectations will change from “what I see” to “as if I am there” and change how we communicate digitally. Images and videos will be as strong as words and will also have an impact on GEOINT. The “eye in the sky” will become the eyes of everyone interacting on the scene, including warfighters, first responders, rouge actors, and humanitarian agents. The acute irony is the closer everyone is to the action visually, the further most people will be from it actually.
The proliferation of imagery and increased data acquisition will also pose challenges. In the new world of petabytes and zettabytes, imaging and cataloguing the entire planet every day is an enormous task with allied concerns related to ethical use. New algorithms, machine learning, and automated image extraction will continue to define how we use and analyze imagery from various platforms.
Innovation and change are the bedrock of human ingenuity. Whether it was the agricultural, industrial, or medical revolutions, each brought a greater level of prosperity and complexity to our world. Today, we are in the midst of an information revolution for which spatial information plays an integral role. Tomorrow we will be in a “visual revolution” with UAVs and small sats at its forefront. This visual information will be integrated into current and future applications as another piece in the information pie that is continuously ingested and applied for myriad innovations.
- Andrew Foerch. “The Genesis of Google Earth,” Trajectory Magazine, November 1, 2017, http://trajectorymagazine.com/genesis-google-earth/.
- Todd C. Patterson. “Google Earth As a (Not Just) Geography Education Tool,” Journal of Geography, 2007:106(2):145-152.
- T.B. Lefer, M.R. Anderson, A. Fornari, A. Lambert, J. Fletcher, M. Baquero. “Using Google Earth As an Innovative Tool for Community Mapping,” Public Health Reports, 2008:123(4):474-480.
- Reg Austin. Unmanned Aircraft Systems: UAVS Design, Development and Deployment. John Wiley & Sons; 2011.
- Matt Alderton. “Imminent Ubiquity,” Trajectory Magazine, June 5, 2015, http://trajectorymagazine.com/imminent-ubiquity/.
- Paul Dickerson. Sputnik: The Shock of the Century. Walker & Co; 2007.
- Curtis Peebles. The CORONA Project: America’s First Spy Satellites. Naval Institute Press; 1997.
- Kristin Quinn. “The Maturation of SmallSats,” Trajectory Magazine, March 7, 2014. http://trajectorymagazine.com/the-maturation-of-smallsats/.
Headline Image: A Marine with Task Force Southwest prepares to launch the Instant Eye small unmanned aerial system at Camp Lejeune, N.C., Feb. 8, 2017. Due to its compact size, the Instant Eye will allow Marines to capture imagery and conduct reconnaissance in buildings and other confined areas. Photo by U.S. Marine Corps photo by Sgt. Lucas Hopkins.
The GEOINT tradecraft is evolving within the aviation industry through the introduction of unmanned aerial systems (UAS) into the National Airspace System