Examining the “why” of GEOINT using the Data/Frame Model of Sensemaking
This article addresses the underlying human factors (HF) of geospatial intelligence (GEOINT) by examining the “why” of GEOINT using the Data/Frame Model of Sensemaking. The article is based upon recent HF research we conducted about the fundamental human factors of GEOINT (Hoffman, In Press).
HF engineering studies interactions between humans and technology to improve overall human-system performance. Formally, the International Ergonomics Association defines human factors as “… the scientific discipline concerned with the understanding of interactions among humans and other elements of a system … in order to optimize human well-being and overall system performance.” The definition we use applies the concept of human factors beyond the design of systems to the study and design of ways to improve “cognitive work.” To do this, we examine the cognitive basis of GEOINT by considering why the analyst completes tasks.
As we are coming to appreciate, the GEOINT Community is overwhelmed with data. The thousands of new small satellites projected in the next few years and the Internet of Things (IoT) are but a few reasons for the data avalanche. This creates a particular problem for the analyst. The collection of geospatial information is the most easily and often automated component of the intelligence cycle. Thus, while the data glut is problematic for intelligence and any big data analytics, some of the most extreme problems exist in the GEOINT arena. The prospect is the data glut will grow and increase analyst uncertainty since systems are now collecting megabytes of data for each human on the earth each minute of the day.
GEOINT’s Cognitive Work
Our research investigates the challenges of GEOINT analysis through the lens of human cognition, focusing on how operators think rather than what specific tasks they perform. By looking at operator goals within the context of a human sensemaking model, we decompose the analytic problem space in a way consistent with human cognition, an important consideration for improving instructional methods, analytic tools, and gaining an advantage in analytic decision-making. This can form the basis for understanding how tools and automation can better integrate with the analyst. In considering how automation can support or extend the GEOINT analyst’s goals, we can create tools that work more effectively with their human counterparts, building a more effective human-systems team.
The human is ultimately where knowledge work is done and insights are produced in intelligence analysis, thus geospatial intelligence is dependent on the geospatial analyst’s know-how. Though often confused for a workflow, the analyst’s cognitive actions are not a sequence of steps through which work passes; however, cognitive activities may be associated with parts of a workflow. The cognitive actions of analysts can generally be classified as “sensemaking,” a concerted cognitive effort to understand the relationships among disparate objects and events to place them within a context or frame that has explanatory power.
Geospatial analysis, sensemaking with geography, begins with the conceptualization of space-time to frame the problem. The nature of the frame is critical because it ultimately determines the interpretation of the analyst’s observations—how they make sense of the geospatial relationships. These are generally considered within the three geographic frames: physical spaces, behavioral spaces, and cognitive spaces. The frame provides the interpretive context that gives meaning to the geospatial data. The three geospatial frames are described as follows:
- Physical space is built on the four-dimensional world of space-time, but focuses on the physical.
- Behavioral space is the four-dimensional space-time that focuses on the spatial relations and interactions between individual actors and objects in the physical environment.
- Cognitive space focuses on concepts and objects that are not themselves necessarily spatial, but the nature of the space is defined by the particular problem. In the geospatial sciences, this is called the “mental map” that exists in the analyst’s mind.
The sensemaking process teaches us that machines automate processes that increase the efficiency in the human performance of sensemaking using the three geospatial frames, and, that at its most basic core component, the data, systems, and humans work in concert. As such, this requires an understanding of the synergy between the human cognitive thought processes involved and the technical systems used.
- This article is part of USGIF’s 2018 State & Future of GEOINT Report. Download the PDF to view the report in its entirety and to read this article with citations.
Cognitive research on intelligence analysis has highlighted that analysts engage in an iterative sensemaking process that involves considering the data available from different viewpoints and perspectives. Sensemaking is a human cognitive function that has been studied and is not to be confused with specific structured analytic techniques. Sensemaking has been defined as “… a motivated, continuous effort to understand connections (which can be among people, places, and events) in order to anticipate their trajectories and act effectively.” Moreover, Gary Klein, et al., posit a generic Data/Frame Theory of Sensemaking, which suggests analysts collect sufficient data to establish an initial frame (or mental map) for making sense of the data. The data used to create the frame are “the interpreted signals of events” and frames are “the explanatory structures that account for data.” This frame can be considered an organizing entity—a specific instance of a mental model of the elements and relationships under analysis.
We conducted a cognitive task analysis using an abbreviated, goal-directed task analysis methodology. The output of the cognitive task analysis is a goal hierarchy describing the analyst’s goals and subgoals as he or she seeks to understand the situation, which creates a framework for understanding the cognitive work of the geospatial analyst. GEOINT analysts may work with any branch of the department of defense, intelligence community, or law enforcement, for example. Though the problems they analyze may vary across domains, at some level of abstraction, the goals and decisions, and, of course, the analytic methods are common. We chose to focus on the commonalities across domains rather than the differences.
The “Why” of GEOINT
Research conducted during the summer/fall 2017 for the preparation of a book chapter on human factors indicated the overall goal for a GEOINT analyst is to use geospatial analysis methods to “find truth” in response to requests for information. The goal of finding truth is not always achievable, particularly in any area of intelligence analysis, where there is often an opponent seeking to hide that truth from view. Finding truth is still the high-level goal, despite the difficulty in attainment. At its core, finding truth is the goal of all sensemaking, though the professional domain operational reality may be that getting close enough to truth to defeat the opponent is sufficient. Sensemaking is the cognitive process of iteratively fitting data to a frame, and fitting a frame to the data. The data-frame model of sensemaking provides a working description of how the geospatial analyst performs at the most rudimentary level. The model describes how people construct and revise internal mental structures when they make sense of events, and the goal of this sensemaking process is to find truth by selecting the right frame to interpret the data.
The primary subgoal identified during the 2017 research project cited above was to complete intelligence tasking in accordance with schedule and priority. This involves finding the answer to a GEOINT question, and the analyst will incorporate the necessary constraints for schedule and priority within their tasking requirements. This indicates that while the goal is to complete all tasking on schedule, higher priority tasks may bump lower priority tasks. Again, sensemaking is always done for a purpose, and this goal reflects the reality that GEOINT professionals are often tasked with multiple requests, and balancing schedule and priority across multiple competing demands requires effort. To achieve this higher-level goal, the analyst also has three subgoals to consider: evaluate geospatial collection requirements; evaluate the best sources for needed GEOINT data; and analyze how and why locations of objects of interest change over time. The analyst determines the requirements of the collection tasking and the best opportunities and options for collecting the data, and will begin the process of geospatial analysis by investigating the geospatial relationships among the objects of interest and looking at activity over time.
Next, analysts strive to identify information needed to respond to requests for GEOINT information. This is analogous to the “representing the situation” facet of sensemaking, the first facet in which the sensemaker tries to pull relevant data from the stream of information, while discarding data deemed not applicable to the current problem. The related subgoals are: identify relevant available GEOINT data; evaluate available GEOINT data for quality, timeliness, and applicability; identify gaps in current data; and request information to fill gaps in the data. To achieve these goals and effectively represent the situation, the analyst will determine what information is available, analyze the quality of that data, identify critical gaps in information needed to meet the tasking, and then determine how to fill those data gaps.
Our research result goal hierarchy next shows that analysts have a subgoal to analyze the available data using GEOINT methodologies. This includes considering and presenting the geospatial and temporal data relationships then trying to extract meaning from these relationships. In the sensemaking model, this is when the analyst examines the relationships in the data to find an initial frame that fits the relationships then continues iterations via elaborating and questioning the frame. Data that has been discarded may be considered again, as the analyst questions the frame to determine whether another frame can better explain the geospatial relationships among the data. As a subgoal, the analyst will define and visualize data relationships, including analyzing the spatial, temporal, and multidimensional aspects of the data. This analysis of relationships assists the analyst in understanding the data to select a frame.
Through a process of pattern matching to domain knowledge familiar frames, the analyst selects a frame, then questions whether the frame really fits the current data. This may involve seeking additional information, or seeking to better understand how the elements are related, as the analyst elaborates the frame and documents the current frame to preserve it. The analyst has a goal to build and test hypotheses to identify “truth.” During the process of elaborating and questioning the frame (testing), the analyst will apply a variety of domain knowledge obtained models (which are often maps), theorems, and axioms to search for patterns and meaning (frames) in the data. In particular, GEOINT analysts seek to gain a visual sense of the data. The analyst strives for reliability and precision in the data as even small variances can affect the perceived relationships among data points. In sensemaking, this corresponds to questioning the frame and reframing when a different frame better explains the relationships among the data. In addition, the skilled analyst considers other possible frames using a method such as the analysis of competing hypotheses (ACH), in which he or she attempts to either validate or nullify the current frame. Depending on the outcome, the analyst may select another frame (reframing) that better explains the geospatial relationships and activity that has been identified.
To complete the intelligence tasking, the analyst must synthesize and disseminate information into comprehensive, organized, prioritized intelligence products. This requires that the analyst is satisfied, for the moment, with the selected sensemaking frame, and is willing to present the final data within a frame that offers explanatory value as an analytic work product. The information is integrated within the final frame, and storytelling is used to create a product that explains the data in a way that targets the needs of the report’s consumers. The work product will include an assessment of the quality and error precision of the data and the related analysis, and will be shared in accordance with their domain-specific quality guidance. It is important to note GEOINT analysts often develop expertise in a specific geographic area, and the delivery of a report is never the end of the analysis. In fact, if additional information is uncovered that causes a shift in the frame, or adds additional explanatory value to the existing frame, the analyst will revise the report to capture the additional data.
In addition, the analyst has a goal to collaborate with others in a team environment. Collaboration is increasingly common within the GEOINT Community, with multiple analysts working together to provide an integrated product that is more complete than any one expert analyst can provide and spans multiple analytic specialties. Collaboration in intelligence extends beyond human to human. As discussed earlier, technology is prevalent within GEOINT, and the GEOINT analyst uses a number of tools to better perform the analysis tasks. Many of these are of benefit, and have extended geospatial analyst capabilities tremendously such as with machine-aided object recognition within imagery. However, as tools become more automated, the challenge is to maintain the analyst’s ability to determine the accuracy and reliability of the data provided. As systems become more autonomous, they become more independent entities that can contribute more directly to team analysis and success—but they can also contribute to breakdowns in communication and may lead to longer decision cycles, rather than shorter.
This article identified the challenges of geospatial intelligence analysis through the lens of human cognition, focusing on how operators think, rather than the computer tasks performed. By looking at the operator’s cognitive actions, we can improve instructional methods, analytic tools, and gain an advantage in analytic decision-making. Perhaps most importantly, this helps develop automated tools that can better integrate with the analyst. In considering how automation can support or extend the GEOINT analyst’s goals, we open the door to machines working more effectively with their human counterparts and building a more effective human-systems team.
The human “why” of GEOINT is to compare observed patterns of details to better understand them. Our research identified HF concepts and methods of GEOINT analysis with the potential to significantly improve the overall “why” of the human-systems team. Something makes sense because the analyst has seen a similar pattern, and the similarities between the two patterns help the analyst make non-obvious inferences and draw conclusions. GEOINT analysis can be said to be the process of fitting geospatial data into a frame, and fitting a frame around the geospatial data. GEOINT insights are developed within a system of humans and machines, in which machines play a role in selecting and preparing information for the analyst’s consumption. The geospatial analyst’s frames are mental maps that account for the data. We suggest a community discussion that adds human and machine refinements and domain knowledge to an analyst’s framing and testing skill set to increase analysts’ ability to explain “why.”
The roles of data collection, storage, management, analysis, and consumption as factors that drive designs