One NASA data scientist’s take on human-machine teaming
Some tasks are too tough for either humans or machines to handle on their own. Jeffrey R. Smith, a data scientist with the SETI Institute at NASA’s Ames Research Center in Moffett Field, Calif., brought a particularly complex task to the attention of GEOINT Foreword attendees Sunday afternoon.
“It has to do with this tiny problem we have of finding planets 600 trillion miles away from the Earth. It’s actually more difficult than it sounds,” Smith joked.
After centuries of Earthbound struggle to locate exoplanets, a series of orbiting observatories have discovered thousands over the last decade or so. These include not just gas giants that probably can’t support life but also approximately Earth-sized worlds at distances from their stars that could allow water to exist—a key precursor to life.
“There are Earthlike planets everywhere,” Smith said. “There are millions in the galaxy.”
Smith focused on one of these specialized satellites: TESS, the Transiting Exoplanet Survey Satellite, launched in April 2018. Like NASA’s earlier Kepler probe, it locates exoplanets by searching for the fluctuations they cause in the light of their stars as they pass in front of them.
Identifying legitimate signals of planetary transits from all the possible errata makes for some tricky work.
“We’re searching for really small signals here,” Smith said.
Everything from the spacecraft wiggling in orbit to distant stars that oscillate and pulsate on their own can introduce noise into images captured by these orbiting observatories.
“All of this adds up to systematic issues that are sometimes thousand of times larger than the actual signals,” Smith said.
To surface the real signals, researchers have built a variety of deep-learning models—Smith refrained from describing them, instead offering variations of, “I’ll just say it’s magic”—to yield a smaller set of potential planetary finds for human evaluation.
As he put it: “The pipeline doesn’t spit out planets, it spits out things that might possibly be planets, these planet candidates, and a human has to look at these and figure out or guess, is this really a planet or not?”
To demonstrate the degree of difficulty, Smith showed a series of TESS imagery and asked the audience if each represented the signature of a planet’s transit. After a couple of reasonably obvious examples—one a large planet, another an imaging artifact—the examples became more ambiguous.
“This is just chance correlations of random noise,” Smith said after showing an image that looked to the uneducated eye as if it could have been yet another sign of an Earthlike world. “We’re searching out signals that are just above the noise floor.”
He closed his presentation by emphasizing the importance of finding the right role for machine learning versus human insight—as in, one that “frees the humans to concentrate on the interesting things.”
In the case of exoplanet sleuthing, that role came down to culling out the clutter, so humans don’t spend as much time on image triage.
“Whatever you can do to eliminate false positives early on in your pipeline, the more efficient you’re doing to be,” Smith said.
But he suggested that a purely biological or completely digital approach will never work for complex problems such as this.
“It really takes the human to decide where do you put the machine, how do you wire up the machine, and to what degree do you trust the machine,” he concluded. “And for that matter, what degree do you trust the human—because humans have biases and intuitions.”
NGA’s procurement leaders outline strategies to meet new strategic objective