Smart Stuff

Having co-designed the communication protocol on which the virtual world was built, computer scientist Vint Cerf is best known as one of the “fathers of the internet.” To his friends, however, he’s better known as an oenophile.

Cerf’s passions for wine and technology are not mutually exclusive. Along with more than 1,000 bottles of vintage vino, the wine cellar in his home near Washington, D.C., stores a sophisticated technology suite that tracks in real time the condition of his fine wine collection. It starts with a network of sensors that monitor the light, temperature, and humidity in every room of his house; if the temperature in the wine cellar rises above 60 degrees Fahrenheit, Cerf receives a text message on his smartphone. The system tracks when the lights are turned on, notifying him if someone enters the cellar without his permission. And because every bottle he owns is outfitted with an RFID chip, Cerf can take a quick inventory with his handheld RFID detector.

Cerf’s high-tech wine cellar isn’t just a technophile’s quirky hobby. It’s also a bellwether for the Internet of Things (IoT), which will be the nervous system of the hyper-connected world currently under construction all around us.

“Anything that is programmable that can communicate through the internet is potentially a ‘thing’ in the Internet of Things,” explained Cerf, a former Defense Advanced Research Projects Agency (DARPA)scientist who is now vice president and chief internet evangelist at Google.

Beyond sophisticated wine cellars, the IoT includes picture frames that can automatically download photos from the cloud, thermostats that can be controlled from one’s smartphone, and light bulbs that can be dimmed wirelessly. And that’s just for starters.

“By 2020, we’re going to have 50 billion ‘things’ that are connected to the IoT and performing various tasks in a massively interconnected way, from phones to blenders to printers to cameras,” said University of Montana Research Fellow Dr. Alex Philp, founder and vice president of special programs at Adelos, which specializes in fiber optic sensors. “The breadth of devices associated with the IoT will create a lot of complexity, but also overwhelming possibilities.”

Because each and every IoT device will have a physical and temporal location alongside its virtual footprint, GEOINT will be a key enabler to manage the challenges and harness the opportunities.

The Internet of Things gives us access to the data from millions of devices. But how does it work, and what can we do with all that data? Find out in this animated tutorial from IBM’s Think Academy.

Behind the Buzz

British technologist Kevin Ashton coined the term “Internet of Things” in 1999 to describe the network connecting objects in the physical world to digital infrastructure in the virtual one. The buzzword, however, is actually the convergence of four much older concepts, according to Dr. Thomas Yen, technical director of the Internet of Things Lab at the University of Wisconsin-Madison.

The first, according to Yen, is embedded sensing, whereby microprocessors make everyday objects “smart.” The second is pervasive connectivity, which allows one to be connected to the internet at all times. The third is ubiquitous computing, which makes computing possible across devices. Finally, there’s virtual interfaces, which describes the ability to engage devices without physically touching them.

“These four technologies coming together is what best defines the Internet of Things,” explained Yen.

According to Barry Barlow, chief technology officer at Vencore, “The Internet of Things is about objects that have embedded sensors that are connected so they can either transmit information or receive information, and in some cases act on that information.”

For example, take a connected refrigerator that photographs its contents and uploads the pictures to the cloud, where they can be accessed alongside recipes in a smartphone app so the consumer knows what to buy at the grocery store. “There are Samsung refrigerators on the market right now that provide that service to the owner,” Philp said.

Aggregate, real-time data captured from refrigerator-generated recipes and shopping lists could tell food manufacturers and marketers what consumers are buying, where, and in what quantities, allowing entire industries to turn on a dime.

Along with smart homes, another IoT growth sector will be health care. Devices like connected insulin pumps and pacemakers already help doctors and patients identify and target risk factors for conditions such as diabetes and heart disease.

“Eventually, I think you’ll see nanoscale devices that are put into our bodies intravenously or orally to help us understand our heart rates and rhythms; measure and sample the microbiomes of our mouth, esophagus, and intestines; monitor cholesterol and blood pressure; and look for certain diseases,” said Philp, who added such devices will collect and transmit biological information via miniaturized versions of the RFID chips Cerf uses in his wine cellar.

In manufacturing, connected machinery could yield similar benefits by helping diagnose problems with equipment, materials, and processes.

“Sensors in machinery generate a great deal of data that can help manufacturers detect a bad product almost instantaneously and correct it as the problem is happening,” noted Yen, who said IoT-enabled diagnostics can help manufacturers eliminate not only wasted product, but also wasted time. “If a factory has 2,000 pieces of identical machinery, and 300 pieces failed after being used in a particular manner for a particular length of time, the company should be able to use that data to predict when the remaining machines will fail and pre-emptively fix them, thereby reducing unscheduled downtime.”

Connected Governments

Although most IoT innovation lies in the commercial sector, some of the most promising IoT use cases belong to government.

One such use case is public safety. “Imagine your house is on fire and the fire department is on its way,” Cerf said. “If you have webcams in the house and temperature sensors, you might want the fire department to be able to access those to determine: Where is the hottest part of the fire? Where did the fire start? Is someone unconscious? If so, in which room of the house?”

Or, imagine a connected camera at a busy intersection, suggests Pete Beckman, co-director of the Northwestern-Argonne Institute of Science and Engineering (NAISE), a collaboration between Argonne National Laboratory and Northwestern University through which he’s developing secure wireless sensors that leverage “edge computing” to quickly process and exploit data locally. With local rather than cloud architecture, Beckman said, sensors in streetlights could automatically deploy salt trucks after detecting cars sliding on ice, or emergency responders after detecting an automobile accident.

“Suppose a camera is looking down on an intersection when it sees a car strike another car. Based on the mass of the vehicles and their deceleration, a sensor connected to that camera could determine whether an airbag deployed and automatically alert 911. It could then immediately start routing traffic around the intersection in different ways by changing the traffic lights,” said Beckman, whose project is named Waggle after the dance bees do to communicate the location of nectar, pollen, and water to others in their hive. “You can imagine a whole set of autonomous reactions in our infrastructure based on collecting good data and being able to process it locally instead of sending it to a central server.”

Waggle is the foundational platform for the Array of Things, a Chicago-based project that will outfit the city with 500 modular sensor boxes, called “nodes,” by the end of 2017. Inside each node, a bundle of sensors will collect a cornucopia of urban intelligence—including 24/7 data on traffic, air quality, weather, and noise—that could help the city optimize resources and enhance services.

“The Array of Things is a science project to understand how we might instrument a city, what we could learn about a city, and how we could use that information to improve a city,” Beckman said.

Like local governments, the federal government can leverage the IoT to fulfill its mission—including that of national security. Where it’s open and accessible, for example, IoT data such as thermostat readings could be used by the military to determine building occupancy when responding to a natural disaster or performing reconnaissance on enemy targets.

“I think the Internet of Things can be an extremely powerful tool for intelligence gathering,” Cerf said.

The National Geospatial-Intelligence Agency (NGA) agrees. As part of its GEOINT Services platform, the agency is researching ways to stand up its own “Array of Things” to serve warfighters on the battlefield, according to NGA Deputy Director for IT Mark Munsell. “If you happen to be in an area with a geofence around it, all the devices in that area—drones, artillery, handheld weapons, and various other sensors—will be interconnected and working together to provide a multitude of services, be they intelligence-, combat-, or logistics-related,” he said. “Instead of your refrigerator, television, and thermostat, your weapons will be hooked up to the internet.”

Consider, for example, a sniper with a connected rifle. “[The rifle] will know its location from GPS, just like your phone would. It will know its orientation from a magnetometer, just like your phone would. And it will know its pitch and yaw from an accelerometer, just like your phone would,” Munsell said. “Now imagine it knows the terrain around you based on a service [NGA is] providing. With that, you can do a viewshed analysis and automatically determine whether your target is in range. If it’s not, when you look through the scope it will guide you to the location on the ground where you can see the target you’re aiming for. That’s what the concept of the Internet of Things will look like when it’s applied to combat operations.”

Powered by GEOINT

In both public and private sectors, the IoT’s promise is manifold, according to Barlow, who cites three benefits in particular.

The first is relevance. “Take an app like Waze,” Barlow said, referring to the smartphone app that uses real-time traffic reporting to crowdsource navigation. “Instead of getting traffic updates every 10 minutes on the radio, you’re getting traffic information that’s current and affecting you right away.”

The IoT’s second benefit is accuracy. “The information you’re getting [from sensors] is raw and untouched. It hasn’t been manipulated,” Barlow continued. “In an emergency room setting, for instance, doctors can act quickly when a patient is connected because they know they’re in a certain room and that they’re having a certain problem.”

Finally, there’s the IoT’s scale. “We’re able to collect information from sensors not just on a local level, but also at regional and even global levels,” Barlow explained. “Because we have connected readers at cash registers all over the world, for example, credit card companies can detect massive fraud as it’s happening and take action to mitigate its impact.”

To Cerf, the benefits are quite practical—the IoT can save lives, time, and money. “For many of these devices, convenience, safety, and efficiency are the drivers,” he said, citing an example from Google, which recently conducted an experiment wherein it used an artificially intelligent neural network instead of human operators to control the cooling system at one of its large data centers. “We used machine learning to determine what was the optimal setting for various parameters, and we discovered that it worked better than people did. It was faster, it recognized patterns, and it saved us about 40 percent on our cooling costs.”

Because GEOINT adds spatial and temporal context to IoT services and analytics, it is the linchpin that makes such returns possible, according to Rob Mott, vice president of geospatial sales and marketing at Intergraph Government Solutions (now Hexagon US Federal). “The GEOINT Community is very important to the Internet of Things because it provides a visual understanding of trends and patterns,” he said, suggesting GEOINT’s major contribution to the IoT would be providing geospatial processing and web services to companies and consumers in the same way NGA seeks to provide them for warfighters. “Working with open standard data that’s readable by GEOINT processes, you can develop web services that provide answers to very tough questions but do not necessarily require looking at a map or interfacing with an app.”

Consider, for example, the Amazon Echo. Using its built-in voice service, Alexa, consumers can ask simple questions about weather, sports, movie times, and more. Thanks to predictive analytics, future Echo-like devices will be able to help commercial, industrial, and government users answer more complex questions in a similar fashion; by accessing web services that leverage geospatial data aggregated from sensors across the IoT, they’ll be able to predict everything from floods to traffic accidents.

“By looking at past trends and other parameters, a geospatial service could give you an answer without you ever having to interact with a map,” Mott said. “It will be a very elegant and powerful GEOINT process that will give you a very reliable, ‘X-marks-the-spot’ kind of answer.”

Next Stop: Interoperability

Indeed, the connected future looks bright. Turning the IoT from a series of cool gadgets to an integrated network providing actionable information, however, will require industry and government to collaborate on policies, procedures, and protocols to mature IoT technology.

Among the areas most in need of attention, experts agree, is interoperability. “The real benefits of the IoT will come from integrating sensors and using multiple data sources to process information and make decisions,” said Dr. Mike Botts, president and chief technology officer of Botts Innovative Research, which designs open standards for sensor systems and geospatial-temporal data. In 2000, Botts developed the OGC Sensor Web Enablement (SWE) standards, a set of IoT standards established to achieve “plug-and-play” functionality of IoT sensors and actuators via web services. “‘Without [standards], you’re left with a big conglomerate of proprietary systems that force you to take a different approach every time you want to engage a different sensor. Standards give us a common language to query the world of sensors and extract observations from them.”

In 2014, Botts co-founded the OpenSensorHub project to support the development of software that automatically fuses data from disparate sensors and actuators. Based on SWE standards, the project’s goal is to accomplish for IoT web services what the USB standard accomplished for computer peripherals.

But SWE is just one potential standard in a single domain. In addition to web services, standards are needed for IoT architecture, communication, connectivity, and more.

“It will take some time to get to a common standard for the kind of data that’s collected, the way in which it’s described, the format it takes, the commands you can give to a device, and the response you can expect in return,” explained Cerf, who expects IoT innovators to spend another five years locking horns over competing standards before a universal one is adopted. At that point, he predicts, it will take another five years to solve the next big challenge: consolidation.

“I don’t want to have a separate app on my smartphone for every device I have in my house or my office; it would take me six minutes to flip through all my apps every time I wanted to flush the toilet or turn off the lights,” Cerf said. “What we need is a much broader kind of control environment—a multi-brand hub we can use to manage and configure all of our devices locally from one place.”

Along with a physical hub for devices, the IoT needs a regulatory hub for policymaking, according to Barlow, who cites security risks as an impetus for government rulemaking. Because the IoT will be so ingrained in citizens’ lives, he argues, the federal government should publish minimum security standards for IoT devices the way it does safety standards for medical devices.

“Medical devices such as catheters and pacemakers must go through certain tests to ensure they’re not going to do more harm than good,” Barlow said. “IoT devices need similar scrutiny, particularly in highly regulated industries like transportation, health care, energy, and the environment.”

IoT challenges and questions abound. But so do the opportunities—especially for GEOINT.

“The IoT adds a whole new dimension to geospatial intelligence by giving us a new way to detect and understand the world at a particular time and in a particular place,” Botts concluded. “It’s no longer just grabbing an image and storing it somewhere to make a map. It’s dynamic, and that’s going to give us a lot of new capabilities.”