A GEOINT-supported Metaverse is Coming

The idea of the “metaverse” – an immersive 3D internet – has recently captured the imagination of consumers and industries alike. Ongoing improvements to internet and cloud technologies coupled with the mainstreaming of realistic, immersive 3D technologies in mod/sim applications and video games have brought to life worlds of exciting possibilities.

As an evolution of the internet, the metaverse’s ultimate benefit—for GEOINT and beyond— will be its ability to improve upon the real-world tools and environments it seeks to re-create. A standards-based internet allowed for applications tied to vast numbers of independent databases, software tools, and data repositories to come together in a 2D, hyperlinked platform. An open metaverse can do the same to enable both real-time visualization and secure data processing.

Featured image caption: 5G modeling in a GEOINT-inspired metaverse, image courtesy of NVIDIA and Ericsson

More than just online video games, the metaverse will comprise interoperable, persistent platforms where users can collaborate and interact within 3D immersive worlds—both fictional and representing real-world locations.

“Although the term ‘metaverse’ is used loosely these days, USGIF defines it broadly as a digital realm where people seek deep insights from the gamut of geospatial data using virtual reality, augmented reality, multi-dimensional models and simulations as well as video and audio,” said USGIF CEO Ronda Schrenk.

A GEOINT-enhanced “real world” metaverse will be a place for digital twins where software developers can test and write software. It will be a place where GEOINT expertise will be relevant to new missions, risks are reduced for decision-makers, and the creativity of analysts will shine.

With decades of collective experience and maturity in relevant data, models, software, talent, standards, and more, the GEOINT and wider geospatial communities are perfectly positioned to provide valuable contributions to the metaverse.

• This year’s GEOINT Symposium theme is “GEOINT: From Maps to Metaverse.” Join us in St. Louis May 21-24 to explore the GEOINT metaverse! Click here to learn more.

A Real-World Metaverse

A key difference from prior simulation environments is our newfound ability to rapidly produce geo-specific (rather than geo-typical) representations of the real world. In other words, the synthetic environments in simulations and games are increasingly precise representations of actual places in the world and no longer generic, representative examples.

Programs such as the U.S. Army’s One World Terrain can provide a high-fidelity synthetic training environment in any location across the globe. And in the gaming world, programs like Microsoft Flight Simulator incorporate a 3D model of the Earth for players to explore. A common thread tying both examples together is their reliance on 3D data solutions pioneered by Maxar Technologies and its industry partners, generated by pairing massive satellite imagery archives with 3D technology.

“Where a user like the Army requires precise geo-registration with absolute accuracy of three meters in all dimensions, gaming applications prioritize appearance over exact geographic accuracy,” said Tony Frazier, Maxar’s Executive Vice President and General Manager, Public Sector Earth Intelligence. “While that difference in priorities requires different approaches to generating 3D data, what is true in both circumstances is that consumers across the military and the public want to visualize their world in fully immersive 3D.”

In terms of training, the move to geo-specific environments will benefit both humans and machines. For humans, a user can immersively experience a place they plan to travel to in the real world, giving them better awareness of the location or the ability to rehearse tasks virtually they may need to perform while evaluating different scenarios and courses of action. For Machine Learning (ML) and Artificial Intelligence (AI) applications, such as those behind autonomous vehicles, digital twins allow for localized training to optimize performance for a particular location and set of tasks. As we look beyond Earth, such modeling and simulation also has the potential to benefit space operations and the exploration of other planets.

• NVIDIA’s GTC Developer Conference March 20-23 will focus on AI and the Metaverse. Click here to learn more about this free event.

An Open Metaverse

When built well, the metaverse will be greater than the sum of its parts. The adoption and use of open standards and platforms, such as 3D Tiles, glTF, Geopose, ARML, and Universal Scene Descriptor (USD), will break down the barriers between systems to bring together the many parts—3D graphics, VR/AR/XR, distributed collaboration, physics modeling, CAD, BIM, IoT inputs, scientific/HPC processing, full-fidelity visualization, synthetic data generation, Earth observation and other real-world data collection, simulation, robotics, and more—required for a fertile metaverse.

Important elements of a GEOINT metaverse workflow could include complementary tools and platforms, such as NVIDIA’s Omniverse and Cesium, which are able to unlock and build upon the expertise of the GEOINT community.

Cesium uses the open standard they created, 3D Tiles, to combine many existing trusted geospatial data sets in one place. The below image shows CityGML data of New York City that has been converted to 3D Tiles 1.1 and visualized in Cesium for Unreal, enabling metadata picking and styling for segmentation and classification. Users can simply touch buildings, objects, or terrains to access relevant metadata such as their height, material composition, Relative Tactical Importance, business name, and more. Cesium leverages the storage and archive benefits of the source data while improving performance and capability at runtime.

3D Tiles of New York City in Cesium for Unreal, highlighting metadata picking and styling. Image courtesy of Cesium

Similarly, NVIDIA Omniverse is a platform for creating and operating metaverse applications. Omniverse is based on the open and extensible Universal Scene Description (USD), which was born in the Visual Effects (VFX) industry and has since been adopted for use in Architecture, Engineering & Construction (AEC), digital engineering, operational digital twins, climate modeling, robotics and autonomous systems, and beyond. By using USD, Omniverse allows users to build metaverse applications using the rich foundation of existing USD data, workflows, and expertise.

“To truly simulate a twin of the physical world, geospatial intelligence is a key ingredient for ensuring simulation accuracy across industrial applications, like in warehouses and gigafactories, and scientific applications, such as global climate modeling,” said Marco Tillmann, senior product manager of Omniverse digital twins at NVIDIA. “The geospatial community’s commitment to building interoperability between geospatial technologies and NVIDIA Omniverse will help accelerate such applications.”

Rapidly updated synthetic environments will be a boon to effective simulation and training. Image courtesy of NVIDIA

Also contributing to an open metaverse is the convergence of geospatial and game engine technology. By enabling Epic Games’ Unreal and Unity Technologies’ Unity game engines to ingest real-world data from the Cesium platform, Cesium’s Open Source plugins ‘Cesium for Unreal’ and ‘Cesium for Unity’ have combined the efficient storage and representation of real-world data developed by the GEOINT/geospatial community with the high-fidelity, performant 3D rendering developed by the gaming and VFX communities.

“Open source plugins like the ones Cesium provides for game engines increase the speed of innovation by enabling creators to build accurate, large-scale simulations quickly and easily,” said Brady Moore, Cesium’s Director of Mission Support. “Working with game engines that are constantly evolving also improves system lifecycle and sustainability, ensuring that simulations are increasingly interoperable and up-to-date.”

Users of these complementary platforms can use existing, trusted data to build immersive, interactive worlds that are accurate enough for advanced simulation and analysis—and pave the way towards building the connected metaverse.

Contributing to the Metaverse

The GEOINT and geospatial communities have much to contribute to the metaverse. Relevant ‘precursor’ technologies matured by our communities include modeling & simulation, digital twins, ML/AI, 3D/4D data streaming and visualization, AR/VR, indoor location, underground infrastructure, and the underlying geographic principles that underpin it all. But how do we bring our community’s knowledge and experience to the many players involved in building the metaverse?

An emerging locus for this knowledge and experience is the Metaverse Standards Forum. Hosted by The Khronos Group, the Metaverse Standards Forum brings together leading standards organizations, companies, and research organizations for industry-wide cooperation on the interoperability standards needed to build an open metaverse. Just as the internet is only successful because open standards allow anyone to contribute a webpage, so too will the metaverse require open standards to see widespread adoption and proliferation. Without standards, the metaverse will remain a series of apps rather than an open platform.

In recognizing the value of developing the metaverse, many principal members of the Metaverse Standards Forum, including Cesium, Maxar, NVIDIA, and OGC, come from the GEOINT and geospatial communities. Much of our community’s collective knowledge has been documented in the form of OGC (and other) Standards and Best Practices, such as 3D Tiles, CDB, I3S, OGC API – Tiles, CityGML, GeoVolumes API, 3D Portrayal, InfraML, GeoPose, and more. As such, OGC is taking active steps to promote the reuse of this valuable knowledge by forming its own Geo For Metaverse Domain Working Group. The Group will align the work of various groups and activities within OGC—such as 3D, Urban Digital Twins, ModSim, and the OGC D&I working groups—with the needs of the metaverse. It will complement the work undertaken in the Metaverse Standards Forum by serving as OGC’s primary point of contact with it.

“OGC sees the metaverse as the ultimate 4D modeling & simulation environment that can help us accelerate our understanding and response to urgent global issues like disasters and climate change,” said OGC CEO Nadine Alameh, Ph.D. “With our collective experience in the integration of heterogeneous and cross-discipline data, our strong grasp of real-data modeling and simulation at scale, and our commitment to open consensus-based standards and interoperability, we are excited about the value that geospatial brings to the realization, scaling, and sustainability of a truly open metaverse.”

The GEOINT community is uniquely positioned to lead the way in contributing to an open metaverse that will also successfully support GEOINT operations. If the vision of a visually stunning, open, secure real-world metaverse is to succeed, our community must also build upon its experience and skills to work in the environment, understand the new tools enabling its development, and establish the infrastructure to handle its processing and communications needs.