Photos by Karl Maasdam
In the virtual version of Oregon’s Elliott State Forest, created in the lab of Raffaele de Amicis, users can control time and space. De Amicis and his close collaborators, graduate students Rundel Samuel and Shahbaz Badr Arash and undergraduate student Hsiao Dominic Dai, can change the weather and move effortlessly over the rough terrain or teleport to any spot they choose. They can see what the forest looks like after a few trees have been chopped down or after large swaths have been clear cut.
“What you see is not just a nice visual effect, it’s the digital version of part of the real Elliott Forest — a digital twin,” said de Amicis, associate professor of computer science in Oregon State’s College of Engineering. “I started this work because I wanted to create a digital dimension that provides a feeling of immersion. I’m fascinated by how this technology can empower us as humans, and I want to see how it can augment our cognitive and physical capabilities and our productivity.”
The five-year project, which began in October 2022, is funded by the Economic Development Administration of the U.S. Department of Commerce. Before joining the Oregon State faculty in 2016, de Amicis worked for nearly 20 years in the private sector as a researcher and manager for European organizations involved in various aspects of interactive computer graphics.
The forest digital twin encompasses about 2.8 million trees over 77.6 km2 in the southwest Oregon Coastal Range. But he plans to greatly expand the digital representation to cover far more acreage. For now, de Amicis and his team have created a high-fidelity, immersive visualization of a fraction of the forest incorporating 26,000 trees. This visualization is best experienced as a projection on a wall, which produces an impressive, three-dimensional cinematic effect. The bigger the image is the better, which is why de Amicis looks forward to the completion of the university’s Jen-Hsun Huang and Lori Mills Huang Collaborative Innovation Complex, a transdisciplinary and teaching facility scheduled for completion in 2025. Its facilities will include an extended reality theater in which he anticipates projecting life-size digital twins on enormous, wraparound screens. “The Oregon State University Collaborative Innovation Complex is a transformative project that will have a positive impact on Oregon for many years to come,” de Amicis said.
Developing the digital forest began with survey data supplied by the Oregon Department of Forestry and later extended by the Oregon State University College of Forestry. The researchers prepared that trove so that it could be processed by geographic information system, or GIS, software, which collects, analyzes, manages, maps, and stores geographic information relative to actual positions on Earth.
Shahbaz Badr was among those who programmed the GIS software so it could transform the data into the desired geospatial information. “Tree density, for example, was used to estimate what areas were forested, what areas were grassland, and which were shrubs. And based on the topography, we could figure out how water flowed and accumulated,” said Shahbaz Badr, who has since earned his doctorate in computer science and is now a software development engineer with Esri, a leader in GIS software, located in Redlands, California. The GIS output was then fed into a game engine (software traditionally used to create video games) which combined all of the geospatial layers and created an accurate visualization. “Every virtual tree is matched to an actual tree in the forest and at the correct scale,” Shahbaz Badr said. The game engine also gives de Amicis and his team the ability to interact with the environment, just as players in video games can interact with those virtual worlds. The engine also helps visualize the impact of logging over time — even years into the future. Such information could help forest managers, policymakers, landowners, and others concerned about forest health to figure out how to balance environmental stewardship with economic gain. De Amicis also intends to embed a climate simulation model capable of portraying what manifestations of climate change — like drought, insect infestations, and supercharged fires — will look like a year, a decade, or 20 years from now. There are, of course, existing models that predict the effects of human and natural forces on the environment, but de Amicis maintains that immersive visual technology evokes a much stronger reaction compared with conventional communications vehicles like reports, graphs, and charts, and is more likely to spark a sense of urgency. De Amicis is also constructing a digital twin of the Oregon State campus that will be used to simulate pedestrian and vehicular traffic. It’s broader purpose will be to establish safe and effective evacuation scenarios in response to various threats, including gas leaks, fires, earthquakes, and other dangers. The foundation of an urban digital twin is a city information model, or CIM. CIMs weave together geospatial data to form a digital representation of an urban area: every street, sidewalk, building, traffic sign, power line, and so on, appears in the digital twin.
However, collecting the vast amount of data needed to build a digital version of even a small urban area can be prohibitively expensive, explains de Amicis. The undertaking would typically require assembling entirely new datasets from sources such as lidar scanning, photogrammetry, and other costly methods. And generating the model from the raw data is a complex task that requires highly specialized expertise and high-powered computer assets, like distributed networks.
“The challenge was to create the model and the traffic simulation without buying expensive software,” de Amicis said.
His approach draws on existing, cost-free, public geospatial data sets. It’s an inexpensive option, but because none of the data sources is accurate enough on its own, multiple data sets must be integrated to establish an accurate CIM. To develop the campus visualization, a powerful but reasonably priced game engine was used, while simulating the movement of people and vehicles is achieved using open-source traffic simulation software.
“The resulting framework is considerably less complex than others and is capable of running comprehensive simulations on a standalone computer,” wrote de Amicis in an article he co-authored in the February 21, 2023 issue of Frontiers in Virtual Reality. The article reported that all but one performance benchmark was attained in proof-of-concept testing.
Pulling back to take a wide view of his research, de Amicis noted that its impact should not be gauged by any particular application, whether it involves a virtual forest or an emergency urban response. A better measure would be the broader influence on his discipline: “Our contribution as computer scientists are the algorithms that represent that information,” he said. “It’s the capability to generate these algorithms that empower us to use the software in a manner that is most helpful. This is what we are targeting.”