An open-source app to harvest field data from Microsoft’s HoloLens mixed-reality headset has been developed by a lab that specializes in applying data science to the study of conservation biology. The app allows researchers to collect data while walking and looking at vegetation. Spatial data is displayed in real-time on the HoloLens screen and can be saved for later analysis.
Ecologists will not always require expensive and cumbersome equipment to measure vegetation in the field. Rice University researchers have discovered that a modern heads-up display works quite well. Rice researchers used a Microsoft HoloLens as a mixed-reality sensor to feed VegSense, their app for measuring understory vegetation, or plant life that grows between the forest canopy and the forest floor.
A proof-of-concept study conducted by graduate student Daniel Gorczynski and bioscientist Lydia Beaudrot demonstrates that VegSense could be a low-cost alternative to traditional classical field measurements.
The hardware-software combination excels at quantifying relatively mature trees in the wild, which is one measure of a forest’s overall health, according to their study published in Methods in Ecology and Evolution.
For this study, we wanted to be really deliberate in trying to replicate more traditional understory vegetation structure measurements. We tried to get that level of detail. The goal is to have the mesh cover as much of the vegetation as possible because that is what gives you the best scan.
Daniel Gorczynski
Gorczynski came up with the idea to try HoloLens, commonly marketed as a productivity tool for manufacturing, health care and education. He developed the open-source software for the device and noted that while the combination is less effective at picking up saplings and small branches, there’s ample room for improvement.
Gorczynski said he was introduced to mixed-reality sensing while an undergraduate at Vanderbilt University and recognized its potential for biological studies. “It seemed sort of like a natural fit,” he said. Gorczynski brought the idea to Beaudrot in 2019 shortly after his arrival at Rice.
The combination of stock hardware and custom software cost far less than systems based onlidar (for “light detection and ranging”) most often used in three-dimensional field studies, said Gorczynski, who developed VegSense on a platform geared more toward 3D games and interactive experiences than hard science.
Field tests in Houston’s Memorial Park revealed that, at least for mature trees, the smaller solution is just as effective. In their case study, VegSense easily detected 48 of 50 such trees in the target area, a circle about 30 feet in diameter that Gorczynski walked, looking up, down, and around to build the 3D database. (“Think of an asterisk with a circle around it,” he said, describing the data-capture pattern.)
“For this study, we wanted to be really deliberate in trying to replicate more traditional understory vegetation structure measurements,” Gorczynski explained. “We tried to get that level of detail.”
He notices a holograph-like grid pattern that tracks the surfaces of vegetation as he scans the environment. “What’s really cool about that is that you can see what the scanner is picking up, as well as the spots you missed,” Gorczynski explained. “The goal is to have the mesh cover as much of the vegetation as possible because that is what gives you the best scan.”
“The results were so good that Dan quickly wrote it up for publication,” Beaudrot said, noting that Gorczynski expanded his validation of the gear during a subsequent field trip to Tanzania, which is the focus of one of 15 tropical forests in the Rice group’s recent rainforest study.
“Because it’s so inexpensive, this device has the potential to facilitate a lot of great ecological research,” she says. “Right now, collecting vegetation information on the forest floor is extremely difficult without a lot of manual labor or a very expensive lidar system.”
“So this is a game-changing, low-cost device,” Beaudrot said. “It will not provide the same resolution data as lidar, but this is only the first application. We hope that making VegSense open-source to the ecological research community will spark all of the potential directions it can take.”
