New Light from LIDAR

Written by Peter A. Buxbaum

The U.S. military has been accumulating a great deal of LIDAR data from aircraft and terrestrial vehicles in Iraq and plans to do the same in Afghanistan. Forces are using BuckEye, a system developed under the auspices of the Army Topographic Engineering Center, for example, to collect data on tens of thousands of square kilometers of Iraqi urban areas.

LIDAR, which was first developed in the early 1990s, uses 1.064 nanometer wavelength laser light pulses to gauge elevations by measuring the time delay between transmission of the pulse and detection of the reflected signal.

A range finder mounted in an aircraft flying at an altitude of between 1,500 and 3,000 meters swings back and forth collecting data on up to 150,000 points per second, providing resolutions of one point per meter on the ground and one point per 15 centimeters vertically. LIDAR has also been used at ground level to collect even more detailed information about terrain topology. The data returned by the LIDAR sensor provides location data on an x-y-z axis, referred to as a point cloud.

In Iraq, the BuckEye system combines airborne LIDAR technology with digital color camera imagery to provide pictures to commanders and planners on the lay of the land. LIDAR elevation data has supported improved battlefield visualization, line-of-sight analysis and urban warfare planning.

One example of the expanding use of LIDAR involves CG2, a wholly owned subsidiary of Quantum3D, which recently announced that the second phase of its LIDAR Database Generation Process is nearly complete. The initiative converts LIDAR scans into visual database terrain and models, an activity that includes placement of natural and manmade features, with little or no human interaction. The effort is part of a Small Business Innovation Research (SBIR) project Phase II sponsored by Naval Air Systems Command (NAVAIR).

A key objective of this initiative is to minimize the manual labor required to build simulation environments by processing high-resolution LIDAR data. Using sophisticated automation and process acceleration that leverages the latest GPU technologies, the LIDAR Database Generation Process has been successful in identifying trees, buildings, roads and the terrain profile within the LIDAR point cloud, and then converting these features into visual database components.

NAVAIR wanted Quantum3D to create a tool that would facilitate the incorporation of LIDAR data into simulation and training systems. “What they asked us to do is to come up with a solution that would use LIDAR data for simulation and training,” said Sandra Vaquerizo, Quantum3D’s director of business development. “We have developed methods to pull multiple LIDAR scans together into 3-D visualizations and to isolate features such as buildings, trees and changes in terrain.”

NEW BENEFITS

Quantum3D’s SBIR project is emblematic of the direction LIDAR developments have been taking of late. Decreases in costs have made LIDAR data ever easier to collect. The question is what to do with the data once it has been collected. Using the LIDAR data in simulation and training systems is part of the growing trend to find new ways to benefit from the technology.

“Point clouds are actually nothing but a pile of x-y-z data,” said Oodi Menaker, marketing product manager at Israel-based Tiltan Systems Engineering. “The main challenge is to extract point cloud data with which you can then work to describe the ground, buildings, power lines, trees, power poles and many other geographical features.”

That process could be done and has been manually, but that process is very labor intensive, explained Lisa Spencer, a senior research scientist at Quantum3D. “The previous methods for building LIDAR databases included consulting blueprints and photos to measure the height of buildings in order to visualize them on a graphic display,” she explained.

“The more modern systems tackle this through automatic processing,” added Menaker. “We have automated the process of transferring point clouds to geographical features.”

The latest improvements in the ability to depict precise features from LIDAR point-cloud data involve advances in the software algorithms used to process this information.

In the case of Quantum3D’s SBIR grant, the initial object was to provide a mechanism to make LIDAR data more usable to the simulation and visualization community, Vaquerizo explained. “The LIDAR data is converted into the standard format for visualization called Open Flight,” she said.

One of the problems associated with LIDAR data is the way it is collected. “Data is collected over a period of years, but terrain and cultural objects change,” said Vaquerizo. “Data is often old and of low resolution. The LIDAR point cloud data is collected in a more random fashion and doesn’t have a gridded structure to it.”

One of Quantum3D’s accomplishments has been to create a tool suite that allows the fusion of LIDAR data from multiple reads and databases. “We are able to align the scans in such a way as to convert the contents of multiple random point clouds to a representation of buildings and terrain,” said Vaquerizo. “The output is in Open Flight, which means that it is with any image generator you have.”

The first phase of the Quantum3D SBIR involved developing a product design that was accepted by the Navy, while the second phase involved building a working prototype. “We are in the process of commercializing that into a product that will be able to utilize the LIDAR data from an Open Flight database through an image generator for training or mission rehearsal," said Vaquerizo.

She expects the company to launch the product toward the end of the second quarter of 2009. NAVAIR has reveised but has not yet fielded the Quantum3D product.

HELICOPTER VISUALIZATION

Another Quantum3D innovation, which proceeded from the NAVAIR SBIR, is the ability to transfer relatively new LIDAR data into immediate visualizations, thus enhancing the mission rehearsal utility of that data. “The beauty of it is that the LIDAR data could be hours rather than years old,” said Spencer. “We can capture all the geometry of a building down to its finest detail all in an automated fashion. This opens up a new level of interest in mission rehearsal applications. The LIDAR data can be used to check for changes in the battlefield terrain or the possible presence of explosives, and it can be converted within hours or minutes without measuring or hand modeling anything.”

This capability is being built upon in yet another Quantum3D SBIR grant, this one emanating from the Patuxent River Naval Air Station, Md. “The focus of this project deals with the critical issue of solving problems of having helicopters land in brownout conditions in desert situations such as Iraq, and the high mortality rate associated with this problem,” said Vaquerizo.

This solution involves the use of data generated from LIDAR sensors to provide helicopter pilots with a visualization of the terrain after the helicopter pilot loses the ability to see what is happening outside of the cockpit. “LIDAR penetrates the dust, and this is helpful to the pilot in successfully landing the helicopter,” said Vaquerizo. “We are applying LIDAR technology to produce a database faster, in immediate mode, on the fly.”

To make that happen, LIDAR sensors mounted on the helicopter are processed in on-board computers to be visualized and presented to the pilot right then and there. The visualization is continuously updated during the course of the flight.

The process involved in penetrating brownout conditions is to figure out which parts of the LIDAR data represent the ground and which do not, explained Spencer. A traditional visualization of the terrain is then built off the ground points. The nonground points are grouped into clusters representing individual features such as buildings or trees.

Each cluster is extracted to develop the geometry of objects, such as the height and shape of buildings or the elevation of the foliage canopy. As with the product Quantum3D generated in its NAVAIR SBIR, the output of the LIDAR data processing is in Open Flight format, allowing it to be visualized and displayed on a variety of image generators.

This scheme represents a dramatic change in the processing of LIDAR data in that it involves processing of streams of LIDAR data rather than processing them in batch. That is the traditional process for data, which is not intended to be displayed in real time. Streaming data is necessary in this application, Spencer explained, “because streaming data gets collected incrementally and all processes are updated incrementally.”

Quantum3D can accomplish the feat by relying on its expertise in real-time software performance and new hardware technologies such as graphical processing unit acceleration and its own high-performance graphics hardware. “There have been some advances in technologies that fit this application very well,” said Spencer. “Our expertise in designing real-time software puts us at the leading edge on how to tackle this problem.”

Quantum3D has completed the first phase of this project to rave reviews, according to Vaquerizo. She expects a determination on whether the Patuxent River Naval Air Station will proceed with a second phase of the project at some point early in 2009.

ROADWAY DATA

Another case of the innovative use of LIDAR data is an Army SBIR grant to TerraSim to improve an existing simulation product by using BuckEye LIDAR data to enhance information on roadways.

Until recently, TerraSim’s RoadMAP product used black and white, panchromatic and color imagery to extract roadway center lines and topologies. These are incorporated in systems that are used by the military and other customers to simulate operations in dense urban areas.

“Roads fall into the category of lines of communications,” said Dave McKeown, the company’s president. “The military is interested in simulating how to get in and out of an operational area. But roads really are the single hardest feature to get out of LIDAR without a fair amount of manual extraction.”

The TerraSim SBIR takes advantage of the fact that LIDAR can be used well in conjunction with other spectral imaging methodologies. This allows LIDAR data to be collected and simultaneously added to data from other sensors such as hyperspectral, short wave, infrared and near-infra detectors.

Automated feature extraction is a capability that allows software to recognize certain specific objects represented in LIDAR point clouds. Programming the software to be on the lookout for topographical features such as hills or manmade “cultural” objects, such as buildings, vehicles or power transmission lines, allows those features to be separately and distinctly portrayed in the LIDAR image.

The TerraSim project is particularly challenging because it seeks to extract data with respect to roads, a feature not characterized by dramatic changes in elevation. Features such as buildings are easier to extract from LIDAR data than roads.

"There is a high level of discontinuity between the top of a building and the ground," MecKeown explained. "Current LIDAR extraction software is relatively mature when it comes to finding buildings or treetops."

But in the case of roads, the difference in elevation between the center of a road and the edge of a road is much smaller. “Elevations over roads tend to be smooth and different from elevations characteristic of other features,” said McKeown. “We use LIDAR data to localize where to look in black and white or color imagery for more information and for further support of the visualization. Since we don’t expect to see roads running across the tops of buildings but between buildings, we are able to throw away a fair amount of imaging and data and concentrate processing on where roads are likely to be.”

The LIDAR data that TerraSim is using as part of its Army SBIR project work provides its RoadMAP tool with a new piece of information from which to draft the measurements of elevations. “It will help us do a better job of finding roads and extracting road delineation information,” McKeown commented.

TerraSim expects that the addition of LIDAR data will improve the accuracy and utility of its RoadMAP product. “LIDAR is being used to enhance and validate features extracted from other media, such as black and white or color imagery,” said Wilson Harvey, a senior computer vision scientist at TerraSim. “LIDAR gives a direct measurement of elevation much more accurate and reliable than trying to calculate elevations based on sets of imagery. It also adds a new set of data for systems to look at to ascertain road characteristics and to help augment what Road- MAP knows about a road. We expect to obtain much more fidelity in terms of the x-y-z position of the road, and this will have a huge impact in the 3-D visualization extracted from the data.”

The TerraSim product will be organized to have independent evaluation modules, which will deal separately with color imagery, black and white imagery, and LIDAR data. “Each of these will talk to a higher level part of program, which will figure out where the roadbed lies and put the pieces together in a certain way to generate the road center line,” said Harvey.

TerraSim’s SBIR is a two-year deal that started in October 2008. “We have various six-month points where we link up with the contract administrators to make sure we all are on the same page and on the right track,” said McKeown. “We are expected to present evaluation versions in six-month intervals, which our Army sponsors run in their environment and give us feedback on the productivity and accuracy of what we are doing.

“As with many SBIRs, we benefit from a close working relationship with the sponsor,” McKeown added. “They have the source data set and know the problems associated with the exploitation of that data. They have the expectation of succeeding in extracting road data from BuckEye. They suspect that process can be improved otherwise would not have let this contract.”

McKeown expects the Army to evaluate TerraSim’s design of the LIDAR enhancement to its RoadMAP product and make a determination whether to go forward with a second phase. If the Army gives the green light, McKeown said, he believes that TerraSim could have a LIDAR-enhanced product in the marketplace within a year to 18 months.

FEATURE EXTRACTION

Perhaps the most important benefit from the incorporation human can do manually with percent accuracy and it takes a day,” he explained. “A machine can do it with 80 percent accuracy in an hour. This still presents an impediment to using the machine solution if 20 percent of the data must be revisited to get it up to human-level performance. The problem is how much user correction you will tolerate before you say it is cheaper to do this manually.”

But the LIDAR data allows significantly more accuracy to automated feature extraction. “If we can privatize the use of LIDAR,” said McKeown, “we can do two things. We are able to tell people that we are using LIDAR as a primary source, and we are able to present a commercial product that is going to provide a cost-effective feature extraction solution.” ♦

Back to Top