LiDAR’s New Dimension
Written by Peter Buxbaum
GIF 2010 Volume: 8 Issue: 7 (October)
Light Detection Technology Shines When Used
With Other Sensor Data to Enhance The Picture
Used By Analysts, Planners and Commanders.
Overflights the U.S. Army is conducting over Afghanistan and Iraq have raised the profile of light detection and ranging (LiDAR) data, as analysts, commanders and warfighters continue to explore its utility for a variety of tasks, from mission planning to training.
LiDAR has an advantage over some other geospatial technologies in that it provides accurate elevation data. Under the right circumstances, it can also detect hidden objects. But LiDAR’s true value as a military and intelligence tool, say the experts, comes when it is used in conjunction with other sensor data to enhance the picture used by analysts, planners and commanders.
LiDAR uses 1.064-nanometer wavelength laser light pulses to gauge distances by measuring the time delay between transmission of the pulse and detection of the reflected signal. A range finder mounted in an aircraft swings back and forth collecting data on up to 150,000 points per second, capable of providing resolutions well under one meter on the ground and one point per 15 centimeters vertically. The data returned by the LiDAR sensor provides location data on an x-y-z axis, referred to as a point cloud.
In Afghanistan and Iraq, U.S. forces use BuckEye, a system developed by the Army Geospatial Center that combines airborne LiDAR technology with digital color camera imagery to provide pictures to commanders and planners on the lay of the land. LiDAR elevation data supports improved battlefield visualization, line of sight analysis and urban warfare planning.
“There is always a need to get better, more accurate intelligence faster,” said Trey Howell, defense solutions manager at Merrick & Company, “whether that means gathering intelligence on what currently exists, or modeling what you think will exist.” Merrick provides LiDAR capabilities to the National Geospatial-Intelligence Agency, Naval Research Laboratory, and Army Research, Development and Engineering Command.
Military entities have been using remote sensing methodologies for many years, Howell noted. “The telescope was used by civilizations to observe objects from a distance. Those observations were taken into consideration to make a decision,” he explained. “Similarly, LiDAR, like other remote sensing technologies today, provides intelligence. LiDAR is an observation tool utilized from a remote location that is considerably more efficient than developing terrain models from stereo photographs. When combined with other remote sensing technologies, more information or intelligence can be extracted without physically being near the object being observed.”
“We certainly have seen a big increase in requests for LiDAR technology to be inserted in our ENVI software,” said Beau Legeer, director of product marketing at ITT Visual Information Solutions. “In the last 12 months we have seen requests, specifically from our defense and intelligence customers, to be able to visualize full point clouds and to be able to automatically extract features like power lines and three-dimensional buildings.” ENVI is ITT VIS’s imaging processing and analysis package.
Sensor Improvements
Improvements in LiDAR sensor technology and in software and hardware are also fueling LiDAR’s newfound popularity. “LiDAR sensors have dramatically improved within the last 10 years,” said Andy Dougherty, a business development manager at Northrop Grumman. “The original LiDAR sensors gathered data at around 15,000 pulses per second. Now you are talking about 150,000 to 200,000 pulses per second. The point clouds have become that much denser. With greater granularity, users are able to create higher quality visualization products. As sensors have improved, so has the ability to survey more area and so has the robustness of the products used to analyze the data sets.”
Providers of software used to process LiDAR data have improved the performance of their products three-fold, according to Dougherty. “Software has enhanced the ability of the collector of the data to identify the quality of the data while airborne or immediately afterwards,” he said. “Before, you had to wait until a disk drive was delivered back to the office.”
Other software improvements allow the instant integration of LiDAR data from multiple collecting flights. “Data from multiple flight lines are ingested around GPS data,” said Dougherty. “This enables you to take a good look at an entire data set in minutes rather than days.”
Improvements in computing also have allowed LiDAR capabilities to be brought within the reach of many more organizations. “In 2003, you needed a very high-end computer just to do basic processing of LiDAR data,” said Dougherty. “Improvements to hardware and software now mean that most any organization can ingest and process the data.” Using LiDAR data in simulation and training systems is also a growing trend, and has motivated simulation and training technology companies to investigate how LiDAR data might be exploited in their systems.
“LiDAR offers a thoroughly efficient way to get objects detected and distance understood,” said Pratish Shah, a spokesperson for Quantum 3D, a developer of training and simulation solutions. “LiDAR data gets you real distances on top of 3-D imagery.”
Imagery generated from LiDAR data can be incorporated in mission rehearsal modules within hours or days, allowing warfighters access to fresh information about the terrain on or over which they will be conducting operations. “Satellite imagery used for mission rehearsal is often months old,” said Shah. “With LiDAR, warfighters can better see what they will be experiencing on a mission.”
Because of these advantages, Quantum 3D has developed software tools that enable the incorporation of LiDAR point cloud data into simulation databases. “We have updated flight simulation databases for the mapping we have done around airports,” said Shah. “One comment we have heard from pilots is that the trees in the vicinity of the airport are now where they are supposed to be.”
Quantum 3D is currently in the process of investigating other ways in which it can enhance its simulation databases with LiDAR data, according to Shah.
LiDAR’s utility includes the inherent advantages it possesses in relation to other geospatial data sensors. “LiDAR extends the data capture window to 24 hours,” noted Dougherty. “Electro-optical sensors that are not infrared are limited to daytime capture. LiDAR expands the capture window and gives users the capability of doing more in the same aircraft. We’re seeing the increased utilization of LiDAR with other sensors.”
“LiDAR sensors are also typically less expensive than visual or thermal sensors and much less than hyperspectral systems,” said Legeer.
Pinpoint Locations<p>
LiDAR outstrips the capabilities of other sensors with its ability to pinpoint the location and elevation of surface elements such as buildings, trees and roads. “LiDAR can go much further by penetrating the canopy, whereas with electro-optical sensors you can only see the treetops,” said Legeer. “LiDAR can penetrate the trees as long as there is a light path to see what is beneath the tree.”
LiDAR data can also provide more accurate vertical, horizontal and digital elevation and terrain measurements. “That is why LiDAR can create very accurate models for urban warfare scenarios,” said Legeer. “It also provides warfighter-enhanced target detection capabilities.”
The detailed mapping of urban and non-urban terrain can bring other benefits to warfighters as well, noted Tom Lobonc, director of defense products at ERDAS. “Having a detailed urban surface model is beneficial to identify lines of sight, which are critical for a variety of tactical applications,” he said. “For example, LiDAR has been used to identify suitable observation posts, locations for cover and concealment during operations, and sites for locating communications transmission and interception equipment.”
Because of LiDAR’s generally superior accuracy, it can be used to “control” other imagery data. Measuring points in today’s best commercial satellite imagery is typically accurate to within seven or eight meters, explained Lobonc. But if an analyst can identify common points in the imagery and the LiDAR (for example a building corner) he or she can then update the parameters of the image sensor model to make the imagery much more geographically accurate, Lobonc said.
LiDAR’s added value, concluded Howell, “comes in the form of providing a third dimension.
“The common element between remote sensors, such as satellite, airborne and terrestrial, is the geospatial component,” he explained. “Being a geospatial dataset, LiDAR allows the relationships we make between two-dimensional datasets more complete and accurate by adding a third dimension.”
Even so, LiDAR by itself has its limitations, noted Howell. One critical requirement for the military use of airborne LiDAR is complete control over the skies. That was a major concern under military scenarios envisioned during the Cold War, but has not been an issue in Iraq or Afghanistan. “That is the primary reason LiDAR adoption in the defense community has lagged behind the commercial community,” said Lobonc. “The military has traditionally not operated under conditions in which relatively low level overhead flights could be conducted without significant risk of loss, and therefore these kinds of collection assets have only recently received strong interest.”
LiDAR also presents logistical challenges, since the geographic accuracy of the data is highly dependent on the quantity of GPS satellites available combined with the distance to the nearest GPS ground reference station and the quality of the positional instrumentation accompanying the sensor according to Lobonc. Ongoing advancements in instrumentation and GPS/IMU processing packages are helping to reduce this challenge.
The large volume of data generated by LiDAR sensors represents another potential disadvantage to the use of LiDAR. “The industry can handle multi-gigabyte image files well, but we need to do more work on point files,” said Lobonc. “Information stored in the form of random point clouds is more complex than the gridded sets of values associated with storing and rendering image files. They must be rendered a certain way, and the addition of the third dimensional aspect, with its attendant requirement for dynamic perspective presentation, renders the data presentation task more hardware intensive. Robust data paging mechanisms are critical to be able to display large data sets without subdividing them into tiles. Tiling large datasets remains quite common.”
Lobonc expects the U.S. military to put more money into the rapid visualization of LiDAR data in coming years. “We are seeing more developers starting to take advantage of processing on video cards,” he said. “That enables visualization much more effectively. We are working on moving some of this processing over to the GPU [graphics processing unit] to achieve much higher performance.”
Not every user of geospatial intelligence needs full-blown analysis and exploitation tools, noted Dougherty. “Sometimes warfighters just want to know that what is over the next hill is what they think it is. At the same time, those users often don’t have a lot of bandwidth at their disposal.”
The Georeferenced PDF, or GeoPDF, developed by TerraGo Technologies allows the sharing of georeferenced maps and data in PDF documents. “These can be put across the Internet or the wire very rapidly,” noted Dougherty. “The ability to use GeoPDFs and other formats that are readily usable by the military and intelligence communities has been a great achievement.”
Howell contended that the real value of LiDAR is when it is combined with other sensor technologies. “When comparing other remote sensing technologies such as hyperspectral imagery to LiDAR,” he said, “it’s really an issue of comparing apples and oranges.
Each technology has its own unique advantages as well as disadvantages. “A common statement you will hear within the geospatial community is LiDAR is another tool in our tool box to help us accomplish the task at hand,” he added. “Available technology allows the user to fuse multiple datasets from multiple platforms, acquired at different times and fuse the data together to extract information that is only present when multiple data is aligned.”
Automatic Extraction
Fusing data from multiple sources increases the probability that features can automatically be extracted. Automated feature extraction is a capability that allows software to recognize certain specific objects represented in LiDAR point clouds. Programming the software to be on the lookout for topographical features such as hills or man-made “cultural” objects such as buildings, vehicles or power transmission lines, allows those features to be separately and distinctly portrayed in the LiDAR image.
This task can be accomplished, with limitations, with LiDAR or spectral data alone, said Lobonc, while adding that the process becomes more accurate when multiple sources of data are fused. “What we are working on now is a process that identifies features such as buildings by using a combination of spectral characteristics and elevation characteristics. We are demonstrating work flows for customers in this area and it is showing a lot of promise.”
Fusing LiDAR data with spectral information is also particularly useful in analyzing vegetation canopies, Lobonc noted.
To that end, ERDAS is working on an improved work flow to ingest LiDAR data into its system so that it can be turned into various products that can be readily used. “Users will be able to visualize specific LiDAR returns or certain classes of features,” said Lobonc. Point cloud data will be able to be used as another geospatial layer in exploitation workflows which will aid in the identification and classification of features in conjunction with multi-spectral imagery. These enhancements will be available in ERDAS Imagine 2011, which will be released later this year.
ERDAS is also rebuilding its viewer technology architecture so that LiDAR data can be viewed and exploited natively as a point cloud. “These days almost everyone uses third party standalone packages to view LiDAR data and perform basic exploitation,” said Lobonc. “But then you often have to pull data out of that package and load it into your main workhorse package to perform a full exploitation and generate products. When this architecture is completed, users will have the ability to view and process point clouds, traditional terrain, imagery, and vectors all simultaneously and linked to other data sources so everything is geospatially aligned and can be algorithmically fused.”
ITT VIS will also be releasing tools to exploit LiDAR data. “Our release of ENVI 4.8 in November will include LiDAR visualization,” said Legeer. “Entire point clouds regardless of size will be capable of being viewed. Users will be able to fly through the cloud and see renderings of the scene in three dimensions.”
That development will be followed up in 2011 with LiDAR exploitation tools specifically requested by ITT VIS customers. “With these tools, users will be able to view LiDAR point clouds in their native form to do such exploitation tasks as line of sight, helicopter landing zones, and digital terrain and elevation modeling,” said Legeer. “We are making these tools very fast so that users can have instant retrieval and display of results on these very complex scenes.”
Late in 2011 or early in 2012, ITT VIS plans to tackle the challenge of fusing LiDAR with multi-spectral and hyper-spectral data. “Together these will provide more accurate and better target identification capabilities,” said Legeer. “We expect this to go into the core of ENVI, and that our target detection algorithms will get better once we have this additional modality of LiDAR at our disposal. Each of these is another step in a multi-year process toward true multi-sensor fusion.”
The flip side of multi-sensor fusion is the collection of raw data with multi-sensor packages. “That will be the next stage of evolution,” said Dougherty. “One of our key examples is a plane we are flying that has five sensors on board, one of those being LiDAR. That will be the story through 2015. You won’t have single sensors onboard aircraft, but you will have multiple sensors optimized for the collection atmosphere and the collection target.”
LiDAR collections are escalating, analysts note, with the additional platforms that are scheduled to go operational in 2010 and 2011 resulting in significant increases in LiDAR collected, to thousands from hundreds of square kilometers per day.
Speed and Robustness
“The operational use of these additional platforms will necessitate the need for automated techniques downstream in processing, exploitation and dissemination environments,” said Ron Krakower, a business development director at Sarnoff Corp. “Our focus has been on delivering software to the Army that removes the touch labor from the process and automatically produces complete and accurate 2-D and 3-D products from LiDAR and imagery in hours.”
Over the last four years, Sarnoff has worked with the Army Geospatial Center under a program that provided for prototyping and technology transfer of tools that comprise Sarnoff’s software pipeline for BuckEye images and LiDAR. Under this program, Sarnoff integrated and delivered to the Army a software pipeline that ingests LiDAR pointclouds and BuckEye imagery and automatically and rapidly produces bare-earth DEMs, ortho-rectified imagery and textured 3-D models.
In addition to speed and automation, there is equal importance that the software is robust enough to work under extreme variations in terrain, noted Krakower. “Ground segmentation algorithms that assume a flat terrain will fail in areas such as Afghanistan, in which height can vary by over 500 meters within one square kilometer,” he said. “The software must work with the same speed and accuracy in exploiting LiDAR from heavily mountainous areas covered in forest to urban areas with dense buildings.”
Krakower agrees that there is tremendous potential when considering multiple sensors onboard an aircraft and the impact of combining LiDAR with other sensor data such as imagery and video.
Industry is on an innovation path to provide a soldier or analyst the ability to produce an accurate 3-D model of terrain and buildings with geo-registered visualizations of video against this new 3-D data set, said Krakower. “And, this will all happen while the platform or UAV is still in the air,” he added. ♦
