Graphics Processing Power
Written by Peter Buxbaum
GIF 2011 Volume: 9 Issue: 6 (September)
As the demand in the military and intelligence communities for access to and analysis and exploitation of imagery and video grows exponentially, providers of geospatial and imagery processing capabilities are turning to technology originally developed for the video gaming industry.
The number of available sensors and platforms that provide full motion video has exploded from a few dozen assets 10 years ago to thousands today. As a case in point, the Predator unmanned aerial vehicle has grown from a one-camera platform to a behemoth that will carry 36 cameras when its next generation is deployed. The volume of video generated annually by the U.S. military in the Southwest Asia theater can be measured in the dozens of years.
Besides the explosion of sensors and platforms, the imagery and video being collected and analyzed is dense with data. The National Geospatial-Intelligence Agency estimates that future U.S. platforms will collect as much as one petabyte of data—1,000 terabytes or one million gigabytes—per day by 2015. That all this requires the automation of the monitoring and analysis of content is clear, but at this level of data consumption the traditional reliance on Moore’s law to provide ever more powerful processors to handle the deluge of data has fallen short.
Enter the graphic processing unit (GPU), which was originally developed by the video gaming industry as the engine to provide realistic 3-D imagery and effects. Unlike the traditional central processing unit (CPU) that powers the typical desktop or server and can accommodate perhaps as many as half a dozen processing cores, GPUs can accommodate hundreds of such cores on a single card.
GPUs speed the processing of large data sets through software that divides processes into many small tasks, each of which is handled separately by the many processing cores—this is called parallel computing—and then later reassembles the calculations into a single result. But therein also lies the challenge of GPU computing: Applications must be rewritten to accommodate the parallel processing paradigm. That is easier said than done, but applications developers have a choice of two programming environments to help them in this effort.
At some point research scientists realized not only that GPUs were good for gaming, but also that their computing power could be harnessed for a wide variety of data-intensive applications. Growing numbers of providers of geospatial intelligence and video analysis and exploitation applications have also undergone the transition from CPU- to GPU-based processing in the last two to three years.
Picture Processing
Industry experts acknowledge the challenges but say that the results achieved by moving to GPUs make a whole new set of capabilities possible.
“Imagery contains massive amounts of data that are impossible for humans to handle,” said Michael Steele, chief of strategic alliances at Nvidia, a company that, together with Advanced Micro Devices, is one of the two major producers of GPUs. “CPUs can’t handle the data in a power-efficient or meaningful way. We are starting to see interesting examples of commercial offthe- shelf software taking advantage of GPUs.”
“UAVs have multiple cameras providing a real-time video streams,” said Sean Varah, chief executive officer of MotionDSP, a video processing software company. “But this video needs to be stabilized, enhanced, and mosaicked in real time to be useful and accurate, and to provide timely information about the potential threats and targets. CPUs don’t have the computing power to process even a single stream of video in real time, and thus don’t meet the critical requirement for real time intelligence.”
DigitalGlobe, a provider of satellite imagery, performs an array of processing procedures on pictures before sending them off to customers. “Every picture needs processing to correct for movements of the satellites and of the earth,” said Jason Bucholtz, a high-performance computing engineer at DigitalGlobe. “We do a basic quality check and make a whole bunch of corrections on the images to make them usable to customers. We also align the pixels with the actual location on Earth, accurate to a distance of 4.1 meters.”
Getting all that right takes a great deal of complex mathematics. “That is what got us started using GPUs to do image analysis and correction,” said Bucholtz. “We can do it faster than on a CPU platform.”
“Parallelism is the only way to achieve the processing throughput that modern geospatial intelligence tasks require,” said a spokesperson for Manifold, a provider of geographic information systems. “A combination of dramatically larger data and significantly more complex analyses means that old-fashioned single-threaded computation doesn’t cut it anymore for modern missions. The ability to put thousands of super fast cores in a single PC provides transformative power far beyond what can be achieved with conventional CPUs, and at a fraction of the cost.”
IntuVision provides a solution called Panoptes, which examines large quantities of video to find things, such as objects and events of interest, that the human eye doesn’t or can’t notice in real time.
“The Panoptes software evaluates large quantities of video quickly and accurately, which would take far too much time for a person to do manually,” said Sadiye Guler, the company’s chief executive officer. “A highend dual CPU is limited to three to four frames per second, not fast enough to be able to quickly and consistently make out movement in many cases. By adding just one GPU to a CPU server, the speed increases by up to 12 times. This means that for realtime processing, the addition of a single GPU could increase the number of simultaneous surveillance video streams from four to over 40, or could speed the time to actionable intelligence by a factor of 12.”
In a test by the Marine Corps, Panoptes was used to analyze video shot from tower-mounted cameras on surveillance system prototypes developed for Marine Expeditionary Units. “The camera collects data on an unsecured area prior to main units moving in,” said Guler. “To find events of interest in a typical scenario you might be looking at 10 to 50 hours of video. The video collected can now be analyzed and acted upon much faster, within an hour of receiving it, versus waiting half a day.”
Real-Time Results
Besides the speed of the GPU processing, Guler emphasized that the costs of processing are dramatically reduced with GPUs. “The real discriminator is being able to do that at low cost,” she said. “Much of the processing can be done on ordinary desktop or laptop computers.”
Trying to execute sophisticated video post-processing algorithms required for effective reconnaissance using just CPUs would result in between four and six hours of processing for each hour of video, according to Varah. “That is not a viable solution where real-time results are critical,” he said. “In contrast, Tesla GPUs [one of the Nvidia line] enable MotionDSP’s ISR software to process any live video source in real time with less than 200 milliseconds of latency. Instead of requiring expensive CPU-clustered computing systems to complete the work, Ikena can perform at full capacity on a standard workstation small enough to fit in military field vehicles. Processing video in real time is done well with parallel processing, and transitioning our software processing to GPUs is an almost ideal match.”
Another advantage to using GPUs for geospatial and video applications is that they leverage developments of commodity components emerging from the gaming market. “The gaming market continues to drive the graphics market,” said Bucholtz. “Every three months they come out with faster, cheaper and better graphics cards, so we get free upgrades along the way. Once you make the investment in the technology, you get to ride the commodity curve and get benefits from it for the foreseeable future.”
Much as developers have the choice between two lines of hardware, from Nvidia or AMD, so can they also choose between two programming environments through which to transition traditional linear type processing characteristic of CPUs to the parallelism required for GPUs. The Compute Unified Device Architecture (CUDA) is supported by Nvidia. Some say it is proprietary to Nvidia, although Steele disputes that characterization.
The alternative to CUDA is OpenCL, an open standard being developed by a consortium that includes both AMD and Nvidia. It would allow applications to run on both companies’ GPUs. At this point CUDA appears to be the architecture of choice for developers in the geospatial and video realms, although OpenCL enthusiasts argue that the open standard is making good progress.
MotionDSP’s product, Ikena ISR, and Intuvision’s Panoptes both used CUDA and Nvidia’s Tesla GPU to implement their transitions to parallel computing. MotionDSP has also been developing code for OpenCL and AMD hardware. “We’ve been working extensively with OpenCL for over one year,” said Varah, “and we’ve partnered with AMD to tune our OpenCL code for their GPUs.”
ITT VIS adapted its orthorectification tool to GPUs using CUDA, but it is waiting for OpenCL to take off before commercializing its GPU product. ITT’s orthorectification tool, which prepares images for use in mapping or GIS systems by removing distortion from the image, is incorporated in its ENVI imaging processing and analysis package.
“Some of our customers were making the evolution towards GPUs and asked us how we could support them,” said Beau Legeer, ITT VIS vice president for product marketing and management.
The company used CUDA to write software for one of its orthorectification tools. “The results were amazing,” said Legeer. “The software ran almost 100 times faster over the traditional approach. We decided to maintain the capability and build the infrastructure into our products.”
At the same time, ITT has hesitated to transition GPU technology to its commercial product. “There is not a lot of customer demand yet,” he said. “Standards have not yet gelled. When vendor technologies mature and standards come in we can turn on the technology for more and more applications like orthorectification.”
Open Standards
AMD supports OpenCL to the exclusion of CUDA, said Patti Harrell, the company’s stream computing director. “We have a core belief in open standards,” she said. “You can achieve productivity gains with CUDA, but how does that translate to the next generation of hardware or alternative solutions? With OpenCL developers can create their applications to these standards and not have to rewrite them with each generation of hardware. Users can pick whatever hardware the applications run best on.” Harrell claims that there has been a recent uptick of interest in OpenCL including mention of the standards in some requests for proposals.
Nvidia supports OpenCL, according to Steele. But he added that due to the cumbersome nature of the standards process, it was necessary to develop a software architecture, CUDA, specific to Nvidia. “It takes longer for standards to grow and mature,” he said. “If we waited for OpenCL we would not have GPU applications today. OpenCL is making some headway,” but its acceptance as an industry standard is still “far into the future.”
At this point, CUDA is the choice for enhanced performance, according to Bucholtz. “If you choose OpenCL, you have the chance your code will run on both the Nvidia and AMD platforms,” he said, “but the tradeoff tends to be in performance.”
ITT is waiting for OpenCL to be approved. “We would like to be able to write standard solutions,” said Legeer. “If it doesn’t happen soon we will have to make a choice.”
For the future, AMD’s Harrell sees greater integration between GPUs and CPUs to create a unified programming environment. “This will enable applications to be developed quicker,” she said. “Using the technologies in an integrated way will mean providing a better application experience for users.”
“I expect software vendors will write all new algorithms to GPUs,” said Legeer. “This is the next evolution of processing power.”
“It’s GPU or you’re dead,” said the Manifold spokesperson. “The data is too much on these types of applications.” ♦
