Displaying for Knowledge
Written by Peter Buxbaum
FROM ANALYSIS AND COMMAND AND CONTROL
TO THE TACTICAL LEVEL, NEW TECHNOLOGIES
ARE BEING DEVELOPED TO IMPROVE
THE DISPLAY OF GEOSPATIAL DATA.
As geospatial intelligence data becomes increasingly available for use in an ever-widening variety of applications, from big-picture command and control to small-unit tactical operations, there is growing interest in improving the means of displaying that data. Doing so within each of those contexts, however, presents its own set of technological challenges.
At the National Geospatial-Intelligence Agency, for example, analysts spend hours viewing and interpreting geospatial imagery on their office computers. The agency is seeking to upgrade its cathode ray tube desktop displays to newer technology.
Command centers could make use of large-scale displays that can be viewed and discussed by a group of staffers, instead of all of them crowding around a computer monitor. Such an application requires the exploitation of the latest projection technology and the synchronization of data from multiple projectors and multiple sources in order to generate data-rich, highresolution imagery on a large display.
Deployed units are finding uses for geospatial data on platforms as diverse as warships and tactical ground vehicles. Here, the challenge is often to display images of acceptable resolution in an area with limited transmission bandwidth.
A number of companies specialize in technologies that facilitate and actuate large-scale, high-resolution displays that can be used to display geospatial intelligence. Mechdyne, for example, specializes in advanced visualization environments, including three-dimensional and stereoscopic displays, as well as large-scale audiovisual solutions. Geospatial intelligence analysts often want 3-D viewing to discern depth in the imagery and to interpret spatial details. This is achieved by presenting two separate images, viewed while wearing polarized glasses that direct one image to the left eye and the other to the right eye, achieving the perception of depth.
Resolution is the name of the game when it comes to Mechdyne’s large-scale displays, according to Jeff Brum, the company’s vice president for marketing and business development. “One of the most frequently requested traits of the system currently is resolution,” he said. “Our customers want greater detail within a displayed image and want to bring up multiple image sources at one time and correlate the data among them.”
Command and control applications of this type use a series of high-resolution projectors to display, at the user’s choice, a single, very large picture, or a number of tiled images of simultaneous video feeds, video conferences, terrain data and other imagery. A series of six high-resolution projectors, each displaying 8 million pixels, results in a display of 48 million pixels at once. Next-generation projectors will have resolutions of 8,000 by 4,000 pixels, for a total of 32 million pixels, according to Brum.
“Mechdyne’s systems allow a user to choose, at any given time, multiple displays from several sources, or a single image from a single or several sources of data to be displayed across the entire wall,” he said.
HIGH-RESOLUTION INTELLIGENCE
Mechdyne was the prime contractor for the design and installation of such an ultrahigh resolution visualization system in the Joint Intelligence Laboratory (JIL) at the Joint Transformation Command for Intelligence (JTC-I) in Norfolk, Va., which is known as the Knowledge Wall.
The JIL is a multi-service laboratory tasked with delivering innovative operating intelligence capabilities to field combat commanders. The laboratory’s development work is based on integration of intelligence data from multiple military and intelligence community resources.
The 37-foot by 13-foot Knowledge Wall provides 53 million pixels of resolution, which enables simultaneous display of data from multiple computer and live feed video sources. The rear-projected unit, utilizing eight Sony SXRD Quad HD projectors, can show multiple 3-D stereoscopic images in the center onethird of the screen while displaying additional images on both sides.
“The JIL system is a super high-resolution tiled display, where each projector could be considered four tiles, since each takes four inputs to fill the 4,000-pixel resolution of the projector,” said Brum. “The projectors used in the Knowledge Wall each have a resolution of 4096 by 2160, more than four times higher than high-definition resolution. It’s like four projectors in one, which is why it requires four graphics inputs.”
Computer and video images from dozens of consolidation points in the JIL, including secure and non-secure sources, are served to the system through a fiber optic matrix switch. Display sources are selected by a “multi-windowing control system” that works with three PC clusters, which are used to create stereoscopic views from Google Earth and ESRI ArcGIS application software.
The multi-windowing control system is a switching device that directs video and computer sources to the display system, Brum explained. “The controller then allows any number of those multiple computer or video sources to be selected and displayed simultaneously on a single or tiled display,” he added. “Users can select how many sources they want open in windows and then can dynamically move them around and resize each window at will. For example, if a video broadcast suddenly becomes of interest, that window can be increased to any size or fill the entire tiled area.
“The JIL Knowledge Wall is one of the largest and most complex single visualization system installations in the world,” Brum added. Displays such as the
Knowledge Wall are facilitated by systems provided by Mersive Technologies. Mersive’s software incorporates an understanding of projectors, cameras and the image patterns they generate to enhance resolution and correct geometric distortion.
“By understanding where the images are in three-dimensional space and how they interact with the surface of the display, we calibrate a line of multi-projector displays and make it much easier to get a lot more pixels out of the display,” said Randall Stevens, the company president.
Display systems such as the Knowledge Wall often use images from multiple projectors to generate a single image on a large display. “The mathematics embedded in our software allows the system to take a pattern and reroute pixels to the right point on the display,” said Stevens. “It also adjusts color intensities, reconciles areas of projector overlap, and straightens out lines.”
TUBE TROUBLES
If the technologies fielded by Mechdyne and Mersive represent some of the grander geospatial data display applications, NGA’s quest to replace its desktop displays is operating at a more down-to-earth level. Last year, the agency released a request for information (RFI) as a preliminary step in replacing its current cathode-ray tube (CRT) monitors with flat-screen plasma or liquid crystal displays (LCD).
NGA currently utilizes high-resolution monochrome and color CRT monitors to view panchromatic and color displays with excellent results, according to Art Cobb, an NGA imagery scientist. Monochrome CRT displays allow excellent sharpness and high dynamic range, enabling the observer to distinguish details in imagery. Color CRT displays have historically been used for color imagery exploitation and presentation of color text and graphics.
NGA analysts use commercial off-theshelf image viewers such as RemoteView, SOCET SET/GXP, ERDAS Imagine, ENVI and Image Scout, which work well with CRT monitors to provide a continuous panning motion through large-area images and to detect faint target objects of interest while searching through an image.
The problem for NGA is that manufacturers of CRT monitors have been migrating away from CRT technology and toward flat-panel displays. The point of the RFI was to determine whether any flat-panel manufacturers could meet the stringent functional requirements of NGA analysts. According to the RFI, NGA was seeking information “on current and future advanced display technologies that can provide flicker-free stereoscopic imagery and exhibit smooth and continuous, blur-free roam capabilities.”
“CRTs’ response time is very fast,” explained Cobb. “An electronic beam excites the phosphors,” luminescent chemical compounds within the screen, “and within microseconds the image comes up or goes away. With flat plasma and liquid crystal displays, the response comes in milliseconds, and that is an issue for us for two reasons.”
The first reason that a response calibrated in milliseconds is a problem is that speed is important when viewing stereoscopic displays. Image distortion can result if the two images encompassing the stereoscopic display are not precisely synchronized. The second reason is that when analysts are searching through an image with a panning motion, the slower speed of the flat-screen technologies often display “artifacts” such as streaks or bright spots on the display screen.
“The biggest help we got from the RFI is that display vendors are beginning to look at and are attempting to solve the very problems of slow response for commercial reasons,” said Cobb. “However, our requirement for resolution and calibration are still a step above what prevails in the commercial market right now.”
The RFI also uncovered problems of lack of compatibility between display drivers and the imagery software packages being used at NGA. In other words, the RFI did not yield a solution for NGA. Nor did the agency run across any government off-the-shelf or government- developed display systems in the desktop form factor that met its requirements, said Cobb.
NGA then followed up with a market survey to see if it could uncover any other alternatives. The market survey identified one display system that Cobb termed “acceptable.”
“There is an evaluation ongoing now, and a recommendation will be delivered,” he said. “We will likely be moving to another facility in a couple of years, and the displays bought for that facility will be based on the findings of the report. In order to have it in time to populate the new building, the best candidate will be procured at that time.”
NGA is currently in the process of building and moving to a consolidated East Coast facility at Fort Belvoir, Va., which is scheduled for completion by 2011.
TACTICAL USES
Geospatial display technologies are also increasingly being adapted to deployed situations. Mechdyne has a portable system it calls the Cave, which can be delivered in a trailer and assembled within hours to provide large, 3-D, stereoscopic displays to the field. “The projectors are contained in boxes that have been ruggedized for field deployment,” said Brum, “and it provides a full range of capabilities.”
Companies like Mechdyne and Canadabased OSI Geospatial are working to bringing high-grade geospatial displays to the lower tactical levels as well. Mechdyne is working on an application called the Geospatial Environment for Command and Control Operations (GECCO), related Mark Ferneau, the company software division program manager.
“GECCO is built on top of Google Earth, and the application can be integrated with military tactical networks,” said Ferneau. “It can be used to display blue force tracking and to view or play back planning and mission briefings. GECCO provides a complete interface for command and control, is interoperable with tactical networks and has Google Earth as its core engine.”
This system is contemplated to be deployed on tactical vehicles. “One of the challenges associated with tactical applications of display technology is having sufficient storage in their onboard systems,” said Ferneau. “Google announced earlier this year that a Google Earth enterprise system could be loaded on a computer hard drive. We are looking to integrate with that for easier supportability and because end users can get what they want at their fingertips.”
GECCO has been tested by the U.S. Army with good results, according to Ferneau. OSI Geospatial grew from a role in warship electronic chart displays located on the bridge to developing advanced situational awareness systems located in shipboard operations rooms. From there, it migrated to small craft, land-side and soldier-centric applications.
“Our original focus was in providing geospatial displays to support naval operations,” said Ken Kirkpatrick, the company’s president. “The situational awareness system is integrated into all the ship’s sensors and is used in support of maritime interdiction and force protection operations. That led to a small boat command and control technology system, which has been installed on small assets such as rigid inflatable boats to support boarding investigations. The system allows naval command and control from the mother ship and provides the ability to monitor operations out of the line of sight.”
Both of these systems have been deployed by the U.K. Royal Navy.
More recently, OSI Geospatial has been called upon to develop a land-side port security system. It is also on a team headed by Raytheon that is working to demonstrate networked tactical situational awareness and communications capabilities for the dismounted warfighter as part of an Army Tank-automotive and Armaments Command (TACOM) project.
OSI’s role in the $11.8 million Ground Soldier Ensemble (GSE) contract is to provide capabilities from its Dismounted Close Combat, Command and Control System (DC4S). “DC4S provides situational awareness right down to the dismounted soldier,” said Kirkpatrick. “The U.S. Army is investing millions of dollars in this.”
Kirkpatrick also noted that there are some two dozen soldier modernization programs being ramped up by military organizations around the world, which are designed to provide advanced technologies to the individual soldier.
“The actual engine that drives the display of data used in this system has a very advanced architecture,” Kirkpatrick added. “It utilizes one of the richest of any data sets available, and the system is also functionally rich. We are not just displaying map data, but also displaying different kinds of operational data of benefits to users that meet specific operational requirements.”
The system envisions that each soldier, for example those working in a small team in an urban environment, would be equipped with a handheld display. “The system gives commanders the ability to view the location of each team member,” said Kirkpatrick, “and it gives the soldier the ability to view the location of friendly, as well as unfriendly, forces. This system should reduce incidences of casualties caused by friendly fire.”
The system also has the capability to track soldiers even when a GPS signal is denied, for example upon entry into a building. “The system is equipped with algorithms that enable it to analyze movements and take out errors,” said Kirkpatrick. “We worked with a government agency for four years, and they’ve spent a million dollars with us to develop this capability for U.S. special operations forces.”
One of the challenges associated with these kinds of tactical systems involves display resolution, noted Mechdyne’s Brum. “If you have limited resolution, how someone interacts with the display and how they access the data is extremely important,” he said. “We are working closely with end users to learn how they want to access data and in what order, so that we can develop the best user interface possible.” ♦