Information Overlay
DEFENSE AGENCY SEEKS EFFICIENT WAYS TO PROVIDE AN ARRAY OF TEXTUAL PRODUCTS FOR EMBEDDING INTO GEOSPATIAL IMAGES.
Open source information is routinely used by military image analysts to bolster the understanding of a location or object under analysis. In the current environment, however, collecting and summarizing this information can cause considerable delays in providing the final geospatial intelligence product to the user.
In response, the Defense Advanced Research Projects Agency (DARPA) is seeking to reduce the delays in using this valuable resource through an initiative called the Open Source Information Geospatial Overlay (OSIGO). The OSIGO program provides one glimpse of a government-industry team’s efforts to more efficiently provide an array of textual products for embedding into geospatial images.
Open sources of textual information for embedding into geospatial imagery—available through press releases, Websites, closed captioned sections of video news programs and other resources—are a powerful way to enhance any imagery product.
“OSIGO technology would allow military image analysts to integrate open source knowledge into their analyses while maintaining focus on images themselves, increasing their analytical efficiency,” said Jan Walker, a DARPA spokesperson.
Walker provided one example of how an OSIGO tool might be used: “Consider an analyst with an image of the open water and an unusual concentration of ships. OSIGO technology should be able to aggregate news content about the event occurring at that location— in this case, an oil-spill cleanup—and project a summary of that content on the image.”
But there is also a technology challenge that must be addressed, which is how to enable OSIGO to build a bridge between the text about what is happening and the images of what is happening.
“The key stumbling block right now is the complexity of the natural language processing task. It is hard to automate the understanding of natural language to an extent that events and their locations can be reliably and accurately tagged and overlayed on geospatial imagery. We need advances in document classification and analysis,” said Erik Larson, founder and chief executive officer of Knexient, which is one of two companies with an accepted OSIGO proposal.
“The state of the technology right now is largely still keyword-based, and we will need to move to a concept based model that is scalable and accurate to usher in the next generation of text-image technologies,” said Larson.
TILES AND POINT
To help bolster the intelligence community’s capability, DARPA initiated a Small Business Technology Transfer Program (STTR) topic for innovative approaches in natural language processing, knowledge-based systems, and other technologies for OSIGO.
“Under the STTR topic, DARPA selected two proposals for funding—Geosemble Technology’s TILES [Text-to Image Linking and Embedding System] and Knexient’s POINT [Processor for Open Source Intelligence]—to address the central technical challenge of aggregating and summarizing geo-referenced textual data,” said Walker.
Both projects are independent of each other and use their own technical approaches.
“The key problem that we want to address here is how do I associate all of these interesting text documents with this facilitation, so that I can look at this and discover, ‘This is an oil spill and it was reported in the news, and I can read all the details about it,’” observed Craig Knoblock, chief scientist, Geosemble Technologies. “I may be looking at an image and ask, ‘What do I know about that and what happened here?’
“The idea here is to look at an image and automatically figure out what news reports or other text documents can be associated with the image so that you can get a clearer picture of what is going on,” Knoblock continued.
While the initial focus of the contract is expected to be on using text documents as the media application, the technology can be expanded to link video and audio media to provide the data chronologically in a userfriendly format. The increasing use of video with closed-captioning features increases its utility as a text tool and mainstream intelligence resource.
While the DARPA OSIGO project will use open-source, unclassified information, the completed TILES system is expected to be easily integrated with classified information.
Geosemble will also use an enabling technology from Fetch Technologies, its parent corporation, to extract on-line data. “Their tools can take on-line sources of information and turn those things into databases or into what they call ‘agents’ to go out and very quickly query their sources,” revealed Knoblock.
The second part of the TILES contract is a collaborative effort between Geosemble and the University of Southern California to summarize the text documents. “If there is a set of news reports, the idea is that you can not only find the news reports and link them to the image, but you can also produce a multi-level summarization of the reports so that I do not have to read the document in front of me. Rather, I would have a one-line summary so that when I click on the location, it shows me a summary of the different news reports that are linked to that image,” elaborated Knoblock.
A TILES-like system is expected to accelerate the flow of information to the on-scene commander, compressing the decision- making timeline. A representative scenario using TILES would involve providing intelligence support for an RQ-1 Predator unmanned air vehicle orbiting over an area of interest.
“You don’t necessarily have the time or people to do all of this research, compile a report and come back and deliver it to the decision-maker. We are accelerating that process by automating the steps necessary using algorithms that have been developed by Geosemble, so that we can accelerate the information needed to make smart decision fast—to take a picture of a target, launch a missile at it, or whatever action you need to take,” said Andre Doumitt, a spokesperson for the company.
This project is a natural progression for Geosemble, whose corporate competencies include the ability to take different types of geospatial sources and accurately link them together. The company’s efforts enable road-vector data or other layers of geospatial information to be accurately aligned with and placed on top of satellite imagery. “This is just one step in the process. Now we are talking about a new type of layer, textual information,” pointed out Knoblock.
This DARPA contract award, as a STTR project, requires a deliverable product to be due during Phase I (in six to nine months). “What we will do is build a proof-of-concept product. This will be an end-to-end system that really demonstrates the idea, built on real-world data to show that the approach we are working on is feasible,” said Knoblock.
Phase II of the project is slated to last 24 months. “What we will really be doing is refining it and doing a lot more work,” he said.
The Knexient POINT system is used to answer two questions about open source text: What happened, and where did it happen?
“For instance, given an article from the Associated Press entitled, ‘Trapped Eastern Timor Rebel Demands Face-to-Face Talks,’ POINT tags the article with information about the primary event, the context of the event and its location,” explained Larson.
This information is translated into Keyhole Markup Language (KML) and displayed on Google Earth as graphics of the event at the location where they occurred. Analysts then use the information to make interpretations of the events in East Timor more easily in light of the open source reports about what is happening. Since the news is overlayed over the geospatial images, it adds context to the interpretation task.
“The state of the technology now is largely still keywordbased, and we will need to move to a concept-based model that is scalable and accurate to usher in the next generation of textimage technologies,” concluded Larson.
PARIS, PARIS AND PARIS
A third solution is offered by MetaCarta and its collaborative partners, including ESRI and Google Earth.
“At the enterprise level we are the company that answers the question: What is going on here?” said Randy Ridley, vice president and general manager, public sector, for MetaCarta. “You can zoom-in to a map any place in the world and all of the text data in the enterprise will be positioned on the map so that you can read the actual articles that talk about what is going on in that area,” he added.
The product foundation of MetaCarta’s technology is made up of Geographic Text Search and GeoTagger. While MetaCarta’s products alone do not embed text in geospatial imagery, they provide that capability when combined with another technology. “These two products, depending how they are used, basically automate the process of putting text on imagery. To make it a completely automated process, you have to use ESRI with the imagery in the geospatial information system platform,” said Ridley.
In one partnering application, MetaCarta’s GeoTagger is a component of ESRI’s ArcWeb Services. This enables ArcWeb Services users to go to geo-reference data found in unstructured documents such as articles, field service documents, internal reports and Web pages, and view the results in custom mapping applications.
MetaCarta’s technology solution has two components. The first part establishes a geographical range, termed a rectangle. “We search for all text information that refers to any places in that range or region,” remarked Ridley. A keyword, or characterstring, search for the characters in English that make up “Paris,” for example, would produce Paris, Texas, Paris, France, Paris Hilton and possibly other results.
“What we then do when we search a range, is search for every place in that range, every little city with its latitude and longitude, and bridges and other landmarks, and all kinds of things. We look in the unstructured text and identify those candidate places, and we use natural language processing to tell the difference between Paris, France, Paris, Texas, and Paris Hilton,” he explained.
An icon is then placed on the map with the specific latitude and longitude on the GIS imagery product.
MetaCarta’s technology “can handle placing documents anywhere in the world,” said Ridley.
The company’s process is automated, accredited by the intelligence community and is in use across the federal government at highly classified levels. ♦
