Two closely related science and technology programs aim to improve image location and search capabilities, saving intelligence ana- lysts significant time and effort. U.S. intelligence analysts often must wade through enormous amounts of imagery—
both photographs and videos—to uncover the exact
information needed. To make matters worse, data often does not contain geolocation tags, which indicate
where the images were taken.
So, the Intelligence Advanced Research Projects Activity (IARPA) Finder program is designed to help analysts locate non-geotagged imagery, whether photographs
or video. The Aladdin Video program seeks to improve
search capabilities for specific events so that analysts can
more quickly find the videos most relevant to their needs.
For imagery without geolocation tags, analysts work
hard to deduce as much as they can using reference data
from many sources, including overhead and ground-based
images, digital elevation data, existing well-understood
image collections, surface geology, geography and even
cultural information. It is a time-consuming and labor-
intensive exercise that often meets with limited success.
“The goal of the Finder program is to develop tools to help
analysts locate where in the world images or video were
taken,” explains Jill Crisman, IARPA program manager for
both Finder and Aladdin Video.
To do this, the system requires a model of the world. The
model includes a wide variety of reference data, such as
satellite imagery, maps and photographs publicly available
on the Internet. “Finder compares what it sees in a query
image or video that it is trying to locate with that world
model,” Crisman says. “With Finder, analysts only have to
look at a few possible locations. Before, they would actually
have to search the entire region where they think it might
be located to see if the information matches various loca-
tions. We’re building automated tools to try to filter out the
stuff that is obviously not in those locations.”
Crisman likens the process to a person sorting through
old photographs and trying to remember where they
were taken. “Basically, Finder’s research is looking to use
every last bit of information that’s in the query image or
video. The visible terrain, the layout of the objects or even
Dual research projects help intelligence
agencies grapple with images and video.
BY GEORGE I.