While some photographs—
the Pentagon, for example—
are easy to geolocate even
without geotags, others offer
more of a challenge. This
image shows mountains
west of Kabul, Afghanistan.
improvements in retrieval methodologies on real-world
problems; and increasing the availability of appropriate
evaluation techniques for use by industry and academia,
including development of new evaluation techniques more
applicable to current systems. “We actually do have people
around the world who participate in these evaluations,”
The systems currently have a mean average precision—
the performance metric for the precision for which they
can find all positive results—of about 10 percent for event
queries with few positive example videos. Crisman’s goal
is to improve that to 15 percent this year. The numbers,
however, are misleading because of the diversity of videos.
Videos about making a sandwich, for example, include
footage of people making sandwiches with their feet,
which is outside the norm. Of the top 10 retrieved videos,
the systems average 70 percent precision. Crisman aims to
increase the precision through the remainder of the program. “The prototypes that the teams have been developing have been very, very good at searching when you have
a lot of video examples. If you have 100 examples of making a sandwich, and you have a nice background collection of events that are not making a sandwich, the teams
are doing quite well at finding those videos and creating a
nice results list,” Crisman reports.
The prototypes are challenged, however, when fewer videos are available. “That’s where we’re pressing the research.
The big challenge I’m pushing is to get the results better
when you have fewer examples, which basically means that
you have to do better content tagging with English words
and do better searches. The two go hand in hand. That’s
how you get better performance,” she says.
Aladdin is a five-year program that just started its fourth
year. It will wrap up in February 2016. SRI International,
Menlo Park, California, has two teams participating in Aladdin. One began the program as Sarnoff Laboratories, Princeton, New Jersey, before being acquired by SRI. The other two
teams are led by Raytheon BBN Technologies, Cambridge,
Massachusetts, and Carnegie Mellon University, Pittsburgh.
Crisman stresses that IARPA does not violate privacy
rights. “All the data that we’re using for the Finder and
Aladdin programs is publicly available data. If we’re using
Internet videos, we go to great lengths to protect the privacy of the people in those videos,” she concludes.
George I. Seffers,
To share or comment
on this article go to