FOR IMMEDIATE RELEASE
ODNI News Release
No. 38-19 Nov. 18, 2019
IARPA Announces the Winners of the OpenCLIR challenge
The Intelligence Advanced Research Projects Activity, within the Office of the Director of National Intelligence, announced the winners of the Open Cross Lingual Information Retrieval – OpenCLIR – prize challenge. Launched in June 2018, this challenge involved innovative approaches to retrieve information from audio and text documents, using English queries against documents that were not in English. The challenge focused on low-resource languages - less studied languages for which large amounts of training data do not exist. One application of such a capability would be to support effective triage and analysis of large volumes of information in a variety of less studied languages. For the OpenCLIR challenge, the language was Swahili.
“A cross-language information retrieval capability that can query both speech and text documents in a lower resource language has not yet been produced for mass consumption. This challenge highlighted state of the art techniques for machine translation and speech recognition development, and joint optimization of all the natural language processing required to achieve a usable end-to-end solution,” said IARPA program manager Carl Rubino. “Aside from limited speech recognition and machine translation data provided by IARPA to train a system, participants were challenged to employ data harvesting methods to improve baseline capabilities. Entries that relied only on the training data provided by the challenge were not expected to be competitive.”
Of the 37 teams registered – from 14 different countries – only five made it to the final evaluation stage. They were evaluated by the National Institute of Standards and Technology on a detection metric that awarded return of relevant documents and penalized false alarms and missed documents. Winners were awarded in the text category but no team met the minimum required threshold performance for speech. To learn more about the challenge and a complete list of winners please click here.
IARPA invests in high-risk, high-payoff research programs to tackle some of the most difficult challenges of the agencies and disciplines in the Intelligence Community. Additional information on IARPA and its research may be found on https://www.iarpa.gov/.