Researchers Mine Old News, Web to Predict Future Events

By Sunday, February 3, 2013 0 , , Permalink 0

The project, if successful, will result into a tool that would provide information that is more than just educated guesses or intuition. To ensure that the software’s output is according to expectation, the team consisting of Eric Horvitz from Microsoft Research and Kira Radinsky from Technion-Israel Institute tested it with articles from New York Times spanning over 20 years from archives between 1986 and 2007. The test data also included data from other sources such as the web enabling the software to see more things.

Describing the methodology adopted, the researchers notes in their paper [PDF], “We say that a chain of events belongs to a domain D, if it consists one of the domain relevant words, denoted as wi(D). For example, for the challenge of predicting future deaths, we consider the words “killed,” “dead,” “death,” and their related terms. For the challenge of predicting future disease outbreak, we consider all mentions of cholera, “malaria, ” and dysentery.”

The researchers want to develop the tool such that it will provide proper guidance in terms of short term actions that can be taken based on present data. According to the researchers their work will go on and that they are looking forward to mining more data from other newspapers, digitized books and such sources in a bid to refine their software. The researchers hope that their work will promote additional research in this field by utilizing past experiences and human knowledge to predict future events and plausible interventions as needed.

The software may assist government agencies and other organizations working towards humanitarian causes to provide better response in case of disasters and epidemics.