NEW YORK (TheStreet) -- Sometimes it can seem like the tsunami of digital blurbs, tweets, "likes" and quantitative data that surround us in the digital age is supplanting the need for traditional journalism and publishing, making any medium that presents information in a form longer than 140 characters seem obsolete.
But another trend contradicts the dire outlook for old-fashioned text: a market for text mining of actual reporting as published in newspapers and magazines. In other words, from "traditional journalism." And while many members of the popular press whose materials are currently being used by traders do not know it, Wall Street and select media companies are seizing the opportunity.
As far back as 2008, before "big data" was big news, a report by the Boston-based Aite Group found that the percentage of financial players mining unstructured data, including content from companies like Dow Jones and Thomson Reuters, rose to 35% from 2%, and spending was projected to almost double over the next two years. Writes Adam Honorè, author of the report, "Firms will be looking for any competitive advantage they can find, and unstructured data offers an untapped reservoir of new ideas waiting to be discovered."
What Is Text Mining?
Text mining is the data analysis of natural language works (articles, books, etc.), using text as a form of data. It is often joined with data mining, the numeric analysis of data works (like filings and reports), and referred to as "text and data mining" or, simply, "TDM."
TDM involves using advanced software that allows computers to read and digest digital information far more quickly than a human being can. TDM software breaks down digital information into raw data and text, analyzes it, and comes up with new connections, from unexpected patterns in protein interactions that eventually lead to the development of a new drug, to subtle shifts in weather patterns that might predict a downturn in the price of wheat.
The latter example is of interest to Wall Street, specifically hedge fund managers and algorithmic traders, who are buying licenses from traditional sources like the Associated Press to gain access to breaking news. Traders then use TDM software to mine those feeds to predict movements of markets for everything from government bonds to commodities.
How the AP Helps Investors Make Money
"The advent of TDM promises new revenue streams for traditional publishing outlets and new sources of insight and efficiency for our customers," says Bruce Glover, deputy director of digital for the Associated Press. According to Glover, the AP licenses "machine-readable news products" (MRN) to financial clients, allowing information to move quicker. Given the importance of algorithmic trading on Wall Street, speed is paramount, and machines can process information found in news articles far more quickly and with better recall than humans.
Moreover, according to Glover, there is a new trading strategy that he's heard referred to as hyper-contextual trading (HCT) which recognizes the benefits of assimilating all reliable information to support decisions. Glover says, "It's all very encouraging and the marketplace is indicating that the AP has a valuable and growing asset."