Practical advances in Natural Language Processing (NLP) and analytics now enable a robust connection from the basics of semantic processing to advanced analytic models, producing predictive analytics with direct value to commercial business and government mission. Multi-language, unstructured text, including social media posts, is enabled as Smart Data and exposed for a variety of analytic purposes.

We will explore applied examples of this "stack" of semantic and predictive analytic technologies. As a technical example, the Intelligence Advanced Research Projects Activity (IARPA) Early Model Based Event Recognition using Surrogates (EMBERS) project fused Named Entity Recognition with mathematical modeling, to predict civil unrest a week ahead of the news, and is now transitioning to application in government intelligence.

As an applied example, the US Centers for Disease Control (CDC) wanted to track and understand large scale discussion about Ebola on social media. The sheer volume of traffic made conventional approaches to interpretation problematic. Luminoso's analytic platform combined Basis Technology's core NLP capabilities, the MIT Media Lab's multi-language Concept Net, and deep learning techniques to capture and understand this social media discussion surrounding the Ebola outbreak and virus. Luminoso uncovered and quantified major drivers of fear about the virus spreading, as well as sources of blame and conspiracy theories. CDC used this understanding to debunk common myths and proactively tailor their messaging to address rising public concerns.

Finally, we aggregate our lessons learned as a set of guidelines for introduction of state-of-the-art NLP and analytics to achieve initial business or mission value within six months, and an agile process to realize additional ROI with incremental improvements. We distinguish the "easy, medium, and hard" problems in current applications of NLP to create smart data, with guidance on leading with the lowest risk techniques early in a project and building toward higher value as competency develops.