DARPA wants to build an AI to find the patterns hidden in global chaos

That most famous characterization of the complexity causality, a butterfly beating its wings and causing a hurricane on the other side of the world, is thought-provoking but ultimately not helpful. What we really need is to look at a hurricane and figure out which butterfly caused it — or perhaps stop it before it takes flight in the first place. DARPA thinks AI should be able to do just that.

A new program at the research agency is aimed at creating a machine learning system that can sift through the innumerable events and pieces of media generated every day and identify any threads of connection or narrative in them. It’s called KAIROS: Knowledge-directed Artificial Intelligence Reasoning Over Schemas.

“Schema” in this case has a very specific meaning. It’s the idea of a basic process humans use to understand the world around them by creating little stories of interlinked events. For instance when you buy something at a store, you know that you generally walk into the store, select an item, bring it to the cashier, who scans it, then you pay in some way, and then leave the store. This “buying something” process is a schema we all recognize, and could of course have schemas within it (selecting a product; payment process) or be part of another schema (gift giving; home cooking).

Although these are easily imagined inside our heads, they’re surprisingly difficult to define formally in such a way that a computer system would be able to understand. They’re familiar to us from long use and understanding, but they’re not immediately obvious or rule-bound, like how an apple will fall downwards from a tree at a constant acceleration.

And the more data there are, the more difficult it is to define. Buying something is comparatively simple, but how do you create a schema for recognizing a cold war, or a bear market? That’s what DARPA wants to look into.

“The process of uncovering relevant connections across mountains of information and the static elements that they underlie requires temporal information and event patterns, which can be difficult to capture at scale with currently available tools and systems,” said DARPA program manager Boyan Onyshkevych in a news release.

KAIROS, the agency said, “aims to develop a semi-automated system capable of identifying and drawing correlations between seemingly unrelated events or data, helping to inform or create broad narratives about the world around us.”

How? Well, they have a general idea but they’re looking for expertise. The problem, they note, is that schemas currently have to be laboriously defined and checked by humans. At that point you might as well inspect the information yourself. So the KAIROS program aims to have the AI teach itself.

At first the system will be limited to ingesting data in massive quantities to build a library of basic schemas. By reading books, watching news reports, and so on it should be able to create a laundry list of suspected schemas, like those mentioned above. It might even get a hint of larger, more hazy schemas that it can’t quite put its virtual finger on — love, racism, income disparity, etc — and how others might fit into them and each other.

Next it will be allowed to look at complex real-world data and attempt to extract events and narratives based on the schemas it has created.

The military and defense applications are fairly obvious: imagine a system that took in all news and social media posts and informed its administrators that it seemed likely there would be a run on banks, or a coup, or a new faction emerging from a declining one. Intelligence officers do their best to perform this task now, and human involvement will almost certainly never cease, but they would likely appreciate a computer companion saying, “there are multiple reports of stockpiling, and these articles on chemical warfare are being shared widely, this could point to rumors of terrorist attack” or the like.

Of course at this point it is all purely theoretical, but that’s why DARPA is looking into it: the agency’s raison d’etre is to turn the theoretical into the practical, or failing that, at least find out why they can’t. Given the extreme simplicity of most AI systems these days it’s hard to imagine one as sophisticated as they clearly want to create. Clearly we have a long way to go.