Deepfakes and AI generated disinformation confuse AI trading systems

The other week a deepfake picture of an apparent explosion at the Pentagon sent the S&P 500 down by 0.3% as Artificial Intelligence (AI) quantitative traders, or Quants as they are known, quickly ditched stocks in response.

While trading firms’ algorithms have grown increasingly likely to filter misinformation, the threat landscape is growing into a high-stakes cat-and-mouse game that few anticipate solutions to in the near term.

Predicted: The Future of Trust and Quants 2017 Keynote, by keynote Matthew Griffin

RavenPack data scientist Peter Hafez says quants have two immediate concerns: “We see quants facing two hurdles: fake images that may fool a journalist, and reports of fake images that fool the algorithm itself.”

Combating deepfakes at scale will likely require significant technological investments to dampen trading mayhem, especially for high-speed trading firms like quants that make their profits through minute differences in the prices of different financial instruments.

In the meantime, some quants are turning to data sources that aggregate news from different sources into a sentiment score - which can be just as risky if they aggregate together lots of fake news. And then others are trading using trends rather than sharp price movements driven by social media and current news stories, and others are also changing algorithms to cross-check sources for validity.

A recent UN report described how private and public sector collusion could drive disinformation campaigns to achieve political or financial agendas and specifically alerted the public to how AI-generated deepfakes can stoke civil unrest.

According to the University of Buffalo Media Forensics Lab, generating a deepfake only requires about 500 images or ten seconds of video. However, MIT Technology Review argues the prevailing view is that AI works better with more data.

OpenAI’s GPT-2 model used 40 Gigabytes, while GPT-3 used 570 Gigabytes. The firm has not disclosed how much data its GPT-4 model uses but experts believe it’s in the 2 Petabyte range.

More data can also potentially generate more realistic deepfakes. A recent European Union draft bill asks companies to reveal the data sources their AI algorithms use to generate content.

But, as the problem of deepfakes – especially as it relates to criminals using them to manipulate markets and for financial gain – grows Buffalo’s forensics lab detects deepfakes by identifying abnormal eye movements while Intel’s FakeCatcher tool can analyse human faces to detect authentic human blood flow.

Google’s forthcoming metadata and watermarking tools can also help identify synthetic content. Metadata appended to a file provides context for the content while watermarking embeds resilient seed data that can survive modest edits.