Autonomous AI could create an autonomous cyber warfare System of Systems

In the future it’s increasingly likely that adequate security against cyber attacks will only be possible through the development of autonomous Artificial Intelligence (AI) cyber defense systems, along the lines of an early prototype that I highlighted years ago. Part of this train of thought is down to the fact that today companies like JP Morgan say they’re seeing over 45 Billion cyber attacks a day, so obviously the ability of humans to keep up with this rate of attack – and even greater rates in the future - is limited and simply unfeasible.

Therefore over the longer term it’s increasingly believed that the human behind a screen in a Security Operations Centre (SOC) will be augmented by and probably replaced in no small way by AI systems that think and act autonomously, as we’ve already seen the start of, and are able to detect and repel attacks early to protect our digital infrastructure.

The future of Cyber Attacks, by keynote Matthew Griffin

In the latest research into this space TNO, a scientific research organisation from the Netherlands, envisages an autonomous AI cyber defense “system of systems” in which AI is the driving force, and they recently wrote a position paper on the subject.

“In the race against malicious actors in the digital domain, humans are a complicating factor. In Security Operations Centres, where computer and network activity in an organisation is monitored, it’s still mainly people who are at the controls. But there’s so much coming at them that we have to accept that security systems are going to become autonomous,” the paper says.

This goes a step further than automation, because autonomy means that, step by step, machines are going to be taking over people’s assessment tasks to a great extent,’ says Berry Vetjens, Market Director ICT, Strategy, and Policy at TNO.

The response devised by TNO to the increasing number and intensity of cyber attacks has been dubbed ‘Athena.’ It is a self-learning and self-repairing system that can perform a wide range of security tasks simultaneously.

Take, for example, a ransomware attack on a large energy company that invades its internal systems and threatens to bring down not only its operations, but also the energy supply to hundreds of thousands of customers.

The planned Athena system would detect the intrusion at a very early stage; assess at lightning speed what measures are most effective to protect the company, customers, and other affected parties; and succeed in repelling the attack without the customer even noticing, while the business processes remained intact.

‘Many people are worried by the idea of handing over ever more human thoughts and actions to autonomous systems by means of AI, but the uncomfortable truth is that not doing so is perhaps even more worrying specifically when it comes to cybersecurity,’ says Berry.

‘Large companies with critical infrastructure are becoming aware that autonomous cyber security is inevitable. It may start with the frontrunners in cyber security, such as financial institutions, but soon it will also be used in other sectors. These will not only be vulnerable sectors such as healthcare, high tech, the manufacturing industry, national and local government, but – in time – SMEs as well.

“Systems that no longer involve humans are on their way and it’s time for us to work with scientific partners, industry, and government to design these systems. There’s no time to lose,” the paper adds.

“We chose the term checkmate with a question mark as the title of the paper because it’s an extremely complex game against invisible and often elusive opponents. You’re playing chess against computers, so to speak,” says Vetjens.

Twenty years after Deep Blue, IBM’s chess computer that played against Kasparov, Google subsidiary DeepMind launched the chess engine AlphaZero, delivering unprecedented performance through the application of AI and machine learning. That said, programming chess and other games is easy in a way because the rules of the game are laid down. In cyber security, the rules keep changing during the game and our opponents constantly flout the rules. That’s what makes it all so complex.’

Going further than has ever been imagined

Cyber security operators are like chess players in that it all comes down to speed, the ability to process ever-increasing amounts of dynamic data from a growing number of sources, and precision. The huge amount of data needed to detect an attack in time and the speed required to respond appropriately have now become far too much for the human brain to handle.

“Athena will be revolutionary. With it, we’re going further than we ever imagined we would. We’ve always had the idea that automation would involve joint control by humans and computers. “Human in the loop” is making way for “human before the loop”. Humans design the system, which then thinks and acts on its own. However, this must be done with the utmost care,” Berry says.

According to Berry, the process is about much more than technology alone, complicated as that already is. From an ethical point of view, the system must meet high values laid down in consultation between experts, policymakers, and stakeholders. This must also involve accountability. Processing a lot of data affects people’s privacy.

There must be legally binding agreements on this. Societally, there are issues of accessibility for SMEs, healthcare, and local authorities. It must be ensured that it is not only large companies that can afford autonomous cyber security. The question of who will be assigned responsibility for what still requires much discussion as well.

“Designing such a system of systems is incredibly complex. You have to combine different technologies related to cyber security and AI, address legal and ethical issues, and reconcile the interests of diverse public and private parties. With our knowledge in all these areas, we want to give clear direction to this process in the coming period,” Berry concludes.