China’s AI anti-corruption program is getting shut down for being too good

What would you do if you had a machine that could catch a thief, or a corrupt politician? Well, according to Chinese news reports if you were a corrupt Chinese bureaucrat you’d want to ditch it. And as a result, resistance by Chinese government officials to a ground breaking new anti-corruption big data experiment is only one of the many challenges the Chinese government is facing as it tries to roll out increasingly sophisticated technology solutions in order to make its giant bureaucracy more transparent and efficient.

According to China’s state media, there were more than 50 million people on China’s government payroll in 2016, although analysts have put the figure at more than 64 million, which is slightly less than the population of the UK.

To turn this behemoth into an efficient operation befitting the information age a little while ago the Chinese government started adapting different technologies to help them create a “better” government, some of these programs, for example, manifested themselves in the creation of the world’s first Artificial Intelligence (AI) government policy maker, which is now being used by the Foreign Ministry, and the more widely reported slightly dystopian Social Credit System, as well as the world-renowned facial recognition systems that are being used to catch criminals and watch citizens around the clock. Elsewhere the Chinese government is also turning to technology to help them create the world’s first autonomous AI police stations, and track the movements of every policeman, and monitor students attention levels in schools. All of which is just for starters.

In the past President Xi Jinping has repeatedly stressed the necessity of promoting scientific and technological innovations such as big data and AI in “government reform,” with other major government programs including the award of several large contracts to ZTE to develop blockchain technology to prevent the modification of government data by unauthorised people or organisations.

While all these are grand projects, in every sense of the word, one of the greatest challenges to realising their benefits is implementing them on the ground. For example, look no further than China’s anti-corruption AI system dubbed by the researchers working it as the “Zero Trust” programme.

Jointly developed and deployed by the Chinese Academy of Sciences and the Chinese Communist Party’s internal control institutions to monitor, evaluate or intervene in the work and personal life of public servants, the system can access and cross reference information held in more than 150 protected central and local government databases.

According to the people involved in the project this level of access lets the program create “sophisticated, multi-layered social relationship maps that analyse the behaviours of government employees” that have “proved particularly useful in detecting suspicious property transfers, infrastructure construction, land acquisitions and house demolitions,” a researcher said.

But the system is not without its weaknesses, however.

“AI may quickly point out a corrupt official, but it is not very good at explaining the process it has gone through to reach such a conclusion,” the researcher said. “Although it gets it right in most cases, you need a human to work closely with it.”

The system can immediately detect unusual increases in bank savings, for instance, or if there has been a new car purchase or bidding for a government contract under the name of an official or one of his family or friends. And once its suspicions have been raised it will calculate the chances of the action being corrupt. If the result exceeds a set marker, then the authorities are alerted.

One computer scientist involved in the programme who asked not to be named said that at that stage a superior could then contact the person under scrutiny and perhaps help him avoid “going down the road of no return with further, bigger mistakes”.

So far the Zero Trust experiment has been limited to 30 counties and cities, just 1 per cent of the country’s total administrative area. The local governments involved, including the Mayang Miao autonomous county in Hunan province, are located in relatively poor and isolated regions far away from China’s political power centres. Meanwhile, another researcher involved in the programme said the reason behind this roll out plan “was to avoid triggering large-scale resistance among bureaucrats”, especially the most powerful ones, to the use of bots in governance.

Since 2012 Zero Trust has caught 8,721 government employees engaging in misconduct such as embezzlement, abuse of power, misuse of government funds and nepotism. While some were sentenced to prison terms, most were allowed to keep their jobs after being given a warning or minor punishment.

Still though, some local governments, including Mayang county, Huaihua city and Li county in Hunan, are now decommissioning the system, according to the researchers, one of whom said they “may not feel quite comfortable with the new technology”.

None of the local authorities cited responded to requests for comment.

Meanwhile, Zhang Yi, an official at the Commission for Discipline Inspection of the Chinese Communist Party in Ningxiang, Hunan province, said his agency was one of the few still using the system.

“It is not easy … we are under enormous pressure,” he said, insisting that the main purpose of the programme was not to punish officials but to “save them” at an “early stage of corruption”.

“We just use the machine’s result as reference,” Zhang said. “We need to check and verify its validity. The machine cannot pick up the phone and call the person with a problem. The final decision is always made by humans.”

Since Xi rose to power in 2012 more than 1.4 million party members and government employees are estimated to have been disciplined, including leaders like former security tsar Zhou Yongkang and former Chongqing strongman Bo Xilai.

A party disciplinary official in Xiushui county, Jiangxi, who took part in the Zero Trust project said initially government officials in several of the provinces involved in the initial roll out “were unwilling to provide the necessary data, but they usually comply with a bit of pressure,” said the official, who asked not to be named because of the sensitivity of the issue.

Disciplinary officials have been used to help scientists train the machine with their experience and knowledge accumulated from previous cases. For instance, disciplinary officials spent many hours manually tagging unusual phenomena in various types of data sets to teach the machine what to look for.

Some officials might fabricate data, but the machine can compare information from different sources and flag discrepancies. It can even call up satellite images, for instance, to investigate whether the government funding to build a road in a village ended up in the pocket of an official, the researchers said.

The system is still running in Xiushui, but its fate is uncertain. Some officials have questioned the machine’s right of access to sensitive databases because there is neither a law nor regulation authorising a computer or robot to do so.

No wonder the system is being decommissioned by counties and cities that had signed up, and those still using it are facing enormous pressure, with the researchers seeing little or no hope of rolling it out nationwide.

The Zero Trust hump notwithstanding, AI’s foray into other government sectors continues as the Chinese government “is determined to apply cutting-edge technology to its advantage.” AI clerks, for example, have been recruited in some courts to read case files and help judges process lawsuits with higher speed and accuracy.

Last month, a court in Shanghai became the first ever in China to use an AI assistant at a public hearing, Xinhua reported.

The machine, code named “206”, has the ability to record conversations, show evidence such as surveillance camera footage when mentioned by lawyers, and compare testimonies to help judges spot discrepancies, the report said, with one judge being quoted as saying it would reduce the likelihood of a wrong verdict.

Source: SCMP