January 23, 2014

Fighting Corruption without Anarchy : Big Data Analytics and RTI-2

Ever since Mohandas G dreamt up the concept of civil disobedience, Indian politicians believe that bandhs, dharnas, rail-rokos, chakka-jams and other anarchic activities are the only tools available to do anything for the welfare of the people. Even educated people like Kejriwal and Jyoti Basu (may he NEVER rest in peace for the damage that he did to West Bengal ) have taken the easy way of out by creating a ruckus on the road to get their point of view across.

Let us see if there is a sane way to fight corruption in this country.

We all know that the basic rules of checks and balances are all there. Approvals, authorizations and the inevitable CAG audit followed by a CBI investigation should have been enough to stop corruption but when the "policeman is himself the thief" then all these checks and balances go right out of the window and are of no use anymore. So what else can be done ?

"Sunlight is the best disinfectant," is a well known quote from the US Supreme Court Justice Louis Brandeis that refers to the benefits of openness and transparency in public life and the RTI Act, passed by the much maligned Congress party in 2005, could have been a game changer in this respect. Under this Act, any citizen of India can request information from a public body and ideally this could / should have been used to scrutinise and sanitize public finances. Unfortunately, the law has been used, abused and misused by both requesters and suppliers of information in a variety of ways so that its efficacy has been largely blunted. While I do not wish to go into the details of the actual lacunae, there are two areas where the abuse has been quite significant. First, people use RTI to ask for junk information or for information that will help them in personal or corporate financial deals. Secondly the information is very often given out by the government agency as reams of paper that cannot be examined to any great detail, nor corroborated or compared with other similar information. So no meaningful analysis is possible.

In an earlier post on how RTI data can used to curb corruption I had tried to describe a mechanism by which every government agency would be mandated to publish its audited financial data on the internet in XBRL format without having to be told to do so under the RTI Act. Then we could have had computer programs ( or "bots") study the reports automatically and point out deviations and discrepancies which the government would have to explain under the formal RTI process.

Today, I am thinking of and advocating something even simpler.

At any point where there is a movement of money from private to public ( or government ) hand there must be some kind of a challan or document generated. Similarly, whenever there is a payment of public money to a private person or company there must be a similar document being generated and signed by someone. In either case, this document is available inside some government premise and in the custody of some government official -- who can be employee of the central, state, municipal, panchayat or some other government body.

Let us demand that the contents of each and every such document, without exception,  be compulsorily made available on the public internet within 7 days of the money changing hands. This means that we will have a public record of all, repeat ALL, movements of public money both inward and outward from the government coffers. However in the interest of privacy and security we could sanitize the data by removing identities of certain payees ( like Income tax payments, refunds ) or block data on sensitive matters like strategic items in defence ( though not the purchase of soap and socks for jawans ). However the vast number of payments that the government makes to employees, vendors, contractors, NGOs etc must compulsorily be made available -- without the need for a formal RTI demand.

Such a project while vast in scope is no more difficult than building the Golden Quadrilateral under NHAI, the UIDAI project or even the Pulse Polio program. In fact since it is only dealing with data lying  inside government premises, and in the custody of a government employee, it can be done far more easily and that to in a phased manner.

Once we have the data, now what ?

Today we have powerful tools for handling Big Data -- Hadoop, Map Reduce and many others -- with which it should be quite simple to create a complete trail of where money is entering the system, how it is flowing across the system and how it is being finally disbursed. For this, there is no need for the government to put in any formal effort because many well meaning RTI activists can use these data analytics tools to locate and pinpoint anomalies and discrepancies in the movement of money and then focus on these areas with more specific RTI queries.

this lovely image of the flow of Big Data is taken from "New Value Comes from Analyzing Diverse Data"

Initially, the numbers may not add up as payments may not equal receipts but then the government must explain the gap in terms of known economic tools like deficit financing, market borrowing and others. Again this could be an ongoing process with the gaps being closed with the passage of time.

Bribes will of course flow outside the system. Which means that money changing hands in 2G or CWG scams will not be trackable here. But what can be spotted is the effect of the  bribes ! When abnormally high payments are made for commonplace items, when thousands of fake recipients get NREGA money, when too much money is being suddenly funneled through specific individuals or offices then deviations from accepted norms will happen and standard data anlytics tools can be used to spot these needles in the vast haystack of government finances.

But that will only happen if government finances are compulsorily made available to the public. That should be the goal of RTI-2