January 22, 2017

Donald Trump -- why am I not perturbed?

I just read in the newspapers that Donald Trump has been sworn in as 45th President of the United States and so the world is about to come to an end. It is the end of globalisation and the beginning of new protectionist order where the new mantra is buy American and hire American. It is the end of liberal democracy and the beginning of a new Putin-style illiberal autocracy. Francis Fukuyama and his thesis of "end of history" is now history -- "communist" USSR, in its new avatar of plutocratic Russia has eventually triumphed over the Anglo American vision of a new world order.

image footwearnews
But should all this matter to me? Obviously it does, as otherwise why am I commenting on Trump's inauguration? Trump is now the key person in the world's richest and most powerful nation and he certainly has the capacity to both help and hurt me -- or rather my interests. My interests are of course related to Calcutta, India and of late, and still rather tangentially, to the United States. So let me how Trump stacks up in my perception.

To begin with I really cannot quarrel with his philosophy of --  Buy American and Hire American. If I was in his place, or if I was the Prime Minister of India I would have said something identical. I need to look after the people near me, around me and most importantly those who have elected me and so I cannot criticize him if he does exactly the same.

One of the claims of globalisation was that it was a win-win for all parties but this far from true. By exporting good or services from region A to B, we make A richer and B poorer. Win-win is possible if there is a bilateral flow of goods and services. A one way flow of goods, as is the case from China to the US ( and also to India) cannot be a win-win however much we twist and turn our words. Its a win for China and a loss for the other. Similarly, one way loss of jobs from the US to India cannot be win-win. It is a win for India and loss of the US. A win-win is possible if there is a balance in the value that flows across the border in both directions.

As long as the US economy was big enough, when it was far, far bigger than, say, China, India, it could tolerate a loss -- or haemorrhage --  of jobs and wealth across its border but, as anyone who has solved the "water flowing out of tank" series of  problems in Jadav Chandra Chakraverty's classic book on arithmetic would know, there will come a time when the tank will run dry and not all the wishy washy theory on the win-win of globalisation will work. Let us admit it, the globalisation that we have had so far was never really a win-win game, its a zero-sum game. Someone wins and someone loses. As long as Americans were rich enough, they could talk big about globalisation and all that, while we, the poor in India would never allow imports of good and services because we had to save our precious "foreign exchange". But now that the US has been bled dry  -- well you can always debate on how dry or poor the US -- it is impossible for them to be as generous as they once were. Let us face, when push comes to shove, everybody looks after his or her own interest.  So is the case with Donald Trump. [ update ] Wonder how many of us in India would cheer if people from Bangladesh were allowed to come in and start working in our shops and establishments at lower wages?

Now let me see the Trump phenomenon from my narrow, selfish perspective of being an Indian, living in India and being a beneficiary of a buoyant Indian economy. What does the US buy from India? I looked at 2012 data and found that the US accounts for only 25% India's export of US$ 142 billion which again would be around 10% of India's  2012 GDP of US$ 1800 trillion. So if the US stops buying anything from India -- and this is  highly unlikely -- the impact is about 2.5% of India's GDP. Now let us see from the other side and estimate the possibility of the US stopping all imports from India? My data shows that India accounts for only 2% of all US imports! China accounts for 22% of all US imports, while Mexico and Canada accounts for 13% each and so any hostile action towards exporting countries would first be directed towards these three. India's 2% is quite safe and so is our 2.5% of the GDP that finds its way to the US!

So if Trump starts to restrict imports, the one to get hurt the most is China, who of course is no friend of India and so what hurts a non-friend, is of little concern to me.

Now let me turn to "Hire American" part of his agenda and as we all know that this can only impact our H1B "IT professionals" who are in the US. India's IT industry -- of which I have been a participant -- is worth US$ 147billion in 2015, out of a GDP of  about US$ 2000 billion. Of this US$ 100 billion is exports mainly to the US. This is about 5% of the GDP. A lot of this, US$100 billion is in the form of offshore delivery that does not need H1B visas and is delivered digitally out of India. The total number of H1B visa holders in the US is about 800,000 in 2013 and about 65,000 visas are issued every year. Assuming that 65% H1B visa holders are from India this would be only 18% of the 3 million people who are directly employed by the IT industry in India. So assuming that all Indians with H1B visas are booted out of the US, the loss is capped to 18% of the IT industry which in turn is about 5% ( US$ 100B / US$ 2000B) of India's GDP. That is an impact of about 1% assuming -- rather ominously -- that all H1B visa holders from India lose their  jobs and all Indian IT professionals are working on US project. In reality, the impact will be far less, because with at worst, new visas will may not be issued or renewed.

In fact, the threat to the IT industry is not so much from Trump's H1B policies but from automation of services and the reduction in the number of people required to deliver IT services. Most of our IT professionals are rather low end "techies" often referred to as cyber-coolies who have joined the profession, not because of any love or aptitude for the trade but because of the easy money to be made there. As automation eliminates many other blue-collared jobs, so too will these jobs gets eliminated and Trump has no role in that. In fact, if the IT industry in India has to survive, it has to wean itself from the labour arbitrage model of delivering inexpensive services and move into the high end of high value products -- but that is another story that has already been explored, ad nauseum, by many others.

Now that fear of the "Buy from America and Hire from America" is behind us, let us look at the other "grotesque" aspects of the Trump presidency!

We are told that he has no respect for women and is hostile to the LGBT community. Condemnable as that may be, our politicians in India are no better and if we can live with them right here in India, I am sure we can live with Trump and his misogynism in distant America. It is up to the American citizen and the American electorate to deal with him as they think fit. Just as I do not want Americans to interfere with our caste, religious and social issues, I have no interest in getting entangled with their affairs. We have enough problems here of our own making right here. In fact I only hope that unlike Barack Hussein Obama he does not instigate a hypocritical firestorm around the fake issue of "intolerance" in India -- on the instructions of the India's secular enemies.

Which brings us to his attitude towards minorities or to put it rather bluntly towards political Islam.

Hillary Clinton was in the pay of her Saudi Arabian paymasters, which is why she and Obama encouraged the rise of ISIS to counter the Iranian influence in the Middle East in general and Iraq and Syria in particular. The Great Game in the Middle East today is that the new Russia, under Putin, is trying to muscle into traditional Anglo-American turf, from Ukraine to Syria and in this they are opposed by the Saudi led Sunnis and hence assisted by the Iran led Shia. In this Great Game, Clinton was firmly in the Saudi - Sunni camp, hence opposed to the Iran - Shias, and so Russia's Putin threw in his weight behind Donald Trump so that he would win and support, or at least not oppose, Putin's grandiose plans in the Middle East.

How does that impact India?

India is in the crosshairs of political Islam with Ghazwa-i-Hind being a long sought political goal of both the Sunni and the Shia factions, neither of whom have any love lost for the Indic way of life and its inclusive philosophy. However, in the near term, Wahabbi Sunni Islam, funded by Saudi Arabia and provisioned by Pakistan is the more potent and dangerous threat both for India and the world. Hence, the Saudi-funded, Saudi-affiliated Hillary Clinton would have been an utter disaster.

But is Trump a friend of India and of Indic civilisation? Unlikely. His first interest is in himself and then his in cronies. Fortunately he and and his cronies are now allied against political Sunni Islam -- and as long as he remains so, their interests would be aligned to mine and ours.

Finally, Trump is supposed to be illiberal or anti-liberal. Unfortunately, the whole concept of being "liberal", of being "tolerant" of others and of "agreeing to differ" has disappeared. Violent intolerance is the new normal. For example, political Islam does not seek dialogue, only dominance. So if you are tolerant and liberal you are considered weak and liable to be hurt if not actually exterminated. In effect, you can be tolerant only if  you tolerate intolerance and if  you do so you would be eliminated along with your tolerant beliefs. So  you can be liberal and tolerant if and only if you have a death wish! As I have explained in my blog post, and in Swarajya, the Second Law of Thermodynamics tells us that we will inexorably move towards more and more chaos, not just in the world of physics but also in society.

In all this rise of violent intolerance, Trump is neither here, nor there. He is just another milestone in the long march to chaos. So what is there to be unduly alarmed by his ascendancy?

December 25, 2016

Habitats in Space : Our Second Home

If not on Earth, where else can humans beings be? Not as a traveller in a spacecraft or as a one-off tourist on the Moon but as an immigrant, or a colonizer who sets up a permanent residency for multiple generations. Given our current knowledge of physics and the state of propulsion technology, we really do not have the ability to travel to other stars in a reasonable amount of time, so our search for that second home for humanity is restricted to our own solar system. Where do we start?

photo credit Tittat427

There are two divergent points of view on where to locate such an extraterrestrial colony. Some believe that the colony should be located on a natural body like a planet, satellite or an asteroid while others think that the best bet would be an artificial structure in orbit around a planet or in space. There are pros and cons for both. What is common to both approaches is the realisation that any environment outside the terrestrial biosphere that we currently inhabit is inhospitable if not actually hostile to all kinds of known lifeforms, including humans. The availability of a magnetosphere that shields us from cosmic radiation, of liquid water that is fundamental to our biology and of an atmosphere that supplies us with oxygen for metabolism is essential but unfortunately there is no place where all of this is readily available. So wherever we go, we would need to create this environment artificially. Question is, where is it that this can be done with the least effort or at lowest cost.

The obvious place to think of is our Moon because it is closest and we have already been there. But because it is so deficient both in water -- with just traces of this precious liquid found by India’s Chandrayaan -- as well as in gaseous oxygen and nitrogen, that it is not thought to be worth exploring. The only real value of the Moon is as a source of minerals that can be mined and more importantly lifted off the surface relatively inexpensively because of the low gravity compared to Earth.

Mars on the the other hand is a more Earth-like location. The physical terrain is very similar to our rocky arid deserts and while no liquid water is present today, there are huge ice caps in the polar region that if melted can provide all the water that we would ever need. What is nice is that because the axis of rotation of Mars is tilted just like that of Earth, there are regular seasons and while winter temperature at the poles can be as low -150O C, summer temperatures near the equator reach a comfortable 35O C. The atmosphere is thin and the air pressure on the surface is about 0.6% of what we have on Earth. But since most of this is carbon dioxide, there is a good possibility of using terrestrial plants to use sunlight and gradually convert carbon dioxide into precious oxygen that both plants and animals can use. Melting the polar ice caps to release water and gradually converting carbon dioxide into oxygen could be the first steps towards terraforming Mars and creating an Earth like environment. After water and oxygen, the third challenge is the lack of a global magnetic field that would shield residents from cosmic rays but the existence of umbrella shaped auroras in some areas show the promise of local magnetic shields that could prove useful. But in areas bereft of such protection, thick walls or living underground would solve the problem.

While Mars may look like a good place to pitch our extraterrestrial camp, there are three other possible locations.

Titan, the largest satellite of Saturn, is a world where liquid hydrocarbons play the role that water plays on Earth. Titan has a dense atmosphere consisting largely of nitrogen and it rains hydrocarbons, like liquid methane, every now and then. This leads to the  formation of liquid methane lakes near the poles. The climate, along with the winds and the rain, creates surface features very similar to Earth with dunes, lakes, rivers, deltas and oceans that are dominated by seasonal weather patterns. Titan is cold, very cold and far away from the sun and so solar energy may not be a feasible solution. But it has water ice and if the energy potential of the vast quantity of methane -- and that is a big IF  -- can be unlocked with the help of oxygen extracted from water then we have one of  largest sources of hydrocarbon energy in the solar system. Again, even if we do not go to live here, Titan could be be source of energy, like the Middle East on Earth, that could be extracted and transported to other worlds inexpensively, because of Titan’s low gravity.

Next stop is Enceladus, a small rocky world covered with water ice that orbits Saturn. But what is interesting in Enceladus is that even though surface temperatures are far below freezing, there exists beneath the thick ice shell, vast oceans of liquid water that are kept warm by the tidal effects of Saturn’s immense gravity. In fact, so hot is the interior that plumes of salty water are regularly seen to erupt out of the surface in a manner similar to geothermal geysers on Earth. These salt-water plumes also contain nitrogen, as ammonia, and organic molecules like methane, propane, acetylene and formaldehyde that prove the existence of  hydrothermal activity and potential energy resources. In fact, the presence of these products also signal the possible existence of life forms similar to those found on Earth.

Our third and final stop on this current tour of possible locations for our second home is Europa, another ice and rocky satellite that orbits Jupiter. Europa too has a vast, subterranean ocean of liquid water warmed by the immense tidal effects of Jupiter’s gravity and it ejects plumes of water spray similar to the kind found on Enceladus. There are two other features that make Europa possibly one of the most habitable locations both for terrestrial as well as non-terrestrial life forms. First it has a magnetic field caused either by a nickel-iron core or a subterranean pool of salty, liquid water that forms a subsurface conductive layer. Second, it has an atmosphere, which though thin and tenuous consists primarily of oxygen!  This is of immense value for any human colony that may be located there. Because of all these factors, Europa has emerged as one of the most likely locations in our Solar System for potential habitability.

Mars, Titan, Enceladus and Europa all seem to have the basic ingredients necessary to support human life but given the lack of an Earth-like environment, man will have to create an independent structural unit with controlled, ecological life support systems that are isolated from the local environment. If this be the case, why build this structure on a planet or a satellite? Why not build the structure in space itself?  This is the logic behind the design of space habitats that could either be in orbit around planets or around Lagrangian points -- gravity wells where the gravitational attraction of two large astronomical bodies tend to, kind of, cancel each other out. These could be huge structures, many kilometers long and shaped like a cylinder or a torus that spin on an axis to generate the sensation of gravity. Powered by an abundance of solar energy and sustained by recycling materials on a mini-planetary scale, these structures could be sustained by advanced agricultural techniques based on greenhouse and hydroponic technology. Many of these technologies already exist but the biggest challenge would be to source the construction materials. Carrying them from Earth would be impossibly expensive but a more feasible approach would be to mine them from other, low gravity locations. That is where robots will come in very handy because they can be made to mine and transport materials from inhospitable worlds.

Now that we have some options for our second home, some of us will ask the question -- why on Earth, pun intended, would we want to go there? One reason of course could be the deterioration of the political and ecological environment, that may make Earth uninhabitable. There is also the finite, non-zero probability of some incredible accident that may wipe out life on Earth. But perhaps the most biggest compulsion would be man’s curiosity. 200,000 years ago, primitive man walked out of Africa and spread out all over the world. Later on, people from south India crossed the Bay of Bengal to colonise south east Asia and Europeans spilled out into America and then Australia. Each such excursion led to vast increase in exploitable resources. However in each case, in each society, there would have been sceptics who would have wondered why?  But then there must have been others who said “Why not?” and it is the latter who would eventually prevail!

So would be the case for man in space. Why Not?

this article originally appeared in Swarajya, the magazine that reads India right

December 09, 2016

Chidananda Rupaha Shivoham

One of the finest expositions of the Indic School of Thought.

Shivoham (शिवोऽहं ) by Adishankara - performed by John Scottus Students from Rutger Kortenhorst on Vimeo.

November 28, 2016

Engineering India for the Space Age

Elon Musk may have suddenly become the glamourous mascot of a new, post Apollo, space age but the fact remains that the destiny of mankind is among the stars. While many people will keep on trying to salvage what remains of spaceship Earth from the environmental, social and political scourges that threaten it, it is but inevitable that a significant, space faring community will emerge that will seek to live and work outside the planet -- like European emigrants going to America. Are we in India ready for this? Will we just watch and cheer while others depart? -- as it was in 1969 when man went to the Moon? Or should we as nation participate in these great voyages?

credit : gwydion1982
Astronomers tell us that there must be, thousands of habitable, earthlike planets in orbit around various stars in our galaxy. Unfortunately, the distance between stars is so great that we have no means yet, not even theoretical ideas, on how to build spacecraft that will be fast enough to travel to another solar system in the lifetime of a human being. But travel within our own solar system is  possible with technology available today. So to be realistic, let us focus on our own solar neighbourhood.

In a recent paper in Universe Today, Nancy Atkinson has identified Mars, Europa and Titan, two satellites of Saturn and Enceladus, a satellite of Jupiter as possible destinations based on the availability of a solid surface, the presence of an atmosphere to protect against cosmic rays, the existence of water and hydrocarbons and a temperature range that supports liquids. There may or may not be pre-existing life in these worlds but the environment can certainly support the carbon based life forms that we are familiar with. More importantly, we already have the rockets and spaceships that can take us to these destinations in a reasonable amount of time. But going there is only a one part of the story -- a story that is being written with great enthusiasm by NASA, ISRO, SpaceX, Blue Origin and other companies. What do we do when we get there?

In the early years of the 15th century, the Chinese government sent out flotillas -- the Zheng He fleets -- to impress people around the world with the size and power of their ships and the glory of the Chinese empire. At the same time, Portuguese and Spanish sailors, with far smaller ships, also set sail but their aim was to create outposts from where they could carry out trade and commerce. The Chinese ships achieved little even though they were far superior to the European ships but as we know from history, it was the latter that ended up ruling the world -- both politically as well as economically. Space voyages tomorrow would do well to keep in mind that it is not enough to just go there. We need to create habitable colonies where humans can stay and plan for the next part of their journey.

Unfortunately, none of the feasible destinations that we have identified can natively support human life as we know it. The first thing that we need to have is a Hab, a habitat, that we human beings can live in. This Hab must be as large as a big house or a perhaps a small township that has its own oxygen rich atmosphere hermetically sealed against the alien atmosphere of the outside world. But a Hab as big as we need cannot be built on Earth and transported to the colony, it needs to be built there. So before we have a Hab as big as we want, we need a Fab -- a fabrication facility -- at the location, that will be used to build components of the Hab.

The Fab, would in turn have three pre-requisites. First it would need a source of energy and while nuclear power carried from Earth would be the initial source, it would eventually need to have gigantic solar cells to harvest sunlight, however weak it may be, and provide sustainable energy. The huge hydrocarbon deposits on Titan could also be used if we can get a source of oxygen to burn them. The second second requirement of  the Fab would be raw materials to build the Hab and also the solar cells to power both the Fab and the Hab. Again, this raw material cannot be transported from Earth but would have to be mined near the Fab / Hab complex itself. The third facility in the Fab would be autonomous manufacturing facilities -- or what we refer to as smart robots..

Human beings with their fragile lives and the need to protect them from accidental death means that it would be far simpler and inexpensive to use robots as much as possible for all mining and manufacturing activities. This is one area where Indian companies may wish to focus on if they want to remain relevant as mankind ventures out into the vastness of space. Space or not, robotic manufacturing has already become a key feature of the terrestrial economy and unless Indian companies adopt to this technology in a big, big way the Make-In-India concept would be rendered irrelevant. However the focus should not just be on using robots but on building robots as well -- robots must become the centrepiece of India’s capital goods industry. This will create a stable platform on which space faring robots can be built.

Robots have sensors to detect cues from the environment, actuators to move and control materials and other machines and all this is held in place by artificial intelligence (AI) software. This same AI, that is the basis of the autonomous, self driving cars pioneered by Google and developed by Tesla should now be extended to autonomous equipment for mining and material handling that would be essential to feed the Fab at the space colony. This presents another window of opportunity for Indian engineering companies to place a toe-hold in the line-up for the space race. Manufacturers of earthmoving equipment must start exploring ways and means to adapt their products for autonomous operations in different worlds. The first step would be of course to build autonomous products for terrestrial operations and once we have a level of confidence in the technology, we can seek to adapt them for space. For example, seabed mining with autonomous submarines could become a model for mining operations on Enceladus that has huge water oceans under a layer of clean white ice.

Autonomous mining is not science fiction. The Mine of the Future initiative of Rio Tinto and the Next Generation Mining program of BHP have both started working on autonomous mining equipment and multiple Chinese companies are also in the fray. In India too, we must start working on these technologies, not just in preparation for their eventual deployment in space but for benefits that will accrue in the terrestrial economy, today.

Once the minerals have been mined, we would need to convert them into building materials that will allow us to create a series of increasingly bigger and more sophisticated Habs. Materials that are easy to work with and yet good enough to protect human lives from the harshness of the extra terrestrial environment would need to be designed along with the capability for autonomous manufacture and fabrication. This calls for new thinking in materials science and in construction technology that can be automated easily with smart robots.

The final component of any space colony is the farm to grow food. In the movie Martian, which demonstrated some very feasible technologies necessary for a human to survive on Mars, we saw the protagonist growing potatoes in a fairly hostile environment. In a recent experiment at the Wageningen University and Research Centre, “scientists have managed to harvest 10 crops, including tomatoes, peas, and rye, from soil that mimics the conditions on Mars”. Farmbot, a precision farming DIY robot built with off-the-shelf components like Arduino and Raspberry Pi and based on traditional CNC machine tool guidance technology, could be a good model for autonomous farming under tightly controlled environments.

Rockets and spaceships are necessary but not sufficient in our quest for new homesteads beyond Earth. A broad consortium of engineering companies is necessary to provide R:U:M  रोटी, ऊर्जा और मकान -- food, energy and habitat -- on extraterrestrial locations and to build the capital equipment capable of autonomous mining, mineral processing, fabrication and assembly. A community that plans to go to space will need to dream, design and demonstrate these technology capabilities here on Earth. Since these technologies have a direct impact on the current terrestrial economy there is a strong business case for investing in such dual-use products right now. Expertise acquired in these areas is not only good for business today but will also provide the platform for that big leap into space.

A national convention on “Engineering India for the Space Age” would be a good launch pad for  the national goal to become a space faring nation in the next 50 years.

This article was originally published in Swarajya.

November 24, 2016

Smart-wallets to facilitate demonetisation

The demon of demonetisation has scared the living daylights out of India’s cash-based economy. Whether the aam-aadmi is indeed suffering or whether the quantity of black money unearthed justifies the collateral discomfort is best debated on the sterile but bloody timelines of social media, but what we need right now is a way to facilitate payments -- small, ad-hoc payments -- that keep the engines of commerce running. Can technology play a role here?

It certainly can. Let us see how.

First we need to popularise software, app-based wallets like SBI Buddy and PayTM with which people can transfer money to each other merely on the basis of their phone numbers. While prima facie these are good options there are some drawbacks. First these are closed-systems, which means that one can only transfer money to someone who already has an identical wallet software. This means, for example, that a person can transfer money from his SBI buddy to someone who has SBI Buddy but not to someone who has PayTM. This is troublesome but can be overcome by transferring money directly to someone’s bank account but this invites a transaction fee -- 1% for PayTM and an incredible 3% for SBI Buddy. Obviously this is too high a transaction fee and cannot be accepted.

One solution that is already available in India is the Universal Payment Interface (UPI). UPI is not a software but a set of protocols -- let us think of them as specification of plugs -- that allow any software wallet to connect to ( or “plug”-into) any bank and transfer money in a safe and secure manner. See this Swarajya article to know more about it. With UPI,  anyone can transfer money to anyone else irrespective of the bank that is used by the other party. This is like NEFT or RTGS but from a mobile phone and for small amounts of money.  A host of Indian banks have already enabled UPI in their mobile banking software but unfortunately the big daddy of them all -- State Bank of India -- has still not made its software compatible with the UPI specifications. The technology is not at all difficult and it already exists. If the government can convince the SBI to fall in line, then, with its vast network and crores and crores of customers, the UPI platform will cross the tipping point and help make it the default money transfer platform in India.

What stops us? Nothing but the will do so …

image "borrowed" from YouthApps

The other significant technology that we can leverage is Point of Sale (PoS) machines that can be used by anyone with credit or debit cards to accept payment. We already understand that sales of such machines are increasing very fast but there is a both a cost and a time factor involved. These machines cost money and need time to build and ship.

What if we had a software App that one could download and use on an Android / IOS phone or tablet? All that this App would need to do would be accept the card details, including the CVV and the subsequent OTP that the payer would get from his bank on SMS and transfer the value to a designated account. Again this is nothing new. Any eCommerce App -- like MakeMyTrip or Flipkart or BookMyShow is doing the same. The only difference would be that the money will be credited to the bank account of the person who has installed the App, not to the account of the company that builds it!

Square is such an App that is already available but it does not work with Indian bank accounts. Given that the basic architecture of the product is already known, it should not take any time for an Indian company to build the same.

Most of these technology solutions are already available -- all that we need is a small task force of people from RBI, SBI, IBA, NPCI, the Finance Ministry, the Ministry of Electronics and Information Technology, NASSCOM and the CII to sit down and create a roadmap for their implementation in the country. If this can be done on a war footing, we can have an unified solution available in less than a month. But for it to happen, this behemoth must made to move and to do so it needs a push from the Prime Minister himself.

Sceptics would of course crib that such technology solutions will cater only to the elite in the cities and leave the poor in the lurch. That is wrong. The extremely poor people are in any case not impacted because they do not deal with ₹500 or ₹1000 notes. What is happening is that traders and shopkeepers are not able to sell or transact and this is having an impact on the daily labourers they employ. Traders and shopkeeper in the country can surely afford to have a smartphone and it is almost certain that a large number of them already do. The rest of them will certainly get one if they need to run their business.

The final argument is that our traders and shopkeepers want to deal with cash because they do not want to leave an audit trail that can be picked up by the taxman. But now that the die has been cast and they realise that neither they,  nor their competitors, have an option, they will, willy nilly, have to  get onto the digital bandwagon. Because the only other option left for them would be to NOT sell -- and instead, sit, twiddle their thumbs, sulk and curse Modi for unleashing this scourge on them.

A slightly different version of this article was published in Swarajya in the aftermath of the Demonetisation drive unleashed on country.

November 19, 2016

K-means Clustering with SQL

Ever since I started my career in software and IT as a database administrator, I have had a great fascination, if not love, for the SQL language. I also know that there are many in the software fraternity (and sorority) who share my comfort with using SQL. Even after moving into the current enthusiasm, or should I call it a fad, with data analytics, I see that there exists many who prefer the ease and convenience of SQL even in big data.

For example, Hadoop had to hide the complexity of its Map-Reduce magic behind HIVE. Spark offers SparkSQL. In an earlier post, I had shown how SQL can be used for scoring Data Mining models developed with R through the PMML route.

In this post, we show how K-means clustering, a very common and widely used data mining activity can be done with SQL, using a technique adopted from Joni Salonen's blog post. The following codes were executed in MySQL.

First we create the tables :

drop table if exists km_data;      -- contains initial data, only 2 dimensions
drop table if exists km_clusters;  -- contains the final clusters centres
drop table if exists km_steps;     -- shows intermediate steps
create table km_data (id int primary key, cluster_id int, x1 float, x2 float);
create table km_clusters (id int auto_increment primary key, x1 float, x2 float);
create table km_steps (id int, x1 float, x2 float);

Then we create a stored procedure to implement the K-means algorithm

drop procedure if exists kmeans;
CREATE PROCEDURE kmeans(v_K int)
TRUNCATE km_clusters;
truncate km_steps;

-- initialize cluster centers
INSERT INTO km_clusters (x1, x2) SELECT x1, x2 FROM km_data LIMIT v_K;
    insert into km_steps select * from km_clusters;
    -- assign clusters to data points
    UPDATE km_data d SET cluster_id = (SELECT id FROM km_clusters c 
        ORDER BY POW(d.x1-c.x1,2)+POW(d.x2-c.x2,2) ASC LIMIT 1);
    -- calculate new cluster center
    UPDATE km_clusters C, (SELECT cluster_id, 
        AVG(x1) AS x1, AVG(x2) AS x2 
        FROM km_data GROUP BY cluster_id) D 
  SET C.x1=D.x1, C.x2=D.x2 WHERE C.id=D.cluster_id;

This procedure updates tables without a WHERE condition. This is not allowed in the default "Safe Update" mode of MySQL. Go to Preferences -> SQL Queries and uncheck the "Safe Updates" option. Otherwise this procedure will give an error.

Next we load some sample data :

truncate km_data;
insert into km_data (id, cluster_id,x1,x2) values (1,0,185,72);
insert into km_data (id, cluster_id,x1,x2) values (2,0,170,56);
insert into km_data (id, cluster_id,x1,x2) values (3,0,168,60);
insert into km_data (id, cluster_id,x1,x2) values (4,0,179,68);
insert into km_data (id, cluster_id,x1,x2) values (5,0,182,72);
insert into km_data (id, cluster_id,x1,x2) values (6,0,188,77);

insert into km_data (id, cluster_id,x1,x2) values (7,0,180,71);
insert into km_data (id, cluster_id,x1,x2) values (8,0,180,70);
insert into km_data (id, cluster_id,x1,x2) values (9,0,183,84);
insert into km_data (id, cluster_id,x1,x2) values (10,0,180,88);
insert into km_data (id, cluster_id,x1,x2) values (11,0,180,67);
insert into km_data (id, cluster_id,x1,x2) values (12,0,177,76);

Finally we call procedure  with the parameter = 2, the expected number of clusters

call kmeans(2);

That is it! As simple as that ..

Of course, one may always wonder why anyone would use SQL for this when R and Python offers possible better functions, but then one sometimes does things just for the heck of it and that is what we have done here.

November 02, 2016

Thriving on the Chaos [Monkey]

As a long time foot-soldier in India’s software industry, I had often wondered why is it that India’s big brand IT companies with literally lakhs of software developers could never, ever come out with a single blockbuster product like Whatsapp, or Skype. It could not be that Indians were dumb -- after all our very own Sabeer Bhatia from BITS Pilani had created Hotmail, precursor to Yahoo and Gmail, but why was it that he had to go the Silicon Valley to be able to do it? What is there in the air and water of California that it causes products to sprout in every other company there?

Chaos Monkeys by Antonio Garcia Martinez (Penguin Randomhouse 2016) talks about the factors, the issues, that seethe in this cauldron of chaotic creativity -- issues that managers in Indian software companies, thrashing around in business-as-usual, BAU, challenges of bench, attrition and utilisation will never know. This 500 page corporate thriller takes the reader right into the frontlines of Silicon Valley as it narrates how the author quits an old economy job at Goldman Sachs in New York to join a start-up in California, quits the start-up to start his own start-up, which he then sells to Twitter, while bailing himself out to join Facebook and then gets sacked after falling foul of Zuckerberg’s inner circle of cronies even though he had delivered an excellent product -- and all this in just five years, 2008 - 2013.

Chaos Monkeys stands out from the general run-of-the-mill management books because it contains nuggets of hard facts and searing insights embedded in the actual story of software product development. For example, “What’s it take to do startups? It certainly isn’t intelligence .. It certainly isn’t technical skill .. It’s not unique product or market vision … [It’s] First, the ability  to monomaniacally and obsessively focus one thing … at the expense of everything else [including family ]… Second, the ability to take and endure an endless amount of shit.”

Martinez has a way of pulling back the covers to reveal how the legends of Silicon Valley are no different from many old-world ancients and in this, two examples stand out. First, as he listens to Zuckerberg rallying his troops before the Facebook IPO, he realises that “The People’s Republic of Facebook is no different from National Socialism, .. , Bolshevism, Islamic State, Crusades … because it depends on the founders, or revolutionaries narcissistic will to power and the mass man’s desire to be a part of something bigger than himself”. What is worse is that like medieval serfs, employees cannot walk away from the tech industry because they are bound by chains of loans to an expensive lifestyle that is unaffordable elsewhere. Second, both Facebook and Goldman Sachs, the two companies where he spent the most time, “would always aim to create monopoly pricing power and maintain information asymmetry rather than drive true innovation. If [they] played with the outside world, it was always played with the loaded dice.”

In August two partners from Y Combinator (YC) touched down in Bangalore to take a look at start-ups here. This is the same company that helped propel the author’s start-up right from the idea stage, through seed funding, defending against hostile court cases to its eventual “sale” to Facebook and Twitter as described in very great detail in this book. Those who have an interest in the start-up ecosystem in India will find this book to be an invaluable manual on how companies like YC operate and one that guides them the past the treacherous shoals and rapids. Software professionals in India who have only seen giant companies like Infosys and TCS delivering back end services with hordes of, what is unkindly referred to as, white-collar coolies, will understand what it means and what it takes to deliver high value, high margin products.

Since the author’s product was all about online ads, another important takeaway from this book is a detailed explanation of how online advertisements are created, targeted, auctioned and delivered on the internet. Eventually, it is these ads, that pay for all the free goodies that we get from the giant internet companies and the process is not fundamentally different from the trading of securities that the traditional economy is so familiar with.

Net-net, a book that is dense in detail and yet a pleasure to read. If you have an interest in the new age economy based on the internet, suggest that  you grab a copy and read it. It will be well worth the time.

This book review had originally appeared in Swarajya, the magazine that reads India right

About This Blog

  © Blogger template 'External' by Ourblogtemplates.com 2008

Back to TOP