December 09, 2016

Chidananda Rupaha Shivoham

One of the finest expositions of the Indic School of Thought.

Shivoham (शिवोऽहं ) by Adishankara - performed by John Scottus Students from Rutger Kortenhorst on Vimeo.

November 28, 2016

Engineering India for the Space Age

Elon Musk may have suddenly become the glamourous mascot of a new, post Apollo, space age but the fact remains that the destiny of mankind is among the stars. While many people will keep on trying to salvage what remains of spaceship Earth from the environmental, social and political scourges that threaten it, it is but inevitable that a significant, space faring community will emerge that will seek to live and work outside the planet -- like European emigrants going to America. Are we in India ready for this? Will we just watch and cheer while others depart? -- as it was in 1969 when man went to the Moon? Or should we as nation participate in these great voyages?

credit : gwydion1982
Astronomers tell us that there must be, thousands of habitable, earthlike planets in orbit around various stars in our galaxy. Unfortunately, the distance between stars is so great that we have no means yet, not even theoretical ideas, on how to build spacecraft that will be fast enough to travel to another solar system in the lifetime of a human being. But travel within our own solar system is  possible with technology available today. So to be realistic, let us focus on our own solar neighbourhood.

In a recent paper in Universe Today, Nancy Atkinson has identified Mars, Europa and Titan, two satellites of Saturn and Enceladus, a satellite of Jupiter as possible destinations based on the availability of a solid surface, the presence of an atmosphere to protect against cosmic rays, the existence of water and hydrocarbons and a temperature range that supports liquids. There may or may not be pre-existing life in these worlds but the environment can certainly support the carbon based life forms that we are familiar with. More importantly, we already have the rockets and spaceships that can take us to these destinations in a reasonable amount of time. But going there is only a one part of the story -- a story that is being written with great enthusiasm by NASA, ISRO, SpaceX, Blue Origin and other companies. What do we do when we get there?

In the early years of the 15th century, the Chinese government sent out flotillas -- the Zheng He fleets -- to impress people around the world with the size and power of their ships and the glory of the Chinese empire. At the same time, Portuguese and Spanish sailors, with far smaller ships, also set sail but their aim was to create outposts from where they could carry out trade and commerce. The Chinese ships achieved little even though they were far superior to the European ships but as we know from history, it was the latter that ended up ruling the world -- both politically as well as economically. Space voyages tomorrow would do well to keep in mind that it is not enough to just go there. We need to create habitable colonies where humans can stay and plan for the next part of their journey.

Unfortunately, none of the feasible destinations that we have identified can natively support human life as we know it. The first thing that we need to have is a Hab, a habitat, that we human beings can live in. This Hab must be as large as a big house or a perhaps a small township that has its own oxygen rich atmosphere hermetically sealed against the alien atmosphere of the outside world. But a Hab as big as we need cannot be built on Earth and transported to the colony, it needs to be built there. So before we have a Hab as big as we want, we need a Fab -- a fabrication facility -- at the location, that will be used to build components of the Hab.

The Fab, would in turn have three pre-requisites. First it would need a source of energy and while nuclear power carried from Earth would be the initial source, it would eventually need to have gigantic solar cells to harvest sunlight, however weak it may be, and provide sustainable energy. The huge hydrocarbon deposits on Titan could also be used if we can get a source of oxygen to burn them. The second second requirement of  the Fab would be raw materials to build the Hab and also the solar cells to power both the Fab and the Hab. Again, this raw material cannot be transported from Earth but would have to be mined near the Fab / Hab complex itself. The third facility in the Fab would be autonomous manufacturing facilities -- or what we refer to as smart robots..

Human beings with their fragile lives and the need to protect them from accidental death means that it would be far simpler and inexpensive to use robots as much as possible for all mining and manufacturing activities. This is one area where Indian companies may wish to focus on if they want to remain relevant as mankind ventures out into the vastness of space. Space or not, robotic manufacturing has already become a key feature of the terrestrial economy and unless Indian companies adopt to this technology in a big, big way the Make-In-India concept would be rendered irrelevant. However the focus should not just be on using robots but on building robots as well -- robots must become the centrepiece of India’s capital goods industry. This will create a stable platform on which space faring robots can be built.

Robots have sensors to detect cues from the environment, actuators to move and control materials and other machines and all this is held in place by artificial intelligence (AI) software. This same AI, that is the basis of the autonomous, self driving cars pioneered by Google and developed by Tesla should now be extended to autonomous equipment for mining and material handling that would be essential to feed the Fab at the space colony. This presents another window of opportunity for Indian engineering companies to place a toe-hold in the line-up for the space race. Manufacturers of earthmoving equipment must start exploring ways and means to adapt their products for autonomous operations in different worlds. The first step would be of course to build autonomous products for terrestrial operations and once we have a level of confidence in the technology, we can seek to adapt them for space. For example, seabed mining with autonomous submarines could become a model for mining operations on Enceladus that has huge water oceans under a layer of clean white ice.

Autonomous mining is not science fiction. The Mine of the Future initiative of Rio Tinto and the Next Generation Mining program of BHP have both started working on autonomous mining equipment and multiple Chinese companies are also in the fray. In India too, we must start working on these technologies, not just in preparation for their eventual deployment in space but for benefits that will accrue in the terrestrial economy, today.

Once the minerals have been mined, we would need to convert them into building materials that will allow us to create a series of increasingly bigger and more sophisticated Habs. Materials that are easy to work with and yet good enough to protect human lives from the harshness of the extra terrestrial environment would need to be designed along with the capability for autonomous manufacture and fabrication. This calls for new thinking in materials science and in construction technology that can be automated easily with smart robots.

The final component of any space colony is the farm to grow food. In the movie Martian, which demonstrated some very feasible technologies necessary for a human to survive on Mars, we saw the protagonist growing potatoes in a fairly hostile environment. In a recent experiment at the Wageningen University and Research Centre, “scientists have managed to harvest 10 crops, including tomatoes, peas, and rye, from soil that mimics the conditions on Mars”. Farmbot, a precision farming DIY robot built with off-the-shelf components like Arduino and Raspberry Pi and based on traditional CNC machine tool guidance technology, could be a good model for autonomous farming under tightly controlled environments.

Rockets and spaceships are necessary but not sufficient in our quest for new homesteads beyond Earth. A broad consortium of engineering companies is necessary to provide R:U:M  रोटी, ऊर्जा और मकान -- food, energy and habitat -- on extraterrestrial locations and to build the capital equipment capable of autonomous mining, mineral processing, fabrication and assembly. A community that plans to go to space will need to dream, design and demonstrate these technology capabilities here on Earth. Since these technologies have a direct impact on the current terrestrial economy there is a strong business case for investing in such dual-use products right now. Expertise acquired in these areas is not only good for business today but will also provide the platform for that big leap into space.

A national convention on “Engineering India for the Space Age” would be a good launch pad for  the national goal to become a space faring nation in the next 50 years.

This article was originally published in Swarajya.

November 24, 2016

Smart-wallets to facilitate demonetisation

The demon of demonetisation has scared the living daylights out of India’s cash-based economy. Whether the aam-aadmi is indeed suffering or whether the quantity of black money unearthed justifies the collateral discomfort is best debated on the sterile but bloody timelines of social media, but what we need right now is a way to facilitate payments -- small, ad-hoc payments -- that keep the engines of commerce running. Can technology play a role here?

It certainly can. Let us see how.

First we need to popularise software, app-based wallets like SBI Buddy and PayTM with which people can transfer money to each other merely on the basis of their phone numbers. While prima facie these are good options there are some drawbacks. First these are closed-systems, which means that one can only transfer money to someone who already has an identical wallet software. This means, for example, that a person can transfer money from his SBI buddy to someone who has SBI Buddy but not to someone who has PayTM. This is troublesome but can be overcome by transferring money directly to someone’s bank account but this invites a transaction fee -- 1% for PayTM and an incredible 3% for SBI Buddy. Obviously this is too high a transaction fee and cannot be accepted.

One solution that is already available in India is the Universal Payment Interface (UPI). UPI is not a software but a set of protocols -- let us think of them as specification of plugs -- that allow any software wallet to connect to ( or “plug”-into) any bank and transfer money in a safe and secure manner. See this Swarajya article to know more about it. With UPI,  anyone can transfer money to anyone else irrespective of the bank that is used by the other party. This is like NEFT or RTGS but from a mobile phone and for small amounts of money.  A host of Indian banks have already enabled UPI in their mobile banking software but unfortunately the big daddy of them all -- State Bank of India -- has still not made its software compatible with the UPI specifications. The technology is not at all difficult and it already exists. If the government can convince the SBI to fall in line, then, with its vast network and crores and crores of customers, the UPI platform will cross the tipping point and help make it the default money transfer platform in India.

What stops us? Nothing but the will do so …

image "borrowed" from YouthApps

The other significant technology that we can leverage is Point of Sale (PoS) machines that can be used by anyone with credit or debit cards to accept payment. We already understand that sales of such machines are increasing very fast but there is a both a cost and a time factor involved. These machines cost money and need time to build and ship.

What if we had a software App that one could download and use on an Android / IOS phone or tablet? All that this App would need to do would be accept the card details, including the CVV and the subsequent OTP that the payer would get from his bank on SMS and transfer the value to a designated account. Again this is nothing new. Any eCommerce App -- like MakeMyTrip or Flipkart or BookMyShow is doing the same. The only difference would be that the money will be credited to the bank account of the person who has installed the App, not to the account of the company that builds it!

Square is such an App that is already available but it does not work with Indian bank accounts. Given that the basic architecture of the product is already known, it should not take any time for an Indian company to build the same.

Most of these technology solutions are already available -- all that we need is a small task force of people from RBI, SBI, IBA, NPCI, the Finance Ministry, the Ministry of Electronics and Information Technology, NASSCOM and the CII to sit down and create a roadmap for their implementation in the country. If this can be done on a war footing, we can have an unified solution available in less than a month. But for it to happen, this behemoth must made to move and to do so it needs a push from the Prime Minister himself.

Sceptics would of course crib that such technology solutions will cater only to the elite in the cities and leave the poor in the lurch. That is wrong. The extremely poor people are in any case not impacted because they do not deal with ₹500 or ₹1000 notes. What is happening is that traders and shopkeepers are not able to sell or transact and this is having an impact on the daily labourers they employ. Traders and shopkeeper in the country can surely afford to have a smartphone and it is almost certain that a large number of them already do. The rest of them will certainly get one if they need to run their business.

The final argument is that our traders and shopkeepers want to deal with cash because they do not want to leave an audit trail that can be picked up by the taxman. But now that the die has been cast and they realise that neither they,  nor their competitors, have an option, they will, willy nilly, have to  get onto the digital bandwagon. Because the only other option left for them would be to NOT sell -- and instead, sit, twiddle their thumbs, sulk and curse Modi for unleashing this scourge on them.

A slightly different version of this article was published in Swarajya in the aftermath of the Demonetisation drive unleashed on country.

November 19, 2016

K-means Clustering with SQL

Ever since I started my career in software and IT as a database administrator, I have had a great fascination, if not love, for the SQL language. I also know that there are many in the software fraternity (and sorority) who share my comfort with using SQL. Even after moving into the current enthusiasm, or should I call it a fad, with data analytics, I see that there exists many who prefer the ease and convenience of SQL even in big data.

For example, Hadoop had to hide the complexity of its Map-Reduce magic behind HIVE. Spark offers SparkSQL. In an earlier post, I had shown how SQL can be used for scoring Data Mining models developed with R through the PMML route.

In this post, we show how K-means clustering, a very common and widely used data mining activity can be done with SQL, using a technique adopted from Joni Salonen's blog post. The following codes were executed in MySQL.

First we create the tables :

drop table if exists km_data;      -- contains initial data, only 2 dimensions
drop table if exists km_clusters;  -- contains the final clusters centres
drop table if exists km_steps;     -- shows intermediate steps
create table km_data (id int primary key, cluster_id int, x1 float, x2 float);
create table km_clusters (id int auto_increment primary key, x1 float, x2 float);
create table km_steps (id int, x1 float, x2 float);

Then we create a stored procedure to implement the K-means algorithm

drop procedure if exists kmeans;
CREATE PROCEDURE kmeans(v_K int)
TRUNCATE km_clusters;
truncate km_steps;

-- initialize cluster centers
INSERT INTO km_clusters (x1, x2) SELECT x1, x2 FROM km_data LIMIT v_K;
    insert into km_steps select * from km_clusters;
    -- assign clusters to data points
    UPDATE km_data d SET cluster_id = (SELECT id FROM km_clusters c 
        ORDER BY POW(d.x1-c.x1,2)+POW(d.x2-c.x2,2) ASC LIMIT 1);
    -- calculate new cluster center
    UPDATE km_clusters C, (SELECT cluster_id, 
        AVG(x1) AS x1, AVG(x2) AS x2 
        FROM km_data GROUP BY cluster_id) D 
  SET C.x1=D.x1, C.x2=D.x2 WHERE;

This procedure updates tables without a WHERE condition. This is not allowed in the default "Safe Update" mode of MySQL. Go to Preferences -> SQL Queries and uncheck the "Safe Updates" option. Otherwise this procedure will give an error.

Next we load some sample data :

truncate km_data;
insert into km_data (id, cluster_id,x1,x2) values (1,0,185,72);
insert into km_data (id, cluster_id,x1,x2) values (2,0,170,56);
insert into km_data (id, cluster_id,x1,x2) values (3,0,168,60);
insert into km_data (id, cluster_id,x1,x2) values (4,0,179,68);
insert into km_data (id, cluster_id,x1,x2) values (5,0,182,72);
insert into km_data (id, cluster_id,x1,x2) values (6,0,188,77);

insert into km_data (id, cluster_id,x1,x2) values (7,0,180,71);
insert into km_data (id, cluster_id,x1,x2) values (8,0,180,70);
insert into km_data (id, cluster_id,x1,x2) values (9,0,183,84);
insert into km_data (id, cluster_id,x1,x2) values (10,0,180,88);
insert into km_data (id, cluster_id,x1,x2) values (11,0,180,67);
insert into km_data (id, cluster_id,x1,x2) values (12,0,177,76);

Finally we call procedure  with the parameter = 2, the expected number of clusters

call kmeans(2);

That is it! As simple as that ..

Of course, one may always wonder why anyone would use SQL for this when R and Python offers possible better functions, but then one sometimes does things just for the heck of it and that is what we have done here.

November 02, 2016

Thriving on the Chaos [Monkey]

As a long time foot-soldier in India’s software industry, I had often wondered why is it that India’s big brand IT companies with literally lakhs of software developers could never, ever come out with a single blockbuster product like Whatsapp, or Skype. It could not be that Indians were dumb -- after all our very own Sabeer Bhatia from BITS Pilani had created Hotmail, precursor to Yahoo and Gmail, but why was it that he had to go the Silicon Valley to be able to do it? What is there in the air and water of California that it causes products to sprout in every other company there?

Chaos Monkeys by Antonio Garcia Martinez (Penguin Randomhouse 2016) talks about the factors, the issues, that seethe in this cauldron of chaotic creativity -- issues that managers in Indian software companies, thrashing around in business-as-usual, BAU, challenges of bench, attrition and utilisation will never know. This 500 page corporate thriller takes the reader right into the frontlines of Silicon Valley as it narrates how the author quits an old economy job at Goldman Sachs in New York to join a start-up in California, quits the start-up to start his own start-up, which he then sells to Twitter, while bailing himself out to join Facebook and then gets sacked after falling foul of Zuckerberg’s inner circle of cronies even though he had delivered an excellent product -- and all this in just five years, 2008 - 2013.

Chaos Monkeys stands out from the general run-of-the-mill management books because it contains nuggets of hard facts and searing insights embedded in the actual story of software product development. For example, “What’s it take to do startups? It certainly isn’t intelligence .. It certainly isn’t technical skill .. It’s not unique product or market vision … [It’s] First, the ability  to monomaniacally and obsessively focus one thing … at the expense of everything else [including family ]… Second, the ability to take and endure an endless amount of shit.”

Martinez has a way of pulling back the covers to reveal how the legends of Silicon Valley are no different from many old-world ancients and in this, two examples stand out. First, as he listens to Zuckerberg rallying his troops before the Facebook IPO, he realises that “The People’s Republic of Facebook is no different from National Socialism, .. , Bolshevism, Islamic State, Crusades … because it depends on the founders, or revolutionaries narcissistic will to power and the mass man’s desire to be a part of something bigger than himself”. What is worse is that like medieval serfs, employees cannot walk away from the tech industry because they are bound by chains of loans to an expensive lifestyle that is unaffordable elsewhere. Second, both Facebook and Goldman Sachs, the two companies where he spent the most time, “would always aim to create monopoly pricing power and maintain information asymmetry rather than drive true innovation. If [they] played with the outside world, it was always played with the loaded dice.”

In August two partners from Y Combinator (YC) touched down in Bangalore to take a look at start-ups here. This is the same company that helped propel the author’s start-up right from the idea stage, through seed funding, defending against hostile court cases to its eventual “sale” to Facebook and Twitter as described in very great detail in this book. Those who have an interest in the start-up ecosystem in India will find this book to be an invaluable manual on how companies like YC operate and one that guides them the past the treacherous shoals and rapids. Software professionals in India who have only seen giant companies like Infosys and TCS delivering back end services with hordes of, what is unkindly referred to as, white-collar coolies, will understand what it means and what it takes to deliver high value, high margin products.

Since the author’s product was all about online ads, another important takeaway from this book is a detailed explanation of how online advertisements are created, targeted, auctioned and delivered on the internet. Eventually, it is these ads, that pay for all the free goodies that we get from the giant internet companies and the process is not fundamentally different from the trading of securities that the traditional economy is so familiar with.

Net-net, a book that is dense in detail and yet a pleasure to read. If you have an interest in the new age economy based on the internet, suggest that  you grab a copy and read it. It will be well worth the time.

This book review had originally appeared in Swarajya, the magazine that reads India right

October 23, 2016

Pokémon Go and the Ghostly Metaphor

Pokémon Go is a relatively new computer game that has, since July 2016, taken the world by storm. As far as its rules go, it is not really different from computer games that have been around for the last twenty years -- the player goes around locating and collecting objects of interest to earn game points. But the real impact is the introduction of an all new level of technology whose potential is yet to be understood by most of us. Unlike every other computer game that you can play from the comfort of a desk or a couch, Pokémon Go needs you to walk around the neighbourhood with your smartphone and “catch Pokémons”. The catch here is that the game merges the virtual reality of Pokémons with the physical reality of the neighbourhood Google Map. Since the game is aware of your location, you need to walk down actual roads, turn past, or enter, actual buildings and then, and only then, will you “see” the Pokémon in your smartphone. If you turn on the smartphone camera, the game cleverly superimposes the hypothetical Pokémon that you want to catch on the actual image that the camera is showing so that it seems as if the Pokémon is really in the room or on the road. This is augmented reality -- where the “virtual” is superimposed on the “real” in a way that makes it difficult to distinguish one from the other.

image from gizmodo

Augmented reality is not new. Digital data has been superimposed on physical images in heads-up displays used by fighter aircraft pilots. But this is the first time that this technology has come down to the mass usage level through an inexpensive smartphone. Pokémon Go has the potential to be what the spreadsheet was to computers in general and the Mosaic browser was to the world wide web -- a path breaking “killer application” that suddenly leads to a sudden jump in the usage of a particular technology. Other than the obvious benefit of forcing players to actually exercise their bodies, Pokémon Go has opened up new opportunities for doing business. For example, physical locations like shopping malls and restaurants can be quickly populated by virtual Pokémons to lure footfalls and customers. Again virtual characters can “appear” in thinly populated physical locations during an emergency to guide people towards areas of safety.

As a game, Pokémon Go may be a bit of an anticlimax and a smartphone camera is not a very sophisticated display device. But together, they show us the potential of what is possible when we have a convergence between the real and the virtual. Massively Multiplayer Online Role Playing Games (MMORPG) like World of Warcraft and Final Fantasy create fantastic virtual worlds where thousands of players “play” and interact with each other while executing quests that lead to goals and rewards. Players, represented by avatars in these games, move through computer generated landscapes, communicate, trade or fight with other avatars to acquire benefits. Earlier these games used to be played on regular computer screens with keyboards and joysticks. But now we have virtual reality (VR) headsets ranging from the inexpensive Google Cardboard that works with any Android phone, through more sophisticated products like Microsoft Hololens, Oculus Rift,  to dedicated gaming devices like the Sony Playstation VR. Using a range of physical sensors, these devices track the movement and position of the user’s body and change the visual perspective so realistically that the mind is fooled into believing that the events are happening in real life and not in the display of a computer.

Going forward in the direction pioneered by Pokémon Go, all these games can and might superimpose the real physical landscape, and people, into the virtual, computer-generated landscape -- or perhaps the other way around. At the Developer Forum in August 2016, Intel unveiled Alloy, a standalone VR device, that if placed on the head will completely isolate the wearer from the physical world and instead, immerse him in the virtual. In parallel,  Microsoft is working with Intel to create the capability of generating three dimensional holograms that will be visible in the physical world using Hololens and Windows 10.  This would mean for example, a player with a VR headset will meet and interact with a virtual character in a real room while other real people in the room will see nothing since they do not have the VR headset. An augmented reality movie will allow a user to see the actors running around the locations set in publicly accessible physical location like the Taj Mahal. The possibilities are endless.

All this is possible with technology that is already available today. But let us now take a leap of imagination and see what could be happening next. Today’s virtual reality devices are big clunky devices, somewhat like binoculars or helmets. They fit in front of the eye or on the head to eliminate the physical world and replace it instead with the virtual world. In augmented reality, the real world will also be allowed to creep into the field of view and will be merged seamlessly with the virtual so that as we discussed earlier, virtual objects will appear in a real setting and real objects will appear in a virtual setting. This is where the mind will start getting confused -- what is real and what is virtual? At the moment this question can be resolved very easily if the person simply takes off the headset. Immediately the virtual will disappear and only the real will be left behind.

But what if he cannot or does not wish to remove the headset?

What if the headset is reduced to the size of a normal pair of spectacles, as in Google Glass, or even smaller to the size of contact lens that is implanted on the cornea of the eye? For such a person, the dividing line between the real and the virtual will disappear completely and he will never know what is real and what is virtual. This is very similar to driving a car whose windshield, and windows, are replaced with a computer screen that shows the “outer world”. Initially the outer world will be real world streets, buildings and cars as captured in realtime by cameras but then the screens can display a desert or jungle and the driver would think that he was indeed driving in some other terrain. In fact the illusion would hold even if  the car did not move but only the images change. But what would be most interesting, or confusing for the driver, would be if we could augment the physical reality of streets & buildings with virtual images of non-existent cars that are whizzing past. With sounds and vibrations being built in, reality will become a confusing continuum stretching from the physical world through the augmented world and then to the virtual world. No one will know where one ends and the other begins.

Spectacles, contact lenses and in-ear hearing aids have already become an intrinsic extension of our bodies and act as an intermediary between our brain and the external world of sensory data. What if these were to become so ubiquitous and smart so that we forget that they exist and are modulating the data that our brain processes to understand the “reality” of the external world?

Would we still know what is physically real and what is virtually real? Which leads us to the next, bigger question -- what indeed is “really” real?

Sankara’s Advaita Vedanta posits that nothing is real, the world is an illusion, an error that we make in our perception -- no different from the virtual image that we might see through a Microsoft Hololens! Some of us might baulk at this repudiation of objective, tangible reality and insist on evidence of the existence of alternate worlds. However, as argued by the author in “Logic to  Magic” in the September 2016 issue of Swarajya, the world of “science” itself is swinging around to look at additional dimensions where other worlds may exist. So there is nothing wrong in exploring worlds that may lie beyond the only one that we are familiar with.

Once we accept that real and virtual worlds might exist with each other and the human mind may not be able to delineate where one ends and the other begins ghosts, spirits and other astral bodies may suddenly become less difficult to accept. After all, these non-physical, non-”real” entities are no different from the Pokémons that the game software inserts into the camera,  between us and our perception of physical reality. In the case of the Pokémon Go game, we know that there is a software and a camera in the smartphone but in the case of ghosts we are currently clueless.

What if that software had an analogue in the DNA sequence in our genome that affects our ability to perceive?  What if the camera had been a “contact lens” implanted in our eye at birth? Would we ever know that the Pokémon, or the ghost, doesn’t “really” exist?

P.S. See this Forbes article on an amazing product called Magic Leap that extends the idea discussed in this post.

This article was originally published in Swarajya, the magazine that reads India right

September 27, 2016

Cryptocurrency at Cypher2016

I was invited to Cypher 2016 - India's biggest Data Science conference organised by Analytics India Magazine at Bangalore where I spoke on Bitcoins, Blockchains and Cryptocurrency.

Here is the slidedeck ( please see in full screen )

and here are the videos -- -- -- --

About This Blog

  © Blogger template 'External' by 2008

Back to TOP