Building CRUD Applications on the Blockchain - Part II

Once the basic architecture or the big picture of the blockchain ecosystem -- as explained in Part I of this article -- is understood, we can now move into the nuts-and-bolts of actually building a software application. To do so, we need the following four pieces :

1. An Ether wallet - to store Ether and pay it out when necessary
2. A Solidity development environment or IDE to build contracts
3. A connection service provider that allows client software to connect to the Ethereum network
4. A platform or IDE to build a client software that can execute transactions against Solidity contracts residing on the blockchain

1. The Ether Wallet - MetaMask

An ether wallet is a Solidity contract with an address ( think Bank Account Number)  and a private-key (think Specimen Signature in a Bank Account) that resides on the blockchain. What is commonly understood as an ether wallet is actually a client application that can execute payment transactions from the Solidity wallet-contract. So we will refer to this client as a wallet-client. 

A wallet-client can be installed on your local laptop or be hosted with a crypto-exchange like Coinbase and is protected with a password that is different from the private-key of the actual wallet-contract. For our purpose we will use MetaMask wallet-client that sits as a Chrome extension in the Chrome browser. A Mozilla extension is also available but I have not tried it. Android / iPhone versions are irrelevant for our exercise.

After installing the extension,  select the network that you want to connect to, which could be the MainNet or in our case a TestNet called Rinkeby. MetaMask allows connections to other public TestNets or also a local blockchain that you might install on your local machine. This last option may be used by experienced users but my advise is to stay away from such adventurism until you get your basics right.

Once connected to the network, MetaMask will allow you to create a number of wallet-contracts. You need to create at least one wallet-contract and and note down its address and private-key. The next step is to fund the wallet-contract with Ether. If you were using the MainNet, then you would have purchase Ethers at a crypto-currency exchange like Coinbase and have them sent to the address of your wallet-contract. In the case of the Rinkeby network,  you can get free Ethers -- that of course have no value in the real MainNet -- by visting  Rinkeby faucet and following instructions. The simplest way is to send out a tweet with your wallet-conract address,   the URL of the tweet in their form and request for Ether. After a minute or two -- remember, nothing happens instantaneously on the blockchain because of synchronization requirements -- you will suddenly find that your wallet-contract has Ethers that you can use.

2. The Remix Ethereum IDE

There are many Ethereum IDEs available but you can start with Remix , a hosted IDE that runs in a browser. Contracts developed in the Remix IDE can be deployed on the MainNet or on any of the TestNets that are supported in the MetaMask wallet-client that you have installed in the previous step after paying the required fees - in Ethers - from the wallet-contracts defined in the MetaMask wallet-client. But since deploying and testing of contracts on the network is a little slow, Remix gives you a local Ethereum node -- complete with wallets etc -- where you can rapidly test the Solidity contracts that you are testing. Remix also gives you a small and generic client with which you can call ( or invoke) functions (or methods), pass parameters and see the results returned.

The code that you write in Remix can be saved either on your local disk or directly as a Gist in Github! 

A CRUD application

The application that we will build is inspired by this post from Rob Hitchens who has not only shared the architecture of his application but also his entire code. The basic idea in our application, that is different from Rob's application, is as follows :

• We have an employee database consisting of EmpID, EmpName, EmpSalary that is stored in a simple array called database9 that is indexed by the EmpID. The EmpID is the primary key or index and is stored separately in another index array called db9index.
• There are six functions that are defined, namely, 
• insertEmp - create new record in the database
• updateEmpSalary - update the salary field
• deleteEmp - delete a record
• getEmpCount - get total number of records
• getEmpData - get record of employee specified by EmpID
• isEmp - to check whether an employee ID exists or not

Open the  Remix  IDE, delete any sample applications and start coding. To get started, create a new .sol file called pmCrudCon.sol and copy the contents of the code available here.   Simply copy and paste this code into your file. You are free to modify this application and save it in your own Gist but for the time being let us deploy and execute the sample code.

The IDE has three important tabs on the left

• The coding tab where you write or edit your Solidity program
•  The compile tab where you choose your compiler and compile your code. On successful compilation, go to the bottom and look for a field called ABI. Copy the contents of the field and store it separately. We will need it later for the client application. You can of course come back and copy it later as well.
• The deployment tab that we will study in some detail

In the deployment tab

• The first key field is Environment where you have three options. Either you can deploy the contract in the Javascript EVM in your local browser or you can choose the Injected Web3 option that will connect to the TestNet that your MetaMask wallet-client is already connected to! Since both the Remix IDE and the MetaMask wallet-client are loaded in the same browser, the connection is delightfully easy. The third option of Web 3 provider is more complicated and can be safely ignored for the time being.
• The Gas limit and the wei value can be ignored or left on default.
• Assuming that the compile is successful, you can now press the Deploy button. Now a couple of things will happen
• Since Deployment costs you Ether, MetaMask will seek your confirmation to debit the required number of Ether ( do not panic, it is a miniscule fraction of an Ether) and send it to the network.
• Once you confirm, the deployment transaction will be be submitted and may take a few seconds to complete. You will get a confirmation from MetaMask that  you transaction (for deployment, or whatever) has been confirmed.
• 
  
  
  
  
• If you scroll down you will see details of the contract ( in this case PMCRUDCON)  that has been deployed. Copy the contract address - that is more important than the name.
• There is a button to execute each of the functions along with a way to send arguments to the same. Go ahead and insert an employee with name, salary, emp id as 'Narendra Modi', 10000, 100 or something like that.
• Note that the Blue boxes represent Read Only functions that do not require Ether payment. The Brown boxes represent state-changing transactions and for each MetaMask will request you to confirm a payment and only then will the transaction be submitted to the network.

3. Connection Service - infura.io

When MetaMask or Remix connects to the network -- mainnet or testnets -- they know the URL to use but how will the client that you and I write, connect. To do so, head over to infura.io and create first a login for yourself and then a Ethereum project. Navigate to the dashboard settings, choose the network that you wish to connect to and get the URL of the corresponding endpoint that will look like this : https://rinkeby.infura.io/v3/68ae8e6787934812814cffe52999999"

4. The Python Client

The contract-client that will allow us to execute transactions built into a deployed Solidity contract can be built with any language that has a web3 library with a set of functions that allow us to interact with the Ethereum network. Both Javascript and Python libraries are available but because I am more comfortable with Python we use Python in this example. Any Python IDE -- like Anaconda with Spyder or Jupyter -- is fine but we prefer to use Google Colab! and the corresponding notebook is available here. Like Remix, Colab is a another browser based IDE so again there is nothing to install. If you are not familiar with Google Colab, suggest that you learn it by reading tutorials.

The notebook is pretty straightforward and goes through the following steps

• Install the web3 client with pip. Note that you need to restart the VM after the installation
• Define the contract address, the wallet address and the wallet private key 
• Unhide the cell containing ABI variable and place your own ABI variable from Remix there
• Define the contract in terms of its address, and ABI value
• Execute the read_only functions and see the results. Data that you have entered in the Remix client should be visible here.The only way you can get an error is if  you have not transferred the following variables correctly
• Endpoint URL from infura.io
• Contract Address & ABI value from Remix
• Wallet Address and Wallet Private Key from MetaMask

For the state-altering ( or insert, update, delete) transactions, the process is little complex because we have to send the function along with a way to make an Ether payment. Look at the pyinsertEmp function that acts a wrapper to the Solidity insertEmp function. Before calling the Solidity we need to specify the maximum quantity of gas that we will expend on executing the function. Gas comes at a price and quantity x price gives us the total that we will spend. To spend we need to specify the address of the wallet-contract as well as its private-key. We also need to specify the chainID ( for Rinkeby this is 4).

Now we call the function or submit the transaction. If all variables have been set correctly and the gas quantity is sufficient, the transaction will be enqueued for the network but it will take time to execute. Hence the python code must keep waiting until a non-null transaction receipt comes back or there is a timeout. Typically, this takes about 20 - 30 secs in the testnet. Both the transaction receipt and a broadcast log ( if set) can be read to understand whether the transaction has succeeded or failed.

---

This application has demonstrated how to write a basic application that allows us to create, read, update and delete data records into the blockchain. This is primarily what any commercial application does and we have shown how to this with a 

• Solidity Contract - pmCrudCon
• Python client demonstrated here notebook is available here

The next step would be to write contracts that will accept payment in Ether and deliver a digital artifact  or service to the initiator of a transaction. To see how that is done look at 

• This Solidity Code : that allows you to deposit and withdraw Ether into a contract and also send it to a third party
• This Colab Notebook that shows the Python Client

Building CRUD Applications on the Blockchain - Part I

With Bitcoin and cryptofinance in the news, blockchain programming has suddenly become very important but most people - including many programmers - do not have a clear idea on how to write a program on the blockchain. This article is not about theoretical concepts like Merkle Trees and SHA256 Hash. It is about how to write an application that creates, reads, updates and deletes a data record from a 'database' on the blockchain. A CRUD application. If you are impressed by Powerpoint slides in Zoom Webinars, you may skip this article, but if you are a programmer -- professionals or hobbyist -- do read on. You will end up writing your first blockchain application without too much sweat.

I first came across Bitcoins as currency in the virtual world of Second Life and wrote about "Bitcoins - my first look at a new currency" in 2013 and then I struggled to wrap my head around this fantastic concept for almost a year until I reached a point where I could write "Bitcoins - as I understood it." While Bitcoin itself was a fascinating concept - an esoteric cocktail of mathematics, programming and economic concepts worthy of a Nobel Prize in Economics - the underlying technology of blockchain was even more fascinating. The incredible potential of this technology became evident with the advent of the Ethereum Virtual Machine and intrigued me enough to explore the concept of the CryptoCorporation, a strange, but successful automatic, manager-less organisation that could create value out of thin air and finally in 2016, I was finally able build my own cryptocoin using Smartcontracts running on the Ethereum blockchain.

Back in 2016, writing blockchain programs were not for the faint-hearted, because there were two big stumbling blocks.

• First, one had to install a whole suite of very complex software -- including wallets -- on the laptop and 
• Second, one had to buy Ether, with credit cards, and use it each time you wanted to run, or even test a program that you had written.

This was frustrating, to say the least. First, if you had to replace or change your computer, there was this great likelihood of not just losing your software stack but also your Ether wallet unless  you had the foresight and the ability  ( which very few had) to keep proper backups. The other huge challenge was the sudden decision of the RBI / Government of India, not too ban cryptocurrency, but to bar banks from allowing people to purchase cryptocurrency using normal, legitimate banking channels like credit cards. Hence while the whole world, including China, raced ahead with blockchain technology, programmers in India were left with no option but to only watch from the sidelines. Fortunately, the Supreme Court has lifted this ban  in March 2020 and blockchain programmers can start work again after losing nearly two years of sitting it out in the wilderness.

image from Forbes.com

But blockchain programming is so very different from 'normal' programming that most programmers have difficulty in wrapping their heads around the  concept. Forget about the concept of smart contracts, even basic stuff like writing, reading and updating a piece of data becomes esoteric because apparently there is no one machine where program resides or executes. It took me quite a while to understand this and once I did, the elegance and beauty of it all took my breath away. Which is why I thought of writing it down, not just for my own clarity but for other programmers who could possibly be in the same boat as well. While I have tried to keep my post as simple as possible, please note that reading beyond this point means that you have some programming experience that goes beyond Excel. An exhaustive manual would be far too long but if you are like me, a hobbyist with a penchant for coding, sit back and enjoy this ride. 

This tutorial is based on two important premises :

1. Only the bare minimum software will be installed on the local laptop, we will use hosted software as much as possible.
2. We will not spend any real money -- as in using credit cards.

Without any further ado, let us start with 

The Big Picture / Architecture

A theoretical and mathematical description of the blockchain is given in this article but for all practical purposes it consists of a network of computers (the MainNet) that run the the full Ethereum node software. All full nodes contain identical copies of two artifacts : (a) a node software application and (b) a huge database file that is called the blockchain. Both the node application and the block chain are kept in continuous sync by constant exchange of messages between nodes, all of which have a peer-to-peer relationship with each other. In addition to the MainNet, there are other private and public chains called Test Nets  with names like Ropsten, Rinkeby, Roerli etc. 

In traditional three-tier client server architecture we have three components. 

• Client software written in, say Python, Javascript, VisualBasic, or in many cases running inside an Internet Browser like Chrome, Mozilla or Edge. This provides the user interface.
•  Application software, written in say, PHP, Ruby, Python that resides within an application server or web server like Apache. This provides the business logic. 
• A persistent data store like MySQL. Obviously the application server (Apache) and the database server (MySQL) may or may not reside in the same physical machine.

Developers build applications by writing both client-side programs (in Javascript, Python) that are hosted in the client machine and server-side program (in PHP, Python etc) that are hosted in the web-server. There is also the case of client side software being stored in the server and downloaded and used inside a browser, but we shall park that aside for the time being. The database engine, MySQL ( or equivalent) is not written but purchased or downloaded and installed on an appropriate machine. If you are a programmer, all this will be very easy for you to understand.

In the blockchain architecture we have a two tier setup

• Client application written in Javascript, Python (there could be others, but I have not used other languages) or running in an Internet Browser. As in traditional client-server architecture, the client software provides the user interface which may or may not be GUI.
• 'Contracts' - a new term used only in the blockchain -- written in a language called Solidity - again a new language used on blockchains -  that holds BOTH business logic AND persistent data.

Since contracts are the new kids in the block, let us look at them a little carefully

• Think of a contract as an instance of a object from the world of Object Oriented programming. This means that the contract has both logic ( as in methods, or functions) and data. But unlike OO objects which can have multiple instances, there is only one instance of a contract. Like OO objects, contracts can inherit properties from other contracts.
• The state of a contract -- as defined by the data that it holds -- can be changed  through a transaction. A transaction can be initiated from a client application either by a human being or some automated process
• A contract is deployed into and resides in the blockchain. Hence it is available on all computers on all networks and all copies are in the same state. Each contract is identified with its own unique address that is generated when a contract is deployed in the blockchain through any node.
• A wallet is special kind of contract that holds a defined quantity of Ether. A transaction can deposit any amount of Ether into a wallet contract but to withdraw the Ether from a wallet contract, one must provide a password ( aka 'the private key') of this specific wallet. So a wallet is a contract defined by its address (similar to a bank account) and its private key (similar to specimen signature in a bank account)
• When a client application connects to any node (through a standard URL) it requests or calls a contract that is now loaded from the blockchain on the node where it resides to the Ethereum Virtual Machine running on the same node. On the EVM, the contract performs the task -- in the process, it may or may not change its state -- and then is put back in its new state back into the blockchain. 
• It is actually a misnomer to say that the state of the contract is changed. The blockchain is immutable, it can be added to but earlier data cannot be changed. When we say that the state of the contract is changed, what we really mean is that the original contract along with all transactions that were supposed to change it are ALL stored in the blockchain. Hence the current state of the contract can -- and is -- recreated. This may sound incredibly inefficient when compared to, say a relational database, but it is the only option when there is no central authority ( like the MySQL database adminstrator) who is trusted by everyone in the network. Which is why it does not make any sense to port an application from a traditional three-tier platform to the blockchain platform where there exists a clear central authority, like a statutory body and its database administrator, to own manage a central database.. Blockchain applications are meant to handle situations where there are multiple operators -- all of whom are peers, without a central hierarchy -- and there is no reason to trust any or all of them.

Finally, in addition to the client application and the Solidity contract, that a developer needs to build there is a third 'joker' that the developer needs to deal with - a payment. Since the transaction is getting executed on network node, the owner of the node needs to be incentivised or compensated for their investment in hardware and electricity. In the Bitcoin network this incentive is in the form of new coin that is mined ( that is another long story -- see here, and here ) but in the Ethereum ecosystem it is a combination of mining and an actual fee that must be paid by the client application to the node owner in the form of Ether. On the Main Net, this is real Ether that has be mined or purchased but in the other Test Nets, there is a lot 'fake' or 'toy' Ether that is freely available. Think of this as monopoly money with which we buy properties in the Monopoly game. This Ether - real or fake, depending on the network being used - must be sent along with the transaction request.

The second part of this tutorial will show you how to to build 

• a Solidity contract pmCrudCon.sol  to manage employee data ( empID, empName, empSalary)
• a Python Application to Insert, Update, Display and Delete this Data

Badshah Alam Shah 1204

 

Located this coin among others in a locker being cleaned out. Charanpreet Singh helped me decode the text that reads 19 Zarb Murshidabad and Badshah Alam Shah 1204. Used this information to get some more information from an auction site.

Deepawali acquisition!

Ethics of AI

Unless you have been living under a rock for the past couple of years you would know for sure that things are happening in the area of Artificial Intelligence. Rapid developments in the area of artificial neural networks has spawned a brood of useful architectures - CNN, RNN, GAN - that have been used to solve a range of very interesting problems. These include, among others

• control of autonomous or self driving vehicles
• identifying visual elements in a scenery
• recognising faces or connecting bio-metrics to individual identities
• automatic translation from one language to another
• generating text and visual content that is indistinguishable from that generated by human intellect.

While these applications have created considerable excitement both in the technical as well as in the commercial community, there has been an undercurrent of resentment among certain people against what they view as ethical issues that are yet to be unresolved. 

To understand what is at stake let us consider two specific issues from the area of autonomous vehicles. 

First who is liable in the case of an accident? In some countries, the liability lies with the owner of the vehicle while in others, it lies with the driver who was at the wheel when the accident occurs. But in the case of autonomous vehicles there is a point of view that says that the liability should be with the manufacturer. If at all it was the fault of the autonomous vehicle, and not the other party to the accident, then the fault lies with the autonomous system - hardware sensors and controlling software - that has been supplied by the manufacturer. This is similar to a brake failure except that the owner, driver has no way to check the equipment prior to starting out to drive.

Second, and this is more interesting, is the question of whose life is more important? Suppose a pedestrian comes in the way of a moving vehicle whose speed is such that an application of brakes will not be able to stop the car from hitting the pedestrian. The only maneuver that is possible is for the car to turn away and hit a wall. In either case the injury or death will happen either to the pedestrian or to the driver. For the sake of this argument, we can simplify the situation by ignoring issues like estimating the expected quantum of injury in the two cases and the subsequent possibility of death or extent of disfigurement and come out with a binary situation - whose life is more valuable? The driver or the pedestrian?

image from berkeley.edu

These may look like very profound questions and are very often portrayed as such but frankly they are not. 

In the first case, there is no need to split hairs on the liability. Lawyers may love the possibility of litigation and accountants may salivate at the the thought of extracting money from car manufacturers but for the technologist, this is a no-brainer. Most car accidents are because of driver error, except of course when a pedestrian behaves randomly, and with the advent of autonomous vehicles the possibility of driver error virtually disappears. So if the vehicle software has been adequately tested - like vaccines! -- before they are released in the 'wild', the number of accidents will, in any case, go down dramatically. So the overall cost of accidents will go down but individual cases will be paid out of the general corpus of funds created by collecting premiums from all vehicle owners, that are calculated by usual statistical ( or actuarial) analysis. In fact, this no different from a mechanical failure which in any case is factored into the economics of insurance. Net-net, there is no issue at all. It is just another unfortunate accident that has to be factored in to the premium calculation process, perhaps with an additional line item.

The second issue can also be dealt with quite easily. Who should die? The pedestrian or the driver? In the case of a human driver both situations are possible. Some drivers will slam on the brakes and hope that the car will stop before hitting the pedestrian while other drivers will turn the car and hit the wall. There is no hard and fast logic and nor is there time for a thorough analysis, ethical or otherwise, of the various options. It is a gut-feel reaction that is best modeled by a random probability. So the simple way to break the tie is to toss a coin -- or simulate the coin toss with a random number generator -- and take a decision on whether the coin shows head or tails.

If it is a fair coin, there is a 50% chance of either outcome and so the software can be programmed to take one decision or the other on the basis of this probability. This would reflect the regular, or underlying, reality of a human driver. So the behaviour of the autonomous vehicle would in no way be different from the behaviour of a vehicle driven by a human being. If we have learnt to live with human drivers we can continue to live with autonomous vehicles.

The 50% rule is a kind of a default starting point. If it is observed that most drivers are altruistic and prefer to save the pedestrian at the cost of their own health then the probability of hitting the wall can be raised from 50% to 60%. On the other hand, if it is observed that most drivers are selfish and prefer to kill the pedestrian and save themselves, then the probability of hitting the wall can be lowered to 40%. These probability numbers mean that the coin being tossed is not a fair coin but a biased one and reflects the inherent bias of society at large.

This solves the problem of the autonomous car but opens up another Pandora's Box.

Should AI ( or Deep Learning ) systems have a bias at all? Or should they always be fair? This is important because Deep Learning systems are trained on the basis of a history of past behaviour of human systems. This training is done by collecting data on how decisions have been taken in the past and using this data to set the parameters. In simple systems, these parameters could be probability values but in neural networks they are the weights that are assigned to different connections between nodes. The exact technology is not important here. What is important is whether the training data has bias and whether this bias is carried through from the non-computer system to the computer system.

For example, it has been observed that in the US, both parole applications and loan applications are more likely to be rejected if the applicant is a black person because of a historical bias against this particular demographic segment. When this data is used train a AI / DL system, this bias is carried through and once again, blacks will be discriminated against. [ Of course, there is another point of view that states that automated, machine based decisions have less bias -- see this (paywalled) link -- but that is another story and another debate.]

Obviously this is patently unfair and should not be allowed and hence there is a strong move to ensure that AI systems do not suffer from bias. There is no question about that ...

But does that mean that AI / DL should not be built until we have resolved the issue of bias? This is where the debate takes on an ugly turn between the proponents of ethics in AI and those who would rather stick to the technology of AI. For the former, the question of ethics is paramount and they would rather not have AI unless it is certified to be bias free. For the latter, the matter of ethics is secondary. They would rather focus on creating innovative technology and leave the matter of ethics for another day.

Faced with this choice, my sympathies clearly lie with the latter, the technologists and the reason is very simple. The world is not fair and can only be so in the dreams of the Utopian idealist. Since we do not have the luxury of living in an ideal utopia, expecting AI to be ethical and bias free is an impossible dream. The world has learn to live with bias and will continue to be so. If ethics was really as essential for the survival of the human race then we should have shut down the armaments business (and possibly a large part of the pharmaceutical and hospital business as well) But we have not done so, because of an irresistible, or inevitable, convergence of economic, political and social power.

Any country, or society, that shuts down its armaments business or disbands its armed forces will be overrun and taken over by another country that does not subscribe to this Gandhian policy of pointless non-violence. This was brutally demonstrated during the 1962 China War where India's idealistic Principles of Panchsheel were brutally shoved aside by the rampaging Chinese PLA. While a measure of ethics is certainly good, making it an absolute framework that is at odds with the ambient reality is neither possible, nor desirable. So is the case with AI. There are many people who feel that the so-called 'liberal' countries like the United States should not use technology like facial recognition at all because it is an unethical violation of privacy. Little do they realise that 'non-liberal' countries like China are already using it in a big way to enhance their own security and if the imbalance continues it will be as stupid as shutting down the armaments industry.

Any technology - from nuclear through genetics and space to artificial intelligence -- can be weaponised. That does not mean that development must stop. Let us go in with our eyes wide open, be aware of the dangers but also be aware of what is happening elsewhere and make sure that we do not vacate or step back from the leading, or bleeding, edge.

To sum up, let us understand that bias is inevitable in any human society. We should try to minimise it but hoping to eliminate it is impossible. So is the case with non-human, silicon based intelligence or for that matter for any non-human sentience that will eventually arise from this technology.