Pokémon Go is a relatively new computer game that has, since July 2016, taken the world by storm. As far as its rules go, it is not really different from computer games that have been around for the last twenty years -- the player goes around locating and collecting objects of interest to earn game points. But the real impact is the introduction of an all new level of technology whose potential is yet to be understood by most of us. Unlike every other computer game that you can play from the comfort of a desk or a couch, Pokémon Go needs you to walk around the neighbourhood with your smartphone and “catch Pokémons”. The catch here is that the game merges the virtual reality of Pokémons with the physical reality of the neighbourhood Google Map. Since the game is aware of your location, you need to walk down actual roads, turn past, or enter, actual buildings and then, and only then, will you “see” the Pokémon in your smartphone. If you turn on the smartphone camera, the game cleverly superimposes the hypothetical Pokémon that you want to catch on the actual image that the camera is showing so that it seems as if the Pokémon is really in the room or on the road. This is augmented reality -- where the “virtual” is superimposed on the “real” in a way that makes it difficult to distinguish one from the other.
Augmented reality is not new. Digital data has been superimposed on physical images in heads-up displays used by fighter aircraft pilots. But this is the first time that this technology has come down to the mass usage level through an inexpensive smartphone. Pokémon Go has the potential to be what the spreadsheet was to computers in general and the Mosaic browser was to the world wide web -- a path breaking “killer application” that suddenly leads to a sudden jump in the usage of a particular technology. Other than the obvious benefit of forcing players to actually exercise their bodies, Pokémon Go has opened up new opportunities for doing business. For example, physical locations like shopping malls and restaurants can be quickly populated by virtual Pokémons to lure footfalls and customers. Again virtual characters can “appear” in thinly populated physical locations during an emergency to guide people towards areas of safety.
As a game, Pokémon Go may be a bit of an anticlimax and a smartphone camera is not a very sophisticated display device. But together, they show us the potential of what is possible when we have a convergence between the real and the virtual. Massively Multiplayer Online Role Playing Games (MMORPG) like World of Warcraft and Final Fantasy create fantastic virtual worlds where thousands of players “play” and interact with each other while executing quests that lead to goals and rewards. Players, represented by avatars in these games, move through computer generated landscapes, communicate, trade or fight with other avatars to acquire benefits. Earlier these games used to be played on regular computer screens with keyboards and joysticks. But now we have virtual reality (VR) headsets ranging from the inexpensive Google Cardboard that works with any Android phone, through more sophisticated products like Microsoft Hololens, Oculus Rift, to dedicated gaming devices like the Sony Playstation VR. Using a range of physical sensors, these devices track the movement and position of the user’s body and change the visual perspective so realistically that the mind is fooled into believing that the events are happening in real life and not in the display of a computer.
Going forward in the direction pioneered by Pokémon Go, all these games can and might superimpose the real physical landscape, and people, into the virtual, computer-generated landscape -- or perhaps the other way around. At the Developer Forum in August 2016, Intel unveiled Alloy, a standalone VR device, that if placed on the head will completely isolate the wearer from the physical world and instead, immerse him in the virtual. In parallel, Microsoft is working with Intel to create the capability of generating three dimensional holograms that will be visible in the physical world using Hololens and Windows 10. This would mean for example, a player with a VR headset will meet and interact with a virtual character in a real room while other real people in the room will see nothing since they do not have the VR headset. An augmented reality movie will allow a user to see the actors running around the locations set in publicly accessible physical location like the Taj Mahal. The possibilities are endless.
All this is possible with technology that is already available today. But let us now take a leap of imagination and see what could be happening next. Today’s virtual reality devices are big clunky devices, somewhat like binoculars or helmets. They fit in front of the eye or on the head to eliminate the physical world and replace it instead with the virtual world. In augmented reality, the real world will also be allowed to creep into the field of view and will be merged seamlessly with the virtual so that as we discussed earlier, virtual objects will appear in a real setting and real objects will appear in a virtual setting. This is where the mind will start getting confused -- what is real and what is virtual? At the moment this question can be resolved very easily if the person simply takes off the headset. Immediately the virtual will disappear and only the real will be left behind.
But what if he cannot or does not wish to remove the headset?
What if the headset is reduced to the size of a normal pair of spectacles, as in Google Glass, or even smaller to the size of contact lens that is implanted on the cornea of the eye? For such a person, the dividing line between the real and the virtual will disappear completely and he will never know what is real and what is virtual. This is very similar to driving a car whose windshield, and windows, are replaced with a computer screen that shows the “outer world”. Initially the outer world will be real world streets, buildings and cars as captured in realtime by cameras but then the screens can display a desert or jungle and the driver would think that he was indeed driving in some other terrain. In fact the illusion would hold even if the car did not move but only the images change. But what would be most interesting, or confusing for the driver, would be if we could augment the physical reality of streets & buildings with virtual images of non-existent cars that are whizzing past. With sounds and vibrations being built in, reality will become a confusing continuum stretching from the physical world through the augmented world and then to the virtual world. No one will know where one ends and the other begins.
Spectacles, contact lenses and in-ear hearing aids have already become an intrinsic extension of our bodies and act as an intermediary between our brain and the external world of sensory data. What if these were to become so ubiquitous and smart so that we forget that they exist and are modulating the data that our brain processes to understand the “reality” of the external world?
Would we still know what is physically real and what is virtually real? Which leads us to the next, bigger question -- what indeed is “really” real?
Sankara’s Advaita Vedanta posits that nothing is real, the world is an illusion, an error that we make in our perception -- no different from the virtual image that we might see through a Microsoft Hololens! Some of us might baulk at this repudiation of objective, tangible reality and insist on evidence of the existence of alternate worlds. However, as argued by the author in “Logic to Magic” in the September 2016 issue of Swarajya, the world of “science” itself is swinging around to look at additional dimensions where other worlds may exist. So there is nothing wrong in exploring worlds that may lie beyond the only one that we are familiar with.
Once we accept that real and virtual worlds might exist with each other and the human mind may not be able to delineate where one ends and the other begins ghosts, spirits and other astral bodies may suddenly become less difficult to accept. After all, these non-physical, non-”real” entities are no different from the Pokémons that the game software inserts into the camera, between us and our perception of physical reality. In the case of the Pokémon Go game, we know that there is a software and a camera in the smartphone but in the case of ghosts we are currently clueless.
What if that software had an analogue in the DNA sequence in our genome that affects our ability to perceive? What if the camera had been a “contact lens” implanted in our eye at birth? Would we ever know that the Pokémon, or the ghost, doesn’t “really” exist?
P.S. See this Forbes article on an amazing product called Magic Leap that extends the idea discussed in this post.
 |
| image from gizmodo |
Augmented reality is not new. Digital data has been superimposed on physical images in heads-up displays used by fighter aircraft pilots. But this is the first time that this technology has come down to the mass usage level through an inexpensive smartphone. Pokémon Go has the potential to be what the spreadsheet was to computers in general and the Mosaic browser was to the world wide web -- a path breaking “killer application” that suddenly leads to a sudden jump in the usage of a particular technology. Other than the obvious benefit of forcing players to actually exercise their bodies, Pokémon Go has opened up new opportunities for doing business. For example, physical locations like shopping malls and restaurants can be quickly populated by virtual Pokémons to lure footfalls and customers. Again virtual characters can “appear” in thinly populated physical locations during an emergency to guide people towards areas of safety.
As a game, Pokémon Go may be a bit of an anticlimax and a smartphone camera is not a very sophisticated display device. But together, they show us the potential of what is possible when we have a convergence between the real and the virtual. Massively Multiplayer Online Role Playing Games (MMORPG) like World of Warcraft and Final Fantasy create fantastic virtual worlds where thousands of players “play” and interact with each other while executing quests that lead to goals and rewards. Players, represented by avatars in these games, move through computer generated landscapes, communicate, trade or fight with other avatars to acquire benefits. Earlier these games used to be played on regular computer screens with keyboards and joysticks. But now we have virtual reality (VR) headsets ranging from the inexpensive Google Cardboard that works with any Android phone, through more sophisticated products like Microsoft Hololens, Oculus Rift, to dedicated gaming devices like the Sony Playstation VR. Using a range of physical sensors, these devices track the movement and position of the user’s body and change the visual perspective so realistically that the mind is fooled into believing that the events are happening in real life and not in the display of a computer.
Going forward in the direction pioneered by Pokémon Go, all these games can and might superimpose the real physical landscape, and people, into the virtual, computer-generated landscape -- or perhaps the other way around. At the Developer Forum in August 2016, Intel unveiled Alloy, a standalone VR device, that if placed on the head will completely isolate the wearer from the physical world and instead, immerse him in the virtual. In parallel, Microsoft is working with Intel to create the capability of generating three dimensional holograms that will be visible in the physical world using Hololens and Windows 10. This would mean for example, a player with a VR headset will meet and interact with a virtual character in a real room while other real people in the room will see nothing since they do not have the VR headset. An augmented reality movie will allow a user to see the actors running around the locations set in publicly accessible physical location like the Taj Mahal. The possibilities are endless.
All this is possible with technology that is already available today. But let us now take a leap of imagination and see what could be happening next. Today’s virtual reality devices are big clunky devices, somewhat like binoculars or helmets. They fit in front of the eye or on the head to eliminate the physical world and replace it instead with the virtual world. In augmented reality, the real world will also be allowed to creep into the field of view and will be merged seamlessly with the virtual so that as we discussed earlier, virtual objects will appear in a real setting and real objects will appear in a virtual setting. This is where the mind will start getting confused -- what is real and what is virtual? At the moment this question can be resolved very easily if the person simply takes off the headset. Immediately the virtual will disappear and only the real will be left behind.
But what if he cannot or does not wish to remove the headset?
What if the headset is reduced to the size of a normal pair of spectacles, as in Google Glass, or even smaller to the size of contact lens that is implanted on the cornea of the eye? For such a person, the dividing line between the real and the virtual will disappear completely and he will never know what is real and what is virtual. This is very similar to driving a car whose windshield, and windows, are replaced with a computer screen that shows the “outer world”. Initially the outer world will be real world streets, buildings and cars as captured in realtime by cameras but then the screens can display a desert or jungle and the driver would think that he was indeed driving in some other terrain. In fact the illusion would hold even if the car did not move but only the images change. But what would be most interesting, or confusing for the driver, would be if we could augment the physical reality of streets & buildings with virtual images of non-existent cars that are whizzing past. With sounds and vibrations being built in, reality will become a confusing continuum stretching from the physical world through the augmented world and then to the virtual world. No one will know where one ends and the other begins.
Spectacles, contact lenses and in-ear hearing aids have already become an intrinsic extension of our bodies and act as an intermediary between our brain and the external world of sensory data. What if these were to become so ubiquitous and smart so that we forget that they exist and are modulating the data that our brain processes to understand the “reality” of the external world?
Would we still know what is physically real and what is virtually real? Which leads us to the next, bigger question -- what indeed is “really” real?
Sankara’s Advaita Vedanta posits that nothing is real, the world is an illusion, an error that we make in our perception -- no different from the virtual image that we might see through a Microsoft Hololens! Some of us might baulk at this repudiation of objective, tangible reality and insist on evidence of the existence of alternate worlds. However, as argued by the author in “Logic to Magic” in the September 2016 issue of Swarajya, the world of “science” itself is swinging around to look at additional dimensions where other worlds may exist. So there is nothing wrong in exploring worlds that may lie beyond the only one that we are familiar with.
Once we accept that real and virtual worlds might exist with each other and the human mind may not be able to delineate where one ends and the other begins ghosts, spirits and other astral bodies may suddenly become less difficult to accept. After all, these non-physical, non-”real” entities are no different from the Pokémons that the game software inserts into the camera, between us and our perception of physical reality. In the case of the Pokémon Go game, we know that there is a software and a camera in the smartphone but in the case of ghosts we are currently clueless.
What if that software had an analogue in the DNA sequence in our genome that affects our ability to perceive? What if the camera had been a “contact lens” implanted in our eye at birth? Would we ever know that the Pokémon, or the ghost, doesn’t “really” exist?
P.S. See this Forbes article on an amazing product called Magic Leap that extends the idea discussed in this post.
This article was originally published in Swarajya, the magazine that reads India right
