Spotlight: Spatial Computing

This is the 6^th column of the series called Navigator MasterClass, wherein we will find our way through the myths and realities of one Bleeding Edge technology each month; in terms of where it truly stands at the time of writing, and its business applications- implemented, being attempted, or speculative. This month’s topic is Spatial Computing (SC). A technology that has hundreds of definitions, almost like the elephant and the blind men. So, let’s trace its origin in very brief.

Simon Greenworld coined the term in his 2003 MIT Graduate Thesis paper, defining it as “human interaction with a machine in which the machine retains and manipulates references to real objects and spaces”. Before we translate this into a non-academic language, let us look at a more precise technical definition, presented by a group of researchers at the 2007 Seminar on Computing Media and Language for Space-Oriented Computation: “a field of research in Computer Science where space is not an abstract notion, but a first order effect that has to be optimized”.

In basic terms, SC extends digital world into the real one, by using light, sounds, images, trackers, sensors, and haptic devices. It does so by using a combination of Virtual Reality, Augmented Reality, Mixed Reality, Edge Computing and Digital Twins platforms. The implementation manifests itself by using physical actions (head and body movements, gestures, speech, eye focus) as inputs for interactive digital systems, and physical spaces as output base for video, audio, and haptics.

A potential industrial application is a single operator operating, monitoring, and managing multiple machines SIMULTANEOUSLY, a la an orchestra conductor. This could be him/ her running an entire assembly line, or just managing multiple similar machines; all at the same time. Not surprisingly, Gaming has been the pioneering user of this technology. It is now being experimented with, in HR (training), Product Development, and Virtual Tours in tourism and real estate.

As can be expected, the usual players are betting billions on SC: Microsoft, Alphabet, Facebook (Metaverse, anyone!), Apple, Amazon, Tesla, and so on. There are also start-up disruptors like Magic Leap which has raised billions of dollars in funding (including during the recent pandemic) in its 12 years of existence and is developing technology to superimposes 3D computer-generated imagery over real world objects; it does so by projecting a digital light field into the user's eye. Its enterprise platform, Magic Leap 2 releases on September 30, 2022; it promises to be an immersive device with optics with up to 70° diagonal FOV. Let us now look at three products being developed by Microsoft; all of which were presented at a 2019 ACM Conference and can be found on YouTube.

The first one is Dreamwalker; it turns the user’s physical walk into a VR experience. Say (s)he wants to walk to the grocery store and wants to experience a walk through the Mughal Gardens in Delhi. The walk will look and feel like what you wanted it to be, with the turns and obstacle avoidance as needed in the real-world walk. This is real time manipulation of VR: insertion of obstacles in the VR walk, so that you physically move around the obstacles (like a person walking towards you).

The second is Mise-Unseen; it senses the eyes to find out where the user’s gaze and attention is focused, and changes objects in the other parts of the VR world without the user realizing it. An example is a VR tour of an art gallery, where paintings on the walls are replaced to suit the user’s tastes without him/ her knowing it (because that particular area is outside the Field Of Vision).

The third stunning platform is Capstan Crunch; it uses haptic controllers to create physical pressures and sensations in a virtual world. An example would be the user catching a ball in mid-air. The controller would provide sensation of a saft ball in the physical hand and put enough pressure to have the fingers and palm in a shape that is holding the ball (i.e., the physical fingers would not close fully into the palm).

Let’s now change gears and look at where the world of SC is heading. McKinsey is convinced that retailers will soon need to become “experience designers”. This will increasingly become easier with Quantum Computing become available on a mass scale (just waiting for one technical breakthrough, that could happen any day); making SC’s mass adoption easier.

Gartner on the other hand believes that SC is one of the six expected primary uses of Metaverse (the other five being Gaming, Digital Humans, Virtual Spaces, Shared Experiences, and Tokenized Assets (NFTs)). They do go on to predict that these will become reality by 2030, because the platforms are still in “innovation” stage. The primary innovations needed are around SC: Spatial Web, Spatial Data Interchange Standards, Spatial Protocols, and Spatial Registries (not to talk of hygiene like security, privacy, network latency, etc.). In other words, new ideas like Metaverse need SC to succeed.

MIT projects three trends that will emerge: dynamic workplace redesign, co-working transformation, and ubiquitous computing. All of these are implementations of SC. They go on to ask a very interesting question to highlight the role of SC: ”Previously, new digital communication technologies caused some theorists to predict “the death of distance” - liberating us from the office space. Today’s technology does allow global and instantaneous communication, but most of us still commute to offices for work every day. Why?”

The second thing they project is that People and Machines will become even closer partners in innovation. An example they quote is Alphabet’s DeepMind that developed a software called AlphaZero in 2017. This program was ONLY taught the rules of chess. After just one day of playing itself, AlphaZero crushed the current leading software (which was unbeaten by any human); and chess masters learnt new tricks and moves from AlphaZero. It is not hard to imagine the level such partnership will reach when you add SC to this.

We end with addressing the elephant in the room. What are the chances of misuse and abuse with SC becoming mainstream. Is it like the nuclear fission technology which started with good intentions, and ended up creating the atom bomb too? Certainly, registry-based distributed computing platforms like blockchain will help; but humans will need some more work on who teaches the chess rules to SC platforms.