As AR and VR become more and more popular, I feel there is a need to look deeply into the nature of this technological phenomenon. It feels like our technologically-driven media is trying to break through the confines of the screen. Through spatial awareness (i.e., tracking, motion capture, point clouds) our imagination is trying to become manifest into reality. Thus, I thought it would be a good exercise to define some philosophical fundamentals around this phenomenon of the Digital Age.
‘Virtual elements’ as perceptual augments
As we progress further into the Digital Age, I expect that augmentation of natural reality using virtual elements will increase significantly. Let’s define virtual elements as something that shows these three main features -
They are mental constructs that can be perceived by technological means
They are representations of any kind of object, subject, concept or person(s) from our natural reality
They are born out of the interaction between humans and computers
Virtual elements connect the physical reality with our mental realities. Mental realities include cultural structures, linguistic structures, social structures, institutions and hierarchies… basically anything that the human mind gives form to and overlays on top of their experience of natural reality.
Augmentation as a process of the mind
Since virtual elements exist in our mind (Sphere of Imagination), perceived through the use of technology, it is subject to the amorphous quality of the mind as well. This amorphous quality results in augmentation, which enables us to retain perceptions beyond their physical forms and conform them to develop with our needs. Augmentation is what converts perception into mental constructs by processing reality through the faculties of the mind. This implies that we are able to manipulate our perception of reality by simply (re)imagining perceptions. However, this can be a double edged sword - either empowering us to change our lives or causing misinformation and delusion.
Virtual elements are a kind of signifiers, i.e., carriers of meaning, rather than the meaning itself. Sometimes the meaning undergoes many levels of representation and re-representation. We are constantly augmenting our natural reality using these kinds of signifiers; like symbols, signs, language, sounds, gestures, mannerisms, etc… and now virtual elements.
VR and AR is truly a revolution in this regard, because it makes the Sphere of Imagination so accessible and easy to interact with. In VR and AR, the tangibility of the imagination is affirmed and reinforced by interaction and response - generating a participatory feedback loop.
Let’s take Angry Birds AR on the Magic Leap for example - it augments the original concept of the Angry Birds game by reproducing a spatial aspect to it. In the AR version, you can walk around the game and interact with it from different angles. Another example is that of Snapchat’s filters which overlay virtual elements based on face mapping and following our head movement. This creates the participatory feedback loop and augments a pre-existing form or concept from our natural reality.
Based on this, I think there are two main requirements for successful augmentation -
Aesthetic appropriation of virtual elements on the basis of function (in other words, meme generation)
Developing a strong participatory feedback loop.
Aesthetic appropriation makes us redefine meaning in terms of our own experience. This helps us change old ideas and create new ones. On the other hand, the participatory feedback loop generates an acceptable Sense of Self to experience with - “I am typing, I am holding a glass, I am walking towards the table” - so on.