What is Augmented Reality – The ‘Rainbow’s End‘?[1]
Augmented reality (‘AR’) is a term used to describe computer generated overlay designed to enhance sensory input. Often, this is done by capturing a real time image and generating additional visuals on a handheld device or screen. Augmented reality is sometimes created via holographic projection in a real life environment or by overlaying written or graphic data onto a device worn on the body (such as Google’s ‘Glass’). AR is used across a range of industries, from marketing, particularly in concept retail, to gamification and increasingly for medical teaching and training. Of course it has been used ‘secretly’ for military purposes for years, but we will confine ourselves here to the consumer end of AR.
Mediated reality (‘MR’) is a more general concept referring to the addition of, or more often removal of, information from a user’s visual perception. Usually ‘mediated reality’ is applied to data to simplify a user’s view of the real world on a smart phone or tablet. For the sake of completeness in these definitions, ‘virtual reality’ is a broad umbrella term, usually used to refer to an entire computer generated environment in which a human participates, usually through a head mounted display, dataglove or similar. Often these are fully immersive as the user swaps all natural sensory input for computer generated artificial stimuli which reads body movements and human reactions and adjusts the sensory input in response. Such VR is found in military flight or parachute simulators. AR and MR are arguably subsets of virtual reality and there is a high degree of cross over between all these fields. These descriptions are not meant to be definitions or terms of art, just general terms of reference for this piece.
How Does AR Work – ‘Virtual Light‘[2]
Technology companies have been experimenting with tracking technology to pin CGI objects into ‘true’ reality since the 1970s and the first recognised AR protocols were developed for tracking in the mid 1980s and 1990s.
Generally AR software uses tracking technology; ‘reading’ the environment recorded by a camera (eg on the user’s desktop or smart phone) and placing the computer generated ‘augmented’ item correctly in that environment by using visual markers such as QR or barcodes or embedded algorithms which recognise pre-defined features (such as a building or face) which assist to place the AR image correctly in the context of what is read.
Such marker or tracking technology significantly improves the user experience for virtual try on and gamification, where it allows a user to play their game in their current real life environment and ‘game through it’ in a hybrid of real life and AR. Entertainment is not the only use of course and significant numbers of companies offer advanced AR software to augment products in the medical, exhibition, broadcast, travel and teaching industries. Associating markers for interaction with the virtual environment is used widely in everything from digital cameras to the Wii and whilst the software to place markers is usually proprietary, it is not new. What is new though is use of AR software to create tracking AR images which ‘move’ or ‘track’ within the users real environment, as if reflecting a real product. Such software is being patented across a vast range of applications such as ‘magic mirror’ apps that enable customers try on clothes from a virtual shop at home and see themselves from every view, using their smartphones and then share those AR images. There are apps to measure and view furniture in situ using 360 views of the users ‘real life’ room, overlaid with a product in situ. The Glasgow School of Art and NHS Education Scotland recently introduced an AR program for student dentists to practice drilling on ‘virtual teeth’ and the list goes on. The application of AR seems to be infinite even if the take up has been slow.
The next step in AR is probably a version of ‘virtual light’. Scientist Stephen Beck first coined this term to refer to technology that is capable of sending a signal direct to the user’s optic nerve, rather like a bionic eye that can perceive light as a human eye does, but which can also receive data. In the book of the same name, William Gibson envisaged use of sinister data-carrying sunglasses (go with it for now) that contained plans to flatten and rebuild San Francisco. The idea that sunglasses could contain data which would be uploaded to present the ‘new view’ of a current street was pure science fiction in the early 1990’s, but seems less so now, when most streets are mapped by Google and it has launched ‘Glass’. With a price point of $1,500, ‘Glass’ is hardly mainstream, but the move by Paul Deneve (former Saint Laurent CEO) to Apple to work on ‘special projects’ suggests an increasing industry focus on worn tech. Once consumer devices comfortably offer users the ability to view AR and MR such as street maps, photos, immediate internet connectivity and personal contact details and to activate audio response data streams in the wearer’s field of vision, the need for a handheld device is removed. Add voice activated VoIP and a data glove or tracking technology that maps hand movements to a heads up screen for typing and suddenly smartphones and tablets seem very old school.
Does widespread adoption of such products mean that the virtual light is already starting to shine? Working with the highly creative tech companies I have as clients, I would say an emphatic ‘yes’.
Experiential Explosion – ‘The Digital Sea‘[3]
The Digital Sea is another sci-fi novel that imagines a world where the whole of the human experience can be uploaded and shared between the populace in a multitude of ways. In this book, the ‘Sea’ is manipulated at will by a dystopian corporation, for its own means. How can we assess what the human experience is in our natural, rather than a sci-fi world? I would suggest that Facebook, with over 120 billion page impressions (at the end of 2012) is a pretty good place to start. Ordinary people share words and pictures of everything from the anticipated highs and lows of ‘hatch match and dispatch’, to alarmingly frank opinions on every topic imaginable. This is their personal reality.
What if a computer generated range of products, people, places and sensory experience could be overlaid to these ‘natural’ experiences? AR can be readily shared and there are already numerous programs and apps to enable distribution and joint play; from digital try-on (‘Does my bum look big in this?’ at a distance) to teaching children in central Africa or undertaking AR led surgery in Borneo? If our day-to-day experience can be added to, presumably in a way we find helpful, entertaining or meaningful, why wouldn’t we do this? There are already programs that track our online behaviour and push advertising content to us, so isn’t AR an obvious next step in expanding that experience by making advertising more ‘real’, more meaningful and more immediate?
As sharing and storing of AR occurs, the digital sea becomes brackish water mixing natural and augmented human experience. In legal terms, muddy water is always abundant feeding ground for lawyers and especially so for the bottom feeders who assist patent trolls!
‘Getting To The Point‘[4]
Reading this I may have piqued your legal interest and your considerable intellect may be churning long-ago-learnt principles, statute or cases. Alternatively you may be thinking ‘get to the point’, so I will. AR is a hot legal topic with impact on many elements of our daily lives. Specific law of AR is unwritten, but many existing areas apply. Just as a taster, there are data protection rights; data transfer issues; personality rights; privacy issues; trade mark rights; patent rights; copyright (in all its forms); defamation; social media regulation; advertising standards; broadcasting rights; and let us never forget the threat to innovation from patent trolls.
In the next article, I will look at these issues in more depth and share my experience of dealing with the conflict of laws, a lack of regulation and the sometimes frustrating legal obstacles I have encountered in this exciting field.
Joanne Frears is a Solicitor and Head of Intellectual Property at Jeffrey Green Russell Limited
[1] (Vernor Vinge, 2006)
[2] William Gibson, 1994
[3] Thomas K Carpenter, 2012
[4] Jeff Lynne 1986
