Metro Train

UIST Vision Talks

08:30 - 09:30 Central Daylight Time (UTC-5)

The Augmented Conversation and the Amplified World

David A. Smith,
CEO, Croquet Corporation

Human communication mediated by computers and Augmented Reality devices will enable us to dynamically express, share and explore new ideas with each other via live simulations as easily as we talk about the weather. This collaboration provides a “shared truth” — what you see is exactly what I see, I see you perform an action as you do it, and we both see exactly the same dynamic transformation of this shared information space. When you express an idea, the computer, a full participant in this conversation, instantly makes it real for both of us enabling us to critique and negotiate the meaning of it. This shared virtual world will be as live, dynamic, pervasive, and visceral as the physical.

Augmented Reality is not just the next wave of collaboration and computing, it is a fundamental shift in how we will engage with our world and each other, and how we will understand and solve the huge problems we face as a species. AR will replace your PC, your phone, your tablet. It will be an always on and always on YOU supercomputer — it will amplify our intentions and ideas enabling all of us to create and explore new universes together.

AR MUST be interoperable between all users and between all devices at all times. Microsoft’s DOS operating system enabled the rise and dominance of the personal computer. It did this by ensuring that every system and every piece of software would interoperate on every PC. It was the single most important factor that drove the intense growth of that industry. Likewise, every phone, regardless of operating system, brand, or carrier can be used to call every other phone in the world. This is a little remarked but extremely significant requirement.

We need to build the DOS of AR — a platform that enables the interoperability, scaling, and development necessary to energize this new ecosystem. Not only must it provide a common platform for applications in the way DOS did, but it needs to enable every AR device to connect and communicate with every other device in the same way the phone does. Every user needs to share and collaborate with every other user in this enhanced digital world — regardless of their particular walled-garden allegiance.

The virtual will not just become as real as real life — it will become real life. We will soon live together INSIDE a shared information space that co-exists with and amplifies the physical plane of existence. Communication, far more than anything else, defines the true value of Augmented Reality. AR displays are transparent not just so we can see the real world with a digital overlay, but because we need to look another person in the eyes as we engage with them in this extended digital space. The emerging AR Cloud is not simply about adding a new digital layer to the real world, it is about using AR to expand the scope of how we collaborate, create and share ideas with each other within it. This next generation of computing capability will not only allow us to extend and annotate the real world — far more importantly, it will allow us to easily and instantly create and explore completely new universes that we will build from scratch as part of our everyday discussions.

AR extends the nature and power of the human narrative. We are defined more by how we communicate that anything else. The very foundation of what it means to be human is about to change.

Democratizing the Production of Interactive Hardware

Steve Hodges,
Microsoft Research, Cambridge, UK


Whether it is intentional user interaction, continuous context or environmental sensing, or situated information display, we are dependent on interactive hardware to provide the vital connection between our ever-expanding networked computer systems and the real world. Indeed, as the “internet of things” computing paradigm becomes established, many predict a dramatic growth in the number and type of these interactive devices. A key factor in this growth is the ability of research communities like UIST to explore new device forms and functions, with the ultimate aim of unlocking new capabilities and experiences.


The development of new interactive hardware can be split into two phases. Firstly, a period of ideation, prototyping and design iteration leads to new device concepts. Hardware researchers in the UIST community often excel at this! Many of us have even created systems and platforms to aid other researchers in their prototyping endeavors – reducing the time, money and/or level of expertise needed. Building prototypes has become democratized.

The second phase typically involves creating tens to thousands of copies of a prototype – either pre-production evaluation units or low-volume product. However, based on my own first-hand experience coupled with interviews with many others, this transition is often unexpectedly difficult. Certainly, the availability of cheap printed circuit board (PCB) manufacturing and assembly services can be enough. But as the complexity of a device moves beyond a single, routine PCB, and as the scale rises from tens to hundreds or thousands of units, the complexity and cost of reliable production can become overwhelming.


The underlying difficulty with transitioning from research prototyping to reliable production comes in the form of the non-recurring engineering (NRE) activities necessary after the prototyping phase. For example, extensive design verification and testing is needed to ensure that the performance shown by a prototype can be replicated consistently. Furthermore, each production unit must be tested to ensure it was assembled correctly, requiring a custom test jig and procedure to be developed – a process that can take longer than designing the product itself! And a final example: an early production unit must be tested for compliance with the regulations of the country/ies in which the devices will be deployed – often necessitating expensive design iteration.

Traditional solutions to the NRE activities listed above require costly expertise and equipment, meaning they are typically only cost-effective for mass production, i.e. in volumes of at least 10k units and often over 100k.


However, there is some good news. Just as our research community has developed a myriad of tools and techniques to reduce the cost and complexity of developing and deploying software, I believe new approaches can democratize hardware production, making it viable in much lower volumes.

Imagine “DevOps for hardware” – everything from circuit simulation tools that predict worst-case operating conditions and thereby simplify design verification, to re-usable hardware test jig designs that are readily ‘forked’ and adapted to suit a new device. Continuous integration and test-driven development could reduce or even eliminate regressions during the hardware design process, just as they do for software. Perhaps an augmented reality telepresence experience could allow device creators to ‘visit’ their chosen factory/ies more often, helping them understand the nuances of the production process and to pick up on and rectify potential issues. Domain-specific and AI powered tools could track the progress of a project, and automatically highlight the most relevant potential pitfalls.


I believe that with new tools and techniques such as those listed above, we can make low volume hardware device production viable. Furthermore, we can also support organic scaling, from tens of units through to hundreds and thousands. In this way, hardware innovations from the UIST community and beyond can have more impact. It will be easier for researchers to evaluate new ideas in the hardware space. Start-ups will face less risk when commercializing new hardware concepts, giving them headroom for iteration in search of product-market fit. Small companies will be able to manage a portfolio of niche but viable hardware devices, addressing the minority needs of individuals and communities. And larger organizations will be better placed to evaluate inventive ideas.

Vision Chair

arrow-up icon