Controller Cards: A System for User-defined Input in Augmented/Mixed Reality Settings; A Paradigm Shift During a Pandemic: Understanding the Implications of Virtual Reality Through Analysis of 20th Century and Contemporary Science Fiction

Following a global pandemic, it’s only logical that there have been many efforts to leverage virtual reality (VR) technology in order to improve the experience of doing work from home, with examples such as Facebook’s Infinite Office and Microsoft Mesh: two systems that strive to emulate in-person interactions between individuals as well as possible. My technical and STS projects attempt to improve this prospect as well as understand the implications of a mass-shift to virtual reality in daily life. For my technical project, I designed a human-computer interface for virtual reality, allowing for the use of a small card as a dynamic controller to increase user expressivity. My STS research analyzed science fiction works in order to assess the validity of technological determinism and the relationship of society’s attitude towards technology and innovation in the context of virtual reality.
My technical project was inspired by the September 2020 release of Facebook’s Infinite Office trailer, which depicted a work from home (WFH) platform where VR users could emulate an office space anywhere with a head-mounted display (HMD). Seeking to improve on the cumbersome requirement to carry around peripheral technology such as a mouse and keyboard, I designed a controller in the form of a simple card: the Controller Card. Users may map this card about any movement axis (x,y,z) in order to emulate traditional controllers such as a slider, a mouse, or a volume knob, for example. Such a system enhances the WFH experience by giving users more expressivity and flexibility, these characteristics being fundamental to the appeal of virtual reality over less contemporary platforms such as mouse/keyboard and simple screens.
In contrast to my technical project, my STS research assessed the relationship between society and technological development by analyzing two science fiction works, the Pedestrian by Ray Bradbury and Ready Player One by Ernest Cline, which are both centered on the implications of new technology dominating the landscape civilization, tying findings to the current move towards common use of virtual reality. Christian Kerschner and Melf-Hinrich Ehler’s Attitudes Towards Technology (ATT) framework was used to understand the attitudes towards television for the Pedestrian and virtual reality for Ready Player One. My research revealed that where Ray Bradbury’s work reflected a pessimistic perspective of automatic determinism, or more commonly referred to as traditional technological determinism, Ready Player One exhibited a Social Forces deterministic attitude towards technology. This inconsistency between attitudes was subsequently incapable of validating or invalidating the ideology of technological determinism, which serves as a primary word of caution against the introduction of technology prominent in society. In conclusion, both works suggest that reliance on a deterministic ideology in the context of technological innovation neglects the fact that we make critical decisions during development which subsequently influence society’s relationship with technology.
My technical and STS projects together contribute to the discussion of moving towards virtual reality for WFH by both improving the aforementioned experience and providing insights towards the effect of this shift on society through literary analysis. Through the process of developing my research paper I was better able to understand the large-scale impact of my controller design, where in the preliminary design stage I had only been focused on how users would receive and use the product. These findings are important towards ethically developing the future of virtual reality, placing an emphasis on considerations regarding the virtual community as a whole rather than interactions between individuals and VR technology.
The technical side of my work would not be possible without the valuable mentorship I received under my advisor, Prof. Seongkook Heo. I am grateful to have been a part of his Human Computer Interactions research group, where I was able to share my findings and progress with peers, as well as learning about their work and meeting individually to problem-solve and collaborate. Seongkook also gave me the opportunity to attend UIST 2020, where I viewed multiple presentations on new technology relevant to my project and engaged with other members of the HCI community. My weekly meetings with Seongkook were not only important for gaining insights and advice regarding the development of my project, but served as a setting for discussing current events in HCI research and learning about the industry as a whole. Through the course of a project that assumed many different forms in the face of a slew of challenges, Seongkook helped me understand that research is never executed quite as planned and the process of trial and error is an opportunity in itself to gain new knowledge.

School of Engineering and Applied Science

Bachelor of Science in Computer Science

Technical Advisor: Seongkook Heo

STS Advisor: Kathryn Neeley

Technical Team Members: Conor Frymire