Creating Digital Twins of Smart Sensors for the HoloLens 2; The Effects of Socialization in Virtual Reality on Self-Confidence

This thesis portfolio details two projects demonstrating different uses of emerging extended reality technology. The STS research paper focuses on the positive impact that social virtual reality (VR) platforms can have on users' self-confidence in non-virtual settings. This research was motivated by the recent growth of the VR industry, particularly with the relatively widespread adoption of the Meta Quest 2. The technical portion of this portfolio describes the development of an augmented reality (AR) app for a Microsoft Hololens 2 to provide easier access to smart sensor readings. This project provides a look into the future uses of augmented reality technology despite the lack of growth that it has seen relative to VR technology. In conjunction, these projects exemplify the variety of uses for new extended reality technology in different contexts.
The technical portion of this thesis portfolio focuses on an app that provides access to sensor data from a large number of smart sensors placed around the Link Lab in Olsson Hall. Most of the sensors throughout the Link Lab have no way of indicating their readings to people. Instead, they upload their readings to a database that the Hololens app queries. When a user walks around the link lab, the Hololens uses a cloud-based special recognition system to determine what sensors are nearby. Once the Hololens recognizes where it is in the Link Lab, it creates holograms representing the smart sensors around it and queries the sensor database for the most recent sensor readings to display in a human-readable format on the holograms. Because sending large numbers of HTTP requests to the database is computationally expensive for the Hololens, the holograms are then only updated when the user is looking at them. Ultimately, the app provides easy access to the sensor data collected in the Link Lab by allowing users to physically walk around the space and look at digital representations of the sensors.
The STS research paper argues that social VR, particularly VRChat and Rec Room, supports the development of users’ self-confidence in social interactions in non-virtual settings by providing them with environments that encourage them to practice using their social skills. The research for this paper was conducted through interviews with VRChat and Rec Room players focusing on the players' experiences with the two social platforms and their perceptions of how their self-confidence has changed as a result of those experiences. The interview results suggest that the ease of approaching other players, the ability for players to find and move between different virtual environments based on their interests, and the freedom that they have in how they present themselves in-game are all important elements of the two social platforms that encourage positive social interactions. The interviews also suggest that the level of interaction that players have in VR-based social platforms based around head tracking, hand tracking, and full body tracking is particularly encouraging for positive social interactions. The paper argues that these features of Rec Room and VRChat facilitate the growth of users’ self-confidence by providing them with lifelike environments with real people that encourage socialization.
Analyzing the different features of VRChat and Rec Room that are prominent in supporting users in developing their self-confidence reveals that many of the features are based on accessibility. Whether the features provide access to different environments users can play in or different ways users can present themselves, the easy access that players have to these customizations is important for promoting positive social interactions between players. This idea of the important role that accessibility plays prompts reflection on the impact that accessibility can have on people in other contexts, such as when dealing with the complex system of smart sensors in the Link Lab. It was valuable to consider and reflect on the importance of accessibility across both contexts while developing the Hololens app for the technical portion of this portfolio and while analyzing the research done for the STS research paper.

School of Engineering and Applied Science
Bachelor of Science in Computer Science
Technical Advisor: Brad Campbell
STS Advisor: MC Forelle
Technical Team Members: Andrew Tamberrino