Enhancing Robustness in Author Verification Systems: Obfuscation-Based Adversarial Attacks on BigBird; Analyzing the Attitude of Authors toward the Use of their Text to Train A.I. Models

The intersection of artificial intelligence (AI) and natural language processing (NLP) with online text data presents a complex sociotechnical landscape, where technological advancements collide with ethical and legal considerations. My concurrent exploration of a technical capstone project focusing on developing obfuscation-based adversarial attacks against AI models, particularly the BigBird model for authorship verification (AV), and an STS research paper delving into the ethical implications of using fanfiction data for AI model training, forms the foundation for this sociotechnical synthesis. The evolution of AI technologies has increasingly intertwined with societal concerns, particularly regarding data privacy, algorithmic bias, and intellectual property rights – as such, by addressing these concerns in tandem with technical innovations, my work aims to contribute to a more nuanced understanding of the sociotechnical dynamics shaping the future of AI.

In my capstone project, I tackled the challenges faced by author verification (AV) systems, pivotal in real-world scenarios like forensic analysis and plagiarism detection. Specifically, I developed obfuscation-based attacks against the state-of-the-art BigBird model, constructed using the PAN20 task’s dataset comprising 53,000 text pairs. Leveraging lexical substitution techniques, backtranslation, and large language model (LLM)-based paraphrasers, my approach aimed to obscure one document in a true trial text pair, previously verified by BigBird, to the extent that the AV system misclassifies the pair as a false trial. Crucially, this document’s original meaning had to be preserved, a criterion evaluated through various content preservation metrics such as QuestEval. Preliminary findings showcased promising results, particularly with backtranslation employing specific intermediary languages and LLM-based paraphrasers with basic prompts. However, this initial success underscores the need for further refinement and exploration. Future iterations could focus on enhancing the efficacy of existing methods or devising novel obfuscation techniques. Such endeavors hold the potential to yield valuable insights into the impact and constraints of obfuscation-based attacks on AV systems. By shedding light on the strengths and limitations of obfuscation-based approaches, my research contributes to the ongoing efforts aimed at fortifying the reliability and security of AV systems, thereby bolstering their effectiveness in critical applications.

My STS research paper delved into the intricate ethical dimensions surrounding the use of online text data, particularly fanfiction, within the realm of AI model training. By exploring the multifaceted perspectives of stakeholders, including authors, technologists, legal experts, and the broader public, I sought to illuminate the complex sociotechnical dynamics shaping the discourse on creativity, ownership, and technological innovation. Drawing on insights from the Social Construction of Technology (SCOT) framework, I analyzed the co-construction of AI systems and societal norms, emphasizing the reciprocal influence between technological advancements and cultural practices. This analytical lens facilitated a nuanced understanding of the divergent viewpoints surrounding AI-generated content, ranging from concerns over copyright infringement to the potential democratization of creative tools. As highlighted in the conclusion of my research, the debate surrounding the ethical implications of AI-generated content underscores fundamental clashes of perspectives between content creators and technologists: while authors advocate for fair compensation and attribution for their creative works, AI technologists argue for the transformative nature of AI-generated content and its potential to foster innovation. By interrogating the intersections of technology, creativity, and ownership, my work aims to inform ethical regulatory frameworks conducive to technological innovation and creative expression.

Engaging in both the technical and sociotechnical dimensions of AI research concurrently has provided invaluable insights into the intricate interplay between technology and society. Through the development of adversarial attacks against AI models, I gained a deeper appreciation for the ethical considerations inherent in AI research, particularly concerning data privacy and algorithmic fairness. Similarly, my exploration of fanfiction data's ethical implications underscored the importance of considering the societal ramifications of AI advancements, emphasizing the need for collaborative dialogue and equitable frameworks to address the ethical challenges posed by the intersection of AI and creative expression. By bridging the gap between technical expertise and social awareness, I have better understood the importance of responsible and equitable advancement of AI technology in the digital age, fostering a future where innovation is guided by ethical principles and a commitment to societal well-being.

School of Engineering and Applied Science

Bachelor of Science in Computer Science

Technical Advisor: Kenny Alperin, Trang Nguyen

STS Advisor: MC Forelle