Comprehensive Analysis of Software Testing for Intrinsically Challenging Systems; The Societal Impact of Untested Software: A Boeing Case Study

Software has become a crucial component of daily life, with many systems relying on and leveraging the power of software to operate and provide services. Public perception of software is mainly that of a black box, where things simply work, as companies are entrusted to handle the logic and logistics on the backend. The problem the technology industry faces is determining how to test software as it becomes integrated into more complex systems since software can only be tested on a good enough basis. The technical report examines the weaknesses of current state-of-the-art mobile application testing tools and techniques and assesses the improvements needed to fill the gap towards improving testing and increasing its accessibility. The STS research paper explores a case study of Boeing’s 737 MAX failures, highlighting the ethical responsibility companies are entrusted with when deciding how and how much their software is tested. Both reports revolve around the role and importance of software testing, as companies are tasked with the responsibility of protecting their customers.

The technical report aims to analyze the current challenges of software testing in mobile applications. Any piece of software can never be fully tested, which makes it unclear what constitutes “sufficient testing”, but inherent properties of mobile application software further complicate the feasibility of comprehensive testing. To better understand current practices, techniques, and proposed solutions, a meta-analysis was conducted. This study aims to identify the limitations of current techniques and determine the improvements required to create more effective and flexible approaches. This would outline the steps required to bridge the gap toward an industry-wide adoption of set testing standards. Obligating all technology companies to agree and satisfy bare-minimum testing standards, especially as intrinsically challenging software continues to be applied to security- and safety-critical applications, would not be plausible without first improving the accessibility and effectiveness of software testing.

The STS research paper analyzes the ethical dilemma companies must consider when releasing software. With people relying on more software-driven products and services every day, it is evident that software has infiltrated every aspect of daily life, however, due to intrinsic properties no piece of software can ever be perfect. How, then, do companies decide when software is tested enough? What happens when companies prioritize profit-driven decisions over the company’s ethical obligation to its clients? With the advent of new technology, it is important to consider the ethical implications of releasing software before the developers or the users completely understand how it works. More specifically, what societal ramifications are caused by the inability to thoroughly test software before it is released? Through the dissection of a case study involving the software failure of Boeing’s MCAS which caused the death of hundreds, the profit-driven decisions and ethical misjudgments of the actors involved will be analyzed. With the facts presented, the goal is to provide a better understanding of the ethical responsibilities technology companies must consider, beyond the typical testing considerations, when creating a piece of software. As well as arming developers with the necessary knowledge to make better judgment calls and the ability to call out when a company is neglecting its ethical duties.

Given that both papers are tightly coupled around the subject of software testing, the simultaneous completion of both provided greater insight into the technical and social aspects of software testing. This could then be leveraged in both papers, allowing for a greater understanding of the limitations of current tools and techniques used in software testing beyond the technical limitations. Conversely, it provided a deeper comprehension as to why companies would be reluctant to extensively test their software, as they know no amount of testing will ever be enough. This knowledge was beneficial in the technical report, as proposed solutions were evaluated based on their accessibility and feasibility, a crucial barrier to entry that must be dismantled. Improving software testing in these areas along with the technical element, would then ease the software testing burden on companies, ensuring that they take the precautions necessary to certify the safety and security of their software. Conclusions that would have been much harder to reach, had the papers been completed separately.

School of Engineering and Applied Science

Bachelor of Science in Computer Science

Technical Advisor: Rosanne Vrugtman

STS Advisor: Bryn Seabrook