Enhancing flight testing leveraging software testing techniques implemented in model-based systems engineering

Abstract

The Department of Defense (DoD) is significantly shifting toward digital engineering across all systems engineering lifecycle phases. A vital aspect of this transformation is the adoption of model-based testing methodologies within the Test and Evaluation (T&E) processes. This dissertation investigates a grey box Model-Driven Test Design (MDTD) approach that leverages model-based systems engineering (MBSE) artifacts to create flight test scenarios and plans and compares this novel approach to the traditional document-centric methods. The study utilizes the Systems Modeling Language (SysML) to represent artifacts, enabling a comparative analysis between traditional and MDTD processes. Through a case study involving a training system used by the Air Force Operational Test and Evaluation Center (AFOTEC), the dissertation evaluates the MDTD process's effectiveness in generating validated test scenarios and plans that align with established methods. Two additional case studies demonstrate the reuse of SysML elements across different systems under test (SUT), highlighting the benefits, costs, and practical applications of this approach in operational flight testing. The findings include metrics such as Model Reuse Percentage (MR%), Reuse Value Added (RVA), and System Usability Survey (SUS) scores, which measure the reusability of model artifacts and the usability and effectiveness of the "AFOTEC Methodology" model approach in generating flight test plans. This research underscores the importance of model-based testing in operational flight testing and supports the DoD T&E community's ongoing move toward a fully integrated digital engineering ecosystem.