This project aims to provide relevant feedback on the reliability of Deep Learning models, as well as, useful improvement strategies to construct trustworthy systems. The novel search-based testing approaches are applied to aircraft design and performance modeling.
As a means of assessing aircraft performance, data-driven models can be used to derive statistical models based on experimental measurements with less detailed knowledge of the system than physics-based modeling. However, this economical model development also raises new challenges when it comes to trustworthiness. Although search-based testing approaches for statistical models have shown effectiveness in synthetic test input generation, they rely on invariant perturbations within limited boundaries to explore the neighbors of each existing data point. In the context of regressive modeling using sensor data, such invariance rules are very limited, making it difficult to generate large-scale synthetic inputs and affecting the test’s assessment of model trustworthiness. For that reason, our research project makes use of the foreknown physics first principles and the system-related design properties in order to expand the search space of synthetic inputs. Our produced synthetic data will challenge the statistical model against physics-grounded craft test cases to reveal inconsistencies of the behavioral model with apriori domain knowledge.
Team
Datasets used for experiments
Real flight test data includes sensor data containing features related to flight conditions, wing configurations, and high-pressure pneumatic system conditions at the wing root. Aircraft design simulation includes both aircraft design variables (related to the geometrical shape, engine thrust, etc.) and aircraft performance criteria (e.g., balanced field length, time to climb, maximum take-off weight, etc.)