Executive summary
Autonomous space systems increasingly rely on AI and machine learning to interpret data, support decisions, and respond to conditions that cannot be managed in real time from Earth. VISIMO developed Model Validation via Precomputed Transformations (MVPT) through a NASA SBIR Phase I effort to explore whether model-validation evidence could be computed before a mission and then retrieved quickly when autonomous systems encounter new or shifted data. The Phase I effort completed all three technical objectives, reached 0.95 +/- 0.05 Fraction of Correctly Computed Validations in proof-of-concept testing, and established a Phase II pathway for deep-space rotorcraft autonomy.
Challenge
AI can help autonomous spacecraft, rovers, rotorcraft, and other mission systems make decisions faster than human operators can support from Earth. But high-consequence autonomy depends on confidence. Mission teams need to know when a model can be trusted, when it is operating outside expected conditions, and when fallback procedures may be necessary.
- AI models may encounter data that differs from pre-mission training or validation data.
- Traditional validation can be computationally expensive and difficult to repeat onboard.
- Mission systems need fast, reviewable estimates of whether models remain valid.
Problem requirements
The Phase I effort needed to determine whether MVPT could provide a technically credible path for AI and machine-learning validation under novel mission conditions. The work had to show compatibility across NASA-relevant models and data, demonstrate a working proof-of-concept, and create a specific Phase II scenario for more mission-relevant simulation and testing.
- Assess compatibility across model families, validation methods, and data modalities.
- Build and test a proof-of-concept with real NASA-relevant data.
- Define a Phase II test path for autonomous mission software.
Solution
MVPT is built around a simple but powerful idea: perform expensive validation work before the mission, then use fast lookup and similarity methods during the mission. Before deployment, the system evaluates how a model behaves across many transformed versions of known data. When new data arrives, the system can compare it to that precomputed validation space and estimate whether the model remains reliable without recomputing full validation onboard.
- Precompute validation behavior across plausible data transformations.
- Use similarity search to retrieve relevant validation evidence when new data arrives.
- Support fast runtime estimates of whether a model remains valid under shifted conditions.
Implementation
VISIMO pursued two research tracks. First, the team conducted a structured compatibility analysis across NASA-relevant machine-learning models, validation methods, and data modalities. Second, the team built a proof-of-concept around NASA solar wind data from the Deep Space Climate Observatory. Historical solar wind data was separated into quiet and volatile regimes so the demonstration could evaluate how model-validity estimates changed when a probabilistic forecasting model encountered data outside its nominal training regime.
Results
MVPT successfully proved Phase I feasibility. VISIMO completed all three Phase I technical objectives and achieved a final 0.95 +/- 0.05 Fraction of Correctly Computed Validations in the proof-of-concept. The compatibility analysis showed a 0.979 +/- 0.015 compatibility fraction across reviewed machine-learning model families, while validation-method and data-modality compatibility were statistically consistent with the project's 0.9 benchmark. The proof-of-concept used real NASA solar wind data and tested model behavior under quiet and volatile regimes, where the volatile regime created a meaningful out-of-sample validation case. The project also produced a concrete Phase II path focused on a deep-space rotorcraft using camera-based navigation and hazard-detection models.
- Completed all Phase I objectives and produced a mission-relevant Phase II simulation plan.
- Reached 0.95 +/- 0.05 FCCV in proof-of-concept validation-state testing.
- Showed broad compatibility with reviewed NASA-relevant AI model families.
