Virtual Experiments — a Lab in the Cloud

Measuring properties of things, be it molecules, computer systems, or markets, is the heart of the scientific method. Many of these real-world measurements can be simulated or replicated on a computer by physics simulations, data-driven, or inference-based methods. We call the combination of setup, compute, and analysis steps which measure a particular characteristic of an input system on a computer, a virtual-experiment.

Virtual experiments have unique characteristics compared to other workflows of tasks: multiple virtual-experiments could be available for measuring the same characteristic using different methods or tools, each best suited for a particular set of objectives; they could also require long execution times and process or produce large amounts of data over many nodes, necessitating a robust support to ensure completion, re-use of outputs from previous calculations, and efficient handling of data movement between steps of the virtual-experiment.

Since 2015 we have been developing technologies for developing and executing virtual-experiments driven by our work with collaborators in UK and beyond on materials design. These technologies are now open-sourced as part of the Simulation Toolkit for Scientific Discovery (ST4SD).

A cloud and HPC toolkit that supports AI-surrogate development

The ST4SD toolkit enables scientists to create and run virtual-experiments. It provides features like flexible memoization, robust execution support, and deployment of the same virtual-experiment across classic HPC and cloud. Additionally, we have included specific support for creating and using AI surrogates of physical models, with the ability to automatically create AI-powered surrogate versions of physics-based virtual-experiments once the core surrogate model is provided.

Easily consume state-of-the-art computational methods

We believe virtual-experiments should be usable by all researchers who need them and not just those who are experts in computational methods. To enable this, we have built a publicly available virtual-experiment registry which allows scientists to share virtual-experiments, and other tools to automatically consume and use them. Follow this example to quickly run an experiment on a nanoporous material on your laptop.