FAST: Developing AI-powered fast, self-driving microscopes

PI Mathew Cherukara, Photon Sciences
Rendering of technology

Figure 1: Left: Representation of FAST, an AI running on an edge computing device, steering a fast-scanning microscope to regions of scientific interest. Right: Scan of a WSe2 thin film. (A) Points measured at 20% scan coverage. (B) Estimated sample at 20%. (C) Ground truth obtained with 100% coverage. Figure reproduced from Kandel, S., Zhou, T., Babu, A.V. et al. Demonstration of an AI-driven workflow for autonomous high-resolution scanning microscopy. Nat Commun 14, 5501 (2023).

Project Summary

New scientific instruments can image samples at high resolutions and frame rates, but this comes with large data and sample dose requirements. Autonomous control with the AI-driven Fast Autonomous Scanning Toolkit (FAST) software achieves a 70% reduction in data & dose required for a scanning microscopy experiment. 

Project Description

The most advanced scientific instruments, such as those in the upgraded Advanced Photon Source (APS-U), will enable very fast imaging at extremely high resolutions, but these experiments will have a commensurate cost in the volume of data generated and the potential beam damage of the sample. For example, x-ray imaging of a 1mm3 volume at 10 nm resolution would require exposure to more than 1017 photons and generate 1015 voxels of data. However, the information of interest is often concentrated in sparse regions that contain interfaces, defects, or other structural elements. Therefore, smart experimental controls that dynamically direct the microscope to focus on the regions of interest becomes vital.

The Fast Autonomous Scanning Toolkit (FAST) combines a neural network, route optimization, and modular hardware controls to make on-the-fly sampling and scan path choices for a synchrotron-based scanning microscopy experiment. It is trained only on a generic natural image, making it sample-agnostic. LCRC resources were used in training FAST. In tests at a fast-scanning microscope, adaptive scans of less than 30% of the sample were sufficient to produce high-fidelity image of the sample. In the process, the FAST computation itself only added a 2% time overhead per acquisition step.

Kandel, S., Zhou, T., Babu, A.V., Di, Z., Li, X., Ma, X., Holt, M., Miceli, A., Phatak, C. and Cherukara, M.J., 2023. Demonstration of an AI-driven workflow for autonomous high-resolution scanning microscopy. Nature Communications, 14(1), p.5501.