The advent of modern, high-speed electron detectors has made the collection of multidimensional hyperspectral transmission electron microscopy datasets, such as 4D-STEM, a routine. However, many microscopists find such experiments daunting since analysis, collection, long-term storage, and networking of such datasets remain challenging. Some common issues are their large and unwieldy size that often are several gigabytes, non-standardized data analysis routines, and a lack of clarity about the computing and network resources needed to utilize the electron microscope. The existing computing and networking bottlenecks introduce significant penalties in each step of these experiments, and thus, real-time analysis-driven automated experimentation for multidimensional TEM is challenging. One solution is to integrate microscopy with edge computing, where moderately powerful computational hardware performs the preliminary analysis before handing off the heavier computation to high-performance computing (HPC) systems. Here we trace the roots of computation in modern electron microscopy, demonstrate deep learning experiments running on an edge system, and discuss the networking requirements for tying together microscopes, edge computers, and HPC systems.

A real-time image reconstruction method for scanning transmission electron microscopy (STEM) is proposed. With an algorithm requiring only the center of mass of the diffraction pattern at one probe position at a time, it is able to update the resulting image each time a new probe position is visited without storing any intermediate diffraction patterns. The results show clear features at high spatial frequency, such as atomic column positions. It is also demonstrated that some common post-processing methods, such as band-pass filtering, can be directly integrated in the real-time processing flow. Compared with other reconstruction methods, the proposed method produces high-quality reconstructions with good noise robustness at extremely low memory and computational requirements. An efficient, interactive open source implementation of the concept is further presented, which is compatible with frame-based, as well as event-based camera/file types. This method provides the attractive feature of immediate feedback that microscope operators have become used to, for example, conventional high-angle annular dark field STEM imaging, allowing for rapid decision-making and fine-tuning to obtain the best possible images for beam-sensitive samples at the lowest possible dose.

The internal electric field of a 2D P-N junction of a semiconductor is mapped out by two techniques: measuring the deflection of the transmitted beam in micro-STEM mode with acquisition and data fitting of an un-scattered beam image, and through the derivative of electrostatic potential maps by dual lens electron holography. Comparable results of the P-N junction internal electric field measured with these two techniques are reported.

Recent work has revived interest in the scattering matrix formulation of electron scattering in transmission electron microscopy as a stepping stone toward atomic-resolution structure determination in the presence of multiple scattering. We discuss ways of visualizing the scattering matrix that make its properties clear. Through a simulation-based case study incorporating shot noise, we shown how regularizing on this continuity enables the scattering matrix to be reconstructed from 4D scanning transmission electron microscopy (STEM) measurements from a single defocus value. Intriguingly, for crystalline samples, this process also yields the sample thickness to nanometer accuracy with no a priori knowledge about the sample structure. The reconstruction quality is gauged by using the reconstructed scattering matrix to simulate STEM images at defocus values different from that of the data from which it was reconstructed.

The use of fast pixelated detectors and direct electron detection technology is revolutionizing many aspects of scanning transmission electron microscopy (STEM). The widespread adoption of these new technologies is impeded by the technical challenges associated with them. These include issues related to hardware control, and the acquisition, real-time processing and visualization, and storage of data from such detectors. We discuss these problems and present software solutions for them, with a view to making the benefits of new detectors in the context of STEM more accessible. Throughout, we provide examples of the application of the technologies presented, using data from a Medipix3 direct electron detector. Most of our software are available under an open source licence, permitting transparency of the implemented algorithms, and allowing the community to freely use and further improve upon them.