Hostname: page-component-76fb5796d-r6qrq Total loading time: 0 Render date: 2024-04-29T01:09:35.870Z Has data issue: false hasContentIssue false
  • English
  • Français

Analysis of Interpretable Data Representations for 4D-STEM Using Unsupervised Learning

Published online by Cambridge University Press:  08 September 2022

Alexandra Bruefach Open the ORCID record for Alexandra Bruefach [Opens in a new window] ,
Colin Ophus  and
Mary C. Scott
Alexandra Bruefach
Affiliation:
Department of Materials Science and Engineering, University of California, Berkeley, CA 94720, USA
Colin Ophus
Affiliation:
National Center for Electron Microscopy, Molecular Foundry, Lawrence Berkeley National Laboratory, 1 Cyclotron Road, Berkeley, CA 94720, USA
Mary C. Scott*
Affiliation:
Department of Materials Science and Engineering, University of California, Berkeley, CA 94720, USA National Center for Electron Microscopy, Molecular Foundry, Lawrence Berkeley National Laboratory, 1 Cyclotron Road, Berkeley, CA 94720, USA
*
*Corresponding author: Mary C. Scott, E-mail: mary.scott@berkeley.edu
Get access

Abstract

Understanding the structure of materials is crucial for engineering devices and materials with enhanced performance. Four-dimensional scanning transmission electron microscopy (4D-STEM) is capable of mapping nanometer-scale local crystallographic structure over micron-scale field of views. However, 4D-STEM datasets can contain tens of thousands of images from a wide variety of material structures, making it difficult to automate detection and classification of structures. Traditional automated analysis pipelines for 4D-STEM focus on supervised approaches, which require prior knowledge of the material structure and cannot describe anomalous or deviant structures. In this article, a pipeline for engineering 4D-STEM feature representations for unsupervised clustering using non-negative matrix factorization (NMF) is introduced. Each feature is evaluated using NMF and results are presented for both simulated and experimental data. It is shown that some data representations more reliably identify overlapping grains. Additionally, real space refinement is applied to identify spatially distinct sample regions, allowing for size and shape analysis to be performed. This work lays the foundation for improved analysis of nanoscale structural features in materials that deviate from expected crystallographic arrangement using 4D-STEM.

Keywords

4D-STEMdiffractionmachine learningnanomaterialsscanning transmission electron microscopy
Type
Software and Instrumentation
Information
Microscopy and Microanalysis , Volume 28 , Issue 6 , December 2022 , pp. 1998 - 2008
Check for updates
Copyright
Copyright © The Author(s), 2022. Published by Cambridge University Press on behalf of the Microscopy Society of America

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Allen, FI, Pekin, TC, Persaud, A, Rozeveld, SJ, Meyers, GF, Ciston, J, Ophus, C & Minor, AM (2021). Fast grain mapping with sub-nanometer resolution using 4D-STEM with grain classification by principal component analysis and non-negative matrix factorization. Microsc Microanal 1, 1–10.Google Scholar
Bellman, RE (1957). Dynamic Programming. Princeton, New Jersey: Princeton University Press.Google ScholarPubMed
Bruma, A, Santiago, U, Alducin, D, Plascencia Billa, G, Whetten, RL, Ponce, A, Mariscal, M & Jose-Yacaman, M (2016). Structure determination of superatom metallic clusters using rapid scanning electron diffraction. J Phys Chem C 120, 1092–1908.CrossRefGoogle Scholar
Cautaerts, N, Crout, P, Anes, HW, Prestat, E, Jeong, J, Dehm, G & Liebscher, CH (2022). Free, flexible and fast: Orientation mapping using the multi-core and GPU-accelerated template matching capabilities in the python-based open source 4D-STEM analysis toolbox Pyxem. Ultramicroscopy 237, 113517.CrossRefGoogle ScholarPubMed
Cooper, D, Denneulin, T, Bernier, N, Beche, A & Rouviere, J-L (2016). Strain mapping of semiconductor specimens with nm-scale resolution in a transmission electron microscope. Micron 80, 145–165.CrossRefGoogle Scholar
Cowley, JM & Moodie, AF (1957). The scattering of electrons by atoms and crystals. I. A new theoretical approach. Acta Crystallogr 10, 609–619.CrossRefGoogle Scholar
DaSilva, JC, Smeaton, MA, Dunbar, TA, Xu, Y, Balazs, DM, Kourkoutis, LF & Hanrath, T (2020). Mechanistic insights into superlattice transformation at a single nanocrystal level using nanobeam electron diffraction. Nano Lett 20, 5267–5274.CrossRefGoogle Scholar
Deng, HD, Zhao, H, Jin, NL, Hughes, L, Savitzsky, B, Ophus, C, Fraggedakis, D, Borbely, A, Yu, Y-S, Lomeli, E, Yan, R, Liu, J, Shapiro, DA, Cai, W, Bazant, MZ, Minor, AM & Chueh, WC (2022). Correlative image learning of chemo-mechanics in phase-transforming solids Nat Mater 21, 547–554.CrossRefGoogle ScholarPubMed
Duan, X, Yang, F, Antono, E, Yang, W, Pianetta, P, Ermon, S, Mehta, A & Liu, Y (2016). Unsupervised data mining in nanoscale X-ray spectro-microscopic study of NdFeB magnet. Sci Rep 6, 1–8.CrossRefGoogle ScholarPubMed
Dwyer, C, Erni, R & Etheridge, J (2010). Measurement of effective source distribution and its importance for quantitative interpretation of STEM images. Ultramicroscopy 110, 952–957.CrossRefGoogle Scholar
Gammer, C, Ozdol, VB, Liebscher, CH & Minor, AM (2015). Diffraction contrast imaging using virtual apertures. Ultramicroscopy 155, 1–10.CrossRefGoogle ScholarPubMed
Greer, JR (2006). Bridging the gap between computational and experimental length scales: A review on nano-scale plasticity. Rev Adv Mater Sci 13, 59–70.Google Scholar
Grimley, ED, Frisone, S, Schenk, T, Park, MH, Fancher, CM, Mikolajick, T, Jones, JL, Schroeder, U & LeBeau, JM (2018). Insights into texture and phase coexistence in polycrystalline and polyphasic ferroelectric HfO2 thin films using 4D-STEM. Microsc Microanal Conf Proc 24, 184–185.CrossRefGoogle Scholar
Groschner, CK, Choi, C & Scott, MC (2021). Machine learning pipeline for segmentation and defect identification from high-resolution transmission electron microscopy data. Microsc Microanal 27, 549–556.CrossRefGoogle Scholar
Guccione, P, Lopresti, M, Milanesio, M & Callandro, R (2021). Multivariate analysis applications in X-ray diffraction. Crystals 11, 12.CrossRefGoogle Scholar
Guillamet, D, Schiele, B & Vitria, J (2002). Analyzing non-negative matrix factorization for image classification. International Conference on Pattern Recognition 2, 116–119.CrossRefGoogle Scholar
Han, Y, Nguyen, K, Cao, M, Cueva, P, Xie, S, Tate, MW, Purohit, PH, Gruner, SM, Park, J & Muller, DA (2018). Strain mapping of two-dimensional heterostructures with subpicometer precision. Nano Lett 18, 3746–3751.CrossRefGoogle ScholarPubMed
Izadi, E, Darbal, A, Sarkar, R & Rajagopalan, J (2017). Grain rotations in ultrafine-grained aluminum films studied using in situ TEM straining with automated crystallogrphic orientation mapping. Mater Des 113, 186–194.CrossRefGoogle Scholar
Kacher, J, Xie, Y, Viogt, SP, Zhu, S, Yuchi, H, Key, J & Kalidindi, SR (2021). In situ transmission electron microscopy: Signal processing challenges and examples. IEEE Signal Process Mag 39, 89–113.CrossRefGoogle Scholar
Kirkland, EJ (2020). Advanced Computing in Electron Microscopy, 3rd ed. New York: Springer Science & Business Media.CrossRefGoogle Scholar
Lee, DD & Seung, HS (1999). Learning the parts of objects by non-negative matrix factorization. Nature 401, 788–791.CrossRefGoogle ScholarPubMed
Londono-Calderon, A, Williams, DJ, Schneider, MM, Savitzsky, BH, Ophus, C, Ma, S, Zhu, H & Pettes, MT (2021). Intrinsic helical twist and chirality in ulrathin tellurium nanowires. Nanoscale 13, 9606–9614.CrossRefGoogle ScholarPubMed
Ma, J, Jiang, X, Fan, A, Jiang, J & Yan, J (2020). Image matching from handcrafted to deep features: A survey. Int J Comput Vis 129, 23–79.CrossRefGoogle Scholar
Mahr, C, Muller-Caspary, K, Grieb, T, Krause, FF, Schowalter, M & Rosenauer, A (2021). Accurate measurement of strain at interfaces in 4D-STEM: A comparison of various methods. Ultramicroscopy 221, 113196.CrossRefGoogle ScholarPubMed
Martineau, BH, Johnstone, DJ, van Helvoort, AT, Midgley, PA & Eggeman, AS (2019). Unsupervised machine learning applied to scanning precession electron diffraction data. Adv Struct Chem Imag 5, 1–14.CrossRefGoogle Scholar
Mehta, AN, Gauquelin, N, Nord, M, Orekhov, A, Bender, H, Cerbu, D, Verbeeck, J & Vandervorst, W (2020). Unravelling stacking order in epitaxial bilayer mx2 using 4D-STEM with unsupervised learning. Nanotechnology 31, 445702.CrossRefGoogle Scholar
Meng, T & Zuo, J-M (2017). Improvements in electron diffraction pattern automatic indexing algorithms. Eur Phys J Appl Phys 80, 107901.CrossRefGoogle Scholar
Mu, X, Chen, L, Mikut, R, Hahn, H & Kubel, C (2021). Unveiling local atomic bonding and packing of amorphous nanophases via independent component analysis facilitated pair distribution function. Acta Mater 212, 116932.CrossRefGoogle Scholar
Mukherjee, D, Gamler, JTL, Skrabalak, SE & Unocic, RR (2020). Lattice strain measurement of core@shell electrocatalysts with 4D scanning transmission electron microscopy nanobeam electron diffraction. ACS Catal 10, 5529–5541.CrossRefGoogle Scholar
Munshi, J, Rakowski, A, Savitsky, BH, Zeltmann, SE, Ciston, J, Henderson, M, Cholia, S, Minor, AM, Chan, MK & Ophus, C (2022). Disentangling multiple scattering with deep learning: application to strain mapping from electron diffraction patterns. Preprint arXiv:2202.00204.Google Scholar
Negishi, Y, Nakazaki, T, Malola, S, Takano, S, Niihori, Y, Kurashige, W, Yamazoe, S, Tsukuda, T & Hakkinen, H (2014). A critical size for emergence of nonbulk electronic and geometric structures in dodecanethiolate-protected Au clusters. J Am Chem Soc 137, 1206–1212.CrossRefGoogle Scholar
Ophus, C (2017). A fast image simulation algorithm for scanning transmission electron microscopy. Adv Struct Chem Imag 3, 1–11.CrossRefGoogle ScholarPubMed
Ophus, C (2019). Four-dimensional scanning transmission electron microscopy (4D-STEM): From scanning nanodiffraction to ptychography and beyond. Microsc Microanal 25, 563–582.CrossRefGoogle ScholarPubMed
Ophus, C, Zeltmann, SE, Bruefach, A, Rakowski, A, Savitzsky, BH, Minor, AM & Scott, M (2021). Automated crystal orientation mapping in py4dSTEM using sparse correlation matching. Microsc Microanal 28, 390–403.CrossRefGoogle Scholar
Panova, O, Chen, XC, Bustillo, KC, Ophus, C, Bhatt, M, Balsara, N & Minor, AM (2016). Orientation mapping of semicrystalline polymers using scanning electron nanobeam diffraction. Micron 88, 30–36.CrossRefGoogle ScholarPubMed
Panova, O, Ophus, C, Takacs, CJ, Bustillo, KC, Balhorn, L, Salleo, A, Balsara, N & Minor, AM (2019). Diffraction imaging of nanocrystalline structures in organic semiconductor thin films. Nat Mater 18, 860–865.CrossRefGoogle Scholar
Pedregosa, F, Varoquaux, G, Gramfort, A, Michel, V, Thirion, B, Grisel, O, Blondel, M, Prettenhofer, P, Weiss, R, Dubourg, V, Vanderplas, J, Passos, A, Cournapeau, D, Brucher, M, Perrot, M & Duchesnay, E (2011). Scikit-learn: Machine learning in Python. J Mach Learn Res 12, 2825–2830.Google Scholar
Pekin, TC, Ding, J, Gammer, C, Ozdol, B, Ophus, C, Asta, M, Ritchie, RO & Minor, AM (2019). Direct measurement of nanostructural change during in situ deformation of a bulk metallic glass. Nat Commun 10, 1–7.CrossRefGoogle ScholarPubMed
Pekin, TC, Gammer, C, Ciston, J, Minor, AM & Ophus, C (2017). Optimizing disk registration algorithms for nanobeam electron diffraction strain mapping. Ultramicroscopy 176, 170–176.CrossRefGoogle ScholarPubMed
Pelz, PM, Johnson, I, Ophus, C, Ercius, P & Scott, MC (2022). Real-time interactive 4D-STEM phase contrast imaging from electron event representation data: Less computation with the right representation. IEEE Signal Process Mag 39, 25–31.CrossRefGoogle Scholar
Ponce, A, Aguilar, JA, Tate, J & Jose Yacaman, M (2021). Advances in the electron diffraction characterization of atomic clusters and nanoparticles. Nanoscale Adv 3, 311–325.CrossRefGoogle ScholarPubMed
Rauch, EF & Veron, M (2019). Methods for orientation and phase identification of nano-sized embedded secondary phase particles by 4D scanning precession electron diffraction. Acta Crystallogr B: Struct Sci, Cryst Eng Mater 75, 505–511.CrossRefGoogle ScholarPubMed
Rauch, ER, Portillo, J, Nicolopoulos, S, Bultreys, D, Rouvimov, S & Moeck, P (2010). Automated nanocrystal orientation and phase mapping in the transmission electron microscope on the basis of precession electron diffraction. Z Kristallogr Cryst Mater 225, 103–109.CrossRefGoogle Scholar
Roy, D, Sing, SS, Mitra, R, Rosinski, M, Michalski, A, Lojkowski, W & Manna, I (2009). Synthesis and characterization of precipitation hardened amorphous matrix composite by mechanical alloying and pulse plasma sintering of Al65Cu20Ti15. Philos Mag 89, 1051–1061.CrossRefGoogle Scholar
Savitzky, BH, Zeltmann, SE, Hughes, LA, Brown, HG, Zhao, S, Pelz, PM, Pekin, TC, Barnard, ES, Rangel DaCosta, L, Kennedy, E, Xie, Y, Janish, MT, Schneider, MM, Herring, P, Gopal, C, Anapolsky, A, Dhall, R, Bustillo, KC, Ercius, P, Scott, MC, Ciston, H, Minor, AM & Ophus, C (2021). py4DSTEM: A software package for four-dimensional scanning transmission electron microscopy data analysis. Microsc Microanal 27, 1–32.CrossRefGoogle ScholarPubMed
Shi, C, Cao, M, Muller, D & Han, Y (2021). Rapid and semi-automated analysis of 4D-STEM data via unsupervised learning. Microsc Microanal Conf Proc 27, 58–59.CrossRefGoogle Scholar
Shukla, AK, Ramasse, QM, Ophus, C, Kepaptsoglou, DM, Hage, FS, Gammer, C, Bowling, C, Gallegos, PAH & Venkatachalam, S (2018). Effect of composition on the structure of lithium- and manganese-rich transition metal oxides. Energy Environ Sci 11, 830–840.CrossRefGoogle Scholar
Thati, SK, Ding, J, Zhang, D & Hu, XH (2015). Feature selection and analysis of diffraction images. IEEE International Conference on Software Quality, Reliability and Security - Companion (QRS) 2017.CrossRefGoogle Scholar
Thornsen, E, Frafjord, J, Friis, J, Marioara, C, Wenner, S, Andersen, S & Holmestad, R (2021). Studying GPI zones in Al-Zn-Mg alloys by 4D-STEM. Mater Charact, 111675.Google Scholar
Treacy, MMJ & Gibson, JM (1996). Variable coherence microscopy: A rich source of structural information from disordered materials. Acta Crystallogr A 52, 212–220.CrossRefGoogle Scholar
Uesugi, F, Koshiya, S, Kikkawa, J, Nagai, T, Mitsuishi, K & Kimoto, K (2021). Non-negative matrix factorization for mining big data using four-dimensional scanning transmission electron microscopy. Ultramicroscopy 221, 113168.CrossRefGoogle ScholarPubMed
van der Walt, S, Schonberger, JL, Nunez-Iglesias, J, Boulgne, F, Warner, JD, Yager, N, Gouilart, E, Yu, T & the scikit-image contributors (2014). scikit-image: Image processing in Python. PeerJ 2, e453.CrossRefGoogle ScholarPubMed
Velazquez-Salazar, JJ, Esparza, R, Mejia-Rosales, SJ, Estrada-Salas, R, Ponce, A, Deepak, FL, Castro-Guerrero, C & Jose-Yacaman, M (2011). Experimental evidence of icosahedral and decahedral packing in one-dimensional nanostructures. ACS Nano 5, 6272–6278.CrossRefGoogle ScholarPubMed
Walton, W (1948). Feret's statistical diameter as a measure of particle size. Nature 162, 329–330.CrossRefGoogle Scholar
Wang, Q-L, Fang, R, He, L-L, Feng, J-J, Yuan, J & Wang, A-J (2016). Bimetallic PdAu alloyed nanowires: Rapid synthesis via oriented attachment growth and their high electrocatalytic activity for methanol oxidation reaction. J Alloys Compd 684, 379–388.CrossRefGoogle Scholar
Wang, Y, Choi, S-II, Zhao, X, Peng, H-C, Chi, M, Huan, CZ & Xia, Y (2014). Polyol synthesis of ultrathin Pd nanowires via attachment based growth and their enhanced activity towards formic acid oxidation. Adv Funct Mater 21, 131–139.CrossRefGoogle Scholar
Wang, Y, Wang, Q, Sun, H, Zhang, W, Chen, G, Wang, Y, Shen, X, Han, Y, Lu, X & Chen, H (2011). Chiral transformation: From single nanowire to double helix. J Am Chem Soc 133, 20060–20063.CrossRefGoogle Scholar
Yang, N, Ophus, C, Savitzsky, BH, Scott, MC, Bustillo, K & Lu, K (2021). Nanoscale characterization of crystalline and amorphous phases in silicon oxycarbide ceramics using 4D-STEM. Mater Charact 181, 1111512.CrossRefGoogle Scholar
Yao, T, Sun, Z, Li, Y, Pan, Z, Wei, H, Xie, Y, Nomura, M, Niwa, Y, Yan, W, Wu, Z, Jiang, Y, Liu, Q & Wei, S (2010). Insights into initial kinetic nucleation of gold nanocrystals. J Am Chem Soc 132, 7696–7701.CrossRefGoogle ScholarPubMed
Yuan, R, Zhang, J, He, L & Zuo, J-M (2021). Training artificial neural networks for precision orientation and strain mapping using 4D electron diffraction datasets. Ultramicroscopy 231, 113256.CrossRefGoogle ScholarPubMed
Zeltmann, SE, Muller, A, Bustillo, KC, Savitsky, B, Hughes, L, Minor, AM & Ophus, C (2020). Patterned probes for high precision 4D-STEM Bragg measurements. Ultramicroscopy 209, 112890.CrossRefGoogle ScholarPubMed
Zintler, A, Eilhardt, R, Wang, S, Krajnak, M, Schramowski, P, Stammer, W, Petzold, S, Kaiser, N, Kersting, K, Alff, L & Molina-Luna, L (2020). Machine learning assisted pattern matching: Insight into oxide electronic device performance by phase determination in 4D-STEM datasets. Microsc Microanal Conf Proc 26, 1908–1909.CrossRefGoogle Scholar
Supplementary material: PDF

Bruefach et al. supplementary material

Bruefach et al. supplementary material

Download Bruefach et al. supplementary material(PDF)
PDF 3.6 MB