The objective of this project is to explore ways to apply machine learning to TEM video data, for the purpose of identifying and characterizing materials defects, inclusions, or other features.
Sponsor
Penn State Department of Engineering Science and Mechanics
Team Members
Madeline Cannon Arpit Singla Qifan Yang Jack Jenny Joshua Haines Shuhao Xu Weipeng Hu
Project Poster
Click on any image to enlarge.
Project Video
Project Summary
Overview
TEM imaging is an important method of understanding material defects and thus material performance. However, the process of identifying and classifying defects manually is highly time-consuming, since TEM images can only cover very small sections of the material at a time, and have many various types of possible defects. Machine learning can be used to expedite this process. Convolutional neural networks (CNN) are able to classify TEM images of material defects at a much faster rate than human experts, and with suitable accuracy.
Objectives
– Manually identify and label defects in TEM images provided by sponsor via graphics editors like Krita or Photoshop.
– Use labeled images to train machine learning models to identify multiple types of defects in TEM images.
– Use rod detection to individually label and track defects for position, length, and speed to learn about defect motion.
Approach
The first step in this effort was to examine and learn the “state-of-the-art” of algorithmic TEM analysis research. This was accomplished via a literature review of recent publications by various authors to narrow down options for machine learning algorithms.
After reviewing literature, we developed our own algorithm inspired by those in the literature, settling on an implementation of the U-Net convolutional neural network (CNN). This algorithm was to be specially configured to detect and characterize line dislocations.
In parallel with this, we collected sample TEM video data to be used to test and train the algorithm. From the videos, we extracted each frame as a separate image file, manually labeling some with a binary mask to be used as training data.
To increase the volume of training data, a short script was written to save multiple rotated and flipped copies of the labeled images. This is common practice for training image processing algorithms.
In November, sponsors communicated that the team should also try configuring the algorithm to detect and characterize radiation bubbles, so the algorithm would have to be tweaked to accommodate this.
Outcomes
– Less time consuming to label data
– Higher accuracy than human experts in identifying defects
– Scientists can focus on other tasks rather than the menial labor associated with TEM image analysis